将 Bedrock Agent 用作支持追踪的 MLflow ChatModel

在这篇博客文章中，我们将深入探讨如何将 AWS Bedrock Agent 作为 ChatModel 集成到 MLflow 中，重点介绍如何利用 Bedrock 的操作组和知识库来构建对话式 AI 应用程序。本博客将指导您设置 Bedrock Agent，配置操作组以通过 Lambda 启用自定义操作，并利用知识库进行上下文感知交互。我们将特别强调在 MLflow 中实现追踪。通过阅读本文，您将对如何将 AWS Bedrock 的高级功能与 MLflow 的功能（例如代理请求追踪、模型追踪和输入示例的一致签名）结合使用有很好的理解。

什么是 AWS Bedrock？

Amazon Bedrock 是 AWS 提供的一项托管服务，可简化生成式 AI 应用程序的开发。它通过单个 API 提供对来自领先 AI 提供商的各种基础模型 (FM) 的访问，使开发人员能够安全高效地构建和扩展 AI 解决方案。

与此集成相关的关键组件

Bedrock Agent：从高层次看，Bedrock Agent 是 Bedrock 中的一个抽象概念，它由基础模型、操作组和知识库组成。

操作组：这些是可自定义的操作集，定义了 Bedrock Agent 可以执行的任务。操作组由 OpenAPI 模式和用于执行工具调用的相应 Lambda 函数组成。OpenAPI 模式用于定义代理可调用以完成任务的 API。

知识库：Amazon Bedrock 支持创建知识库以实现检索增强生成工作流。它由数据源（在 S3 或网页上）和包含对这些数据嵌入引用的向量存储组成。

Bedrock 的代理执行过程和代理检测的相应追踪分组如下：

预处理：此步骤验证、上下文化和分类用户输入。

编排：此步骤处理用户输入的解释，决定何时执行以及执行哪些任务，并迭代地完善响应。

后处理（可选）：此步骤在返回给用户之前格式化最终响应。

追踪：上述每个步骤都有一个执行追踪，其中包括代理响应的每个步骤中的理由、操作、查询和观察。这包括操作组和知识库查询的输入和输出。

我们将在下面详细查看这些追踪。

MLflow 中的 ChatModel 是什么？

ChatModel 类专门设计用于更容易地实现与流行的 LLM 聊天 API 兼容的模型。它使您能够无缝地引入自己的模型或代理，并利用 MLflow 的功能，即使这些模型在 MLflow 中不原生支持作为一种风格。此外，它提供了默认签名，这些签名对于 ChatModel 是静态的，与 PythonModel 不同。

在以下部分中，我们将使用 ChatModel 封装 Bedrock Agent。

有关 ChatModel 的更多详细信息，您可以在 MLflow 文档中此处和此处阅读

使用操作组设置 AWS Bedrock Agent

在本节中，我们将部署 Bedrock Agent 的所有组件，以便我们可以在 MLflow 中将其作为 ChatModel 调用。

先决条件

您需要设置以下项目（通过 AWS 控制台或 SDK）

• 为代理和 Lambda 函数设置角色。示例
• 创建/部署代理。示例
  • 重要提示：在此处保存代理 ID，我们稍后将需要它。
• 创建 Lambda 函数。示例
• 配置代理与 Lambda 交互的 IAM 权限。示例和示例
• 创建操作组以链接代理和 Lambda。示例
  • 重要提示：保存在此处保存代理别名 ID，我们稍后将需要它。
• 使用别名部署 Bedrock 代理。示例

在我们的案例中，我们将部署以下示例操作组，该操作组根据航天器的质量和比冲计算从地球到火星霍曼转移的下一个最佳出发日期。

操作组的 OpenAPI 模式

如上所述，以下是我们示例操作组的 OpenAPI 模式

openapi: 3.0.0
info:
  title: Time API
  version: 1.0.0
  description: API to get the next optimal departure date for a Hohmann transfer from Earth to Mars.
paths:
  /get-next-mars-launch-window:
    get:
      summary: Gets the next optimal launch window to Mars.
      description: Gets the next optimal launch window to Mars.
      operationId: getNextMarsLaunchWindow
      parameters:
        - name: total_mass
          in: query
          description: Total mass of the spacecraft including fuel (kg)
          required: true
          schema:
            type: string
        - name: dry_mass
          in: query
          description: Mass of the spacecraft without fuel (kg).
          required: true
          schema:
            type: string
        - name: specific_impulse
          in: query
          description: Specific impulse of the propulsion system (s).
          required: true
          schema:
            type: string
      responses:
        "200":
          description: The next optimal departure date for a Hohmann transfer from Earth to Mars, based on the spacecraft's mass and specific impulse.
          content:
            "application/json":
              schema:
                type: object
                properties:
                  next_launch_window:
                    type: string
                    description: Next Mars Launch Window

操作组 - Lambda 函数

以下是操作组示例 Lambda 的代码部署

import json
import math
from datetime import datetime, timedelta


def lambda_handler(event, context):
    def _calculate_optimal_departure_window(
        total_mass, dry_mass, specific_impulse
    ):
        """
        Calculate the next optimal departure date for a Hohmann transfer from Earth to Mars,
        based on the spacecraft's mass and specific impulse.

        Parameters:
        - total_mass (float): Total mass of the spacecraft including fuel (kg).
        - dry_mass (float): Mass of the spacecraft without fuel (kg).
        - specific_impulse (float): Specific impulse of the propulsion system (s).

        Returns:
        - dict: {
            'next_launch_date': datetime,
            'synodic_period_days': float,
            'transfer_time_days': float,
            'delta_v_available_m_s': float,
            'delta_v_required_m_s': float,
            'is_feasible': bool
          }
        """
        current_date = None
        # Constants
        G0 = 9.80665  # m/s^2, standard gravity
        MU_SUN = (
            1.32712440018e20  # m^3/s^2, standard gravitational parameter for the Sun
        )
        AU = 1.496e11  # meters, astronomical unit
        EARTH_ORBITAL_PERIOD = 365.25  # days
        MARS_ORBITAL_PERIOD = 686.98  # days
        SYNODIC_PERIOD = 1 / abs((1 / EARTH_ORBITAL_PERIOD) - (1 / MARS_ORBITAL_PERIOD))
        TRANSFER_TIME = 259  # days, approximate duration of Hohmann transfer
        BASE_LAUNCH_DATE = datetime(2020, 7, 1)  # A reference past launch window date

        # Orbital Radii (assuming circular orbits for simplicity)
        r1 = AU  # Earth's orbital radius in meters
        r2 = 1.524 * AU  # Mars' orbital radius in meters

        # Calculate Required Delta-V for Hohmann Transfer
        # Using vis-viva equation for Hohmann transfer
        def calculate_hohmann_delta_v(mu, r_start, r_end):
            # Velocity of departure orbit (Earth)
            v_start = math.sqrt(mu / r_start)
            # Velocity of transfer orbit at departure
            a_transfer = (r_start + r_end) / 2
            v_transfer_start = math.sqrt(mu * (2 / r_start - 1 / a_transfer))
            delta_v1 = v_transfer_start - v_start

            # Velocity of arrival orbit (Mars)
            v_end = math.sqrt(mu / r_end)
            # Velocity of transfer orbit at arrival
            v_transfer_end = math.sqrt(mu * (2 / r_end - 1 / a_transfer))
            delta_v2 = v_end - v_transfer_end

            return delta_v1, delta_v2

        delta_v1, delta_v2 = calculate_hohmann_delta_v(MU_SUN, r1, r2)
        delta_v_required = abs(delta_v1) + abs(delta_v2)  # Total delta-v in m/s

        # Delta-V using Tsiolkovsky Rocket Equation
        if dry_mass <= 0 or total_mass <= dry_mass:
            raise ValueError("Total mass must be greater than dry mass.")

        delta_v_available = (
            specific_impulse * G0 * math.log(total_mass / dry_mass)
        )  # m/s

        is_feasible = delta_v_available >= delta_v_required

        if current_date is None:
            current_date = datetime.now()

        days_since_base = (current_date - BASE_LAUNCH_DATE).days
        if days_since_base < 0:
            # Current date is before the base launch date
            next_launch_date = BASE_LAUNCH_DATE
        else:
            synodic_periods_passed = days_since_base / SYNODIC_PERIOD
            synodic_periods_passed_int = math.floor(synodic_periods_passed)
            next_launch_date = BASE_LAUNCH_DATE + timedelta(
                days=(synodic_periods_passed_int + 1) * SYNODIC_PERIOD
            )

        next_launch_date = next_launch_date.replace(
            hour=0, minute=0, second=0, microsecond=0
        )

        return {
            "next_launch_date": next_launch_date,
            "synodic_period_days": SYNODIC_PERIOD,
            "transfer_time_days": TRANSFER_TIME,
            "delta_v_available_m_s": delta_v_available,
            "delta_v_required_m_s": delta_v_required,
            "is_feasible": is_feasible,
        }

    query_params = {
        event["name"]: event["value"] for event in event.get("parameters", [])
    }

    total_mass = float(query_params.get("total_mass"))
    dry_mass = float(query_params.get("dry_mass"))
    specific_impulse = float(query_params.get("specific_impulse"))

    response = {
        "next_launch_window": _calculate_optimal_departure_window(
            total_mass, dry_mass, specific_impulse
        )
    }

    response_body = {"application/json": {"body": json.dumps(response)}}

    action_response = {
        "actionGroup": event["actionGroup"],
        "apiPath": event["apiPath"],
        "httpMethod": event["httpMethod"],
        "httpStatusCode": 200,
        "responseBody": response_body,
    }

    session_attributes = event["sessionAttributes"]
    prompt_session_attributes = event["promptSessionAttributes"]

    return {
        "messageVersion": "1.0",
        "response": action_response,
        "sessionAttributes": session_attributes,
        "promptSessionAttributes": prompt_session_attributes,
    }

接下来，我们将把 Bedrock 代理封装为 ChatModel，以便我们可以注册并加载它进行推理。

为 Bedrock 代理编写 ChatModel

以下是在 Python 3.12.7 中本地运行以下示例所使用的顶级包：

boto3==1.35.31
mlflow==2.16.2

将 Bedrock Agent 实现为带有追踪的 MLflow ChatModel

import copy
import os
import uuid
from typing import List, Optional

import boto3
import mlflow
from botocore.config import Config
from mlflow.entities import SpanType
from mlflow.pyfunc import ChatModel
from mlflow.types.llm import ChatResponse, ChatMessage, ChatParams, ChatChoice


class BedrockModel(ChatModel):
    def __init__(self):
        """
        Initializes the BedrockModel instance with placeholder values.

        Note:
            The `load_context` method cannot create new instance variables; it can only modify existing ones.
            Therefore, all instance variables should be defined in the `__init__` method with placeholder values.
        """
        self.brt = None
        self._main_bedrock_agent = None
        self._bedrock_agent_id = None
        self._bedrock_agent_alias_id = None
        self._inference_configuration = None
        self._agent_instruction = None
        self._model = None
        self._aws_region = None

    def __getstate__(self):
        """
        Prepares the instance state for pickling.

        This method is needed because the `boto3` client (`self.brt`) cannot be pickled.
        By excluding `self.brt` from the state, we ensure that the model can be serialized and deserialized properly.
        """
        # Create a dictionary of the instance's state, excluding the boto3 client
        state = self.__dict__.copy()
        del state["brt"]
        return state

    def __setstate__(self, state):
        """
        Restores the instance state during unpickling.

        This method is needed to reinitialize the `boto3` client (`self.brt`) after the instance is unpickled,
        because the client was excluded during pickling.
        """
        self.__dict__.update(state)
        self.brt = None

    def load_context(self, context):
        """
        Initializes the Bedrock client with AWS credentials.

        Args:
            context: The MLflow context containing model configuration.

        Note:
            Dependent secret variables must be in the execution environment prior to loading the model;
            else they will not be available during model initialization.
        """
        self._main_bedrock_agent = context.model_config.get("agents", {}).get(
            "main", {}
        )
        self._bedrock_agent_id = self._main_bedrock_agent.get("bedrock_agent_id")
        self._bedrock_agent_alias_id = self._main_bedrock_agent.get(
            "bedrock_agent_alias_id"
        )
        self._inference_configuration = self._main_bedrock_agent.get(
            "inference_configuration"
        )
        self._agent_instruction = self._main_bedrock_agent.get("instruction")
        self._model = self._main_bedrock_agent.get("model")
        self._aws_region = self._main_bedrock_agent.get("aws_region")

        # Initialize the Bedrock client
        self.brt = boto3.client(
            service_name="bedrock-agent-runtime",
            config=Config(region_name=self._aws_region),
            aws_access_key_id=os.environ["AWS_ACCESS_KEY"],
            aws_secret_access_key=os.environ["AWS_SECRET_ACCESS_KEY"],
            aws_session_token=os.environ["AWS_SESSION_TOKEN"],
            region_name=self._aws_region,
        )

    @staticmethod
    def _extract_trace_groups(events):
        """
        Extracts trace groups from a list of events based on their trace IDs.

        Args:
            events (list): A list of event dictionaries.

        Returns:
            dict: A dictionary where keys are trace IDs and values are lists of trace items.
        """
        from collections import defaultdict

        trace_groups = defaultdict(list)

        def find_trace_ids(obj, original_trace, depth=0, parent_key=None):
            if depth > 5:
                return  # Stop recursion after 5 levels if no traceId has been found
            if isinstance(obj, dict):
                trace_id = obj.get("traceId")
                if trace_id:
                    # Include the parent key as the 'type'
                    item = {
                        "type": parent_key,
                        "data": obj,
                        "event_order": original_trace.get("trace", {}).get(
                            "event_order"
                        ),
                    }
                    trace_groups[trace_id].append(item)
                else:
                    for key, value in obj.items():
                        find_trace_ids(
                            value, original_trace, depth=depth + 1, parent_key=key
                        )
            elif isinstance(obj, list):
                for item in obj:
                    find_trace_ids(item, item, depth=depth + 1, parent_key=parent_key)

        find_trace_ids(events, {})
        return dict(trace_groups)

    @staticmethod
    def _get_final_response_with_trace(trace_id_groups: dict[str, list[dict]]):
        """
        Processes trace groups to extract the final response and create relevant MLflow spans.

        Args:
            trace_id_groups (dict): A dictionary of trace groups keyed by trace IDs.

        Returns:
            str: The final response text extracted from the trace groups.
        """
        trace_id_groups_copy = copy.deepcopy(trace_id_groups)
        model_invocation_input_key = "modelInvocationInput"

        def _create_trace_by_type(
            trace_name, _trace_id, context_input, optional_rationale_subtrace
        ):
            @mlflow.trace(
                name=trace_name,
                attributes={"trace_attributes": trace_id_groups[_trace_id]},
            )
            def _trace_agent_pre_context(inner_input_trace):
                return optional_rationale_subtrace.get("data", {}).get("text")

            trace_id_groups_copy[_trace_id].remove(context_input)
            _trace_agent_pre_context(context_input.get("data", {}).get("text"))

        def _extract_action_group_trace(
            _trace_id, trace_group, action_group_invocation_input: dict
        ):
            @mlflow.trace(
                name="action-group-invocation",
                attributes={"trace_attributes": trace_id_groups[_trace_id]},
            )
            def _action_group_trace(inner_trace_group):
                for _trace in trace_group:
                    action_group_invocation_output = _trace.get("data", {}).get(
                        "actionGroupInvocationOutput"
                    )
                    if action_group_invocation_output is not None:
                        action_group_response = str(
                            {
                                "action_group_name": action_group_invocation_input.get(
                                    "actionGroupName"
                                ),
                                "api_path": action_group_invocation_input.get(
                                    "apiPath"
                                ),
                                "execution_type": action_group_invocation_input.get(
                                    "executionType"
                                ),
                                "execution_output": action_group_invocation_output.get(
                                    "text"
                                ),
                            }
                        )
                        trace_group.remove(_trace)
                        return action_group_response

            _action_group_trace(str(action_group_invocation_input))

        def _extract_knowledge_base_trace(
            _trace_id, trace_group, knowledge_base_lookup_input
        ):
            @mlflow.trace(
                name="knowledge-base-lookup",
                attributes={"trace_attributes": trace_id_groups[_trace_id]},
            )
            def _knowledge_base_trace(inner_trace_group):
                for _trace in trace_group:
                    knowledge_base_lookup_output = _trace.get("data", {}).get(
                        "knowledgeBaseLookupOutput"
                    )
                    if knowledge_base_lookup_output is not None:
                        knowledge_base_response = str(
                            {
                                "knowledge_base_id": knowledge_base_lookup_input.get(
                                    "knowledgeBaseId"
                                ),
                                "text": knowledge_base_lookup_input.get("text"),
                                "retrieved_references": knowledge_base_lookup_output.get(
                                    "retrievedReferences"
                                ),
                            }
                        )
                        trace_group.remove(_trace)
                        return knowledge_base_response

            _knowledge_base_trace(str(trace_group))

        def _trace_group_type(
            _trace_id, trace_group, _trace, optional_rationale_subtrace
        ):
            trace_name = "observation"
            pre_processing_trace_id_suffix = "-pre"
            if pre_processing_trace_id_suffix in _trace_id:
                trace_name = "agent-initial-context"
            else:
                for _inner_trace in trace_group:
                    action_group_invocation_input = _inner_trace.get("data", {}).get(
                        "actionGroupInvocationInput"
                    )
                    if action_group_invocation_input is not None:
                        action_group_name = action_group_invocation_input.get(
                            "actionGroupName"
                        )
                        trace_name = f"ACTION-GROUP-{action_group_name}"
                        _create_trace_by_type(
                            trace_name, _trace_id, _trace, optional_rationale_subtrace
                        )
                        _extract_action_group_trace(
                            _trace_id, trace_group, action_group_invocation_input
                        )
                        trace_group.remove(_trace)
                    knowledge_base_lookup_input = _inner_trace.get("data", {}).get(
                        "knowledgeBaseLookupInput"
                    )
                    if knowledge_base_lookup_input is not None:
                        knowledge_base_id = knowledge_base_lookup_input.get(
                            "knowledgeBaseId"
                        )
                        trace_name = f"KNOWLEDGE_BASE_{knowledge_base_id}"
                        _create_trace_by_type(
                            trace_name, _trace_id, _trace, optional_rationale_subtrace
                        )
                        _extract_knowledge_base_trace(
                            _trace_id, trace_group, knowledge_base_lookup_input
                        )
                        trace_group.remove(_trace)
            return trace_name

        for _trace_id, _trace_group in trace_id_groups_copy.items():
            trace_group = sorted(_trace_group, key=lambda tg: tg["event_order"])
            model_invocation_input_subtrace = None
            optional_rationale_subtrace = None
            for _trace in _trace_group:
                if model_invocation_input_key == _trace.get("type", ""):
                    model_invocation_input_subtrace = _trace
                elif "rationale" == _trace.get("type", ""):
                    optional_rationale_subtrace = _trace
            _trace_group_type(
                _trace_id,
                trace_group,
                model_invocation_input_subtrace,
                optional_rationale_subtrace,
            )

        final_response = (
            list(trace_id_groups_copy.values())[-1][-1]
            .get("data", {})
            .get("finalResponse", {})
            .get("text")
        )
        return final_response

    @mlflow.trace(name="Bedrock Input Prompt")
    def _get_agent_prompt(self, raw_input_question):
        """
        Constructs the agent prompt by combining the input question and the agent instruction.

        Args:
            raw_input_question (str): The user's input question.

        Returns:
            str: The formatted agent prompt.
        """
        return f"""
        Answer the following question and pay strong attention to the prompt:
        <question>
        {raw_input_question}
        </question>
        <instruction>
        {self._agent_instruction}
        </instruction>
        """

    @mlflow.trace(name="bedrock-agent", span_type=SpanType.CHAT_MODEL)
    def predict(
        self, context, messages: List[ChatMessage], params: Optional[ChatParams]
    ) -> ChatResponse:
        """
        Makes a prediction using the Bedrock agent and processes the response.

        Args:
            context: The MLflow context.
            messages (List[ChatMessage]): A list of chat messages.
            params (Optional[ChatParams]): Optional parameters for the chat.

        Returns:
            ChatResponse: The response from the Bedrock agent.
        """
        formatted_input = messages[-1].content
        session_id = uuid.uuid4().hex

        response = self.brt.invoke_agent(
            agentId=self._bedrock_agent_id,
            agentAliasId=self._bedrock_agent_alias_id,
            inputText=self._get_agent_prompt(formatted_input),
            enableTrace=True,
            sessionId=session_id,
            endSession=False,
        )

        # Since this provider's output doesn't match the OpenAI specification,
        # we need to go through the returned trace data and map it appropriately
        # to create the MLflow span object.
        events = []
        for index, event in enumerate(response.get("completion", [])):
            if "trace" in event:
                event["trace"]["event_order"] = index
            events.append(event)
        trace_id_groups = self._extract_trace_groups(events)
        final_response = self._get_final_response_with_trace(trace_id_groups)
        with mlflow.start_span(
            name="retrieved-response", span_type=SpanType.AGENT
        ) as span:
            span.set_inputs(messages)
            span.set_attributes({})

            output = ChatResponse(
                choices=[
                    ChatChoice(
                        index=0,
                        message=ChatMessage(role="user", content=final_response),
                    )
                ],
                usage={},
                model=self._model,
            )

            span.set_outputs(output)

        return output

以下是关于此 BedrockModel 实现的一些重要说明

• AWS 访问密钥 ID、秘密密钥和会话令牌在此处外部化。这些必须在环境中存在，然后我们才能运行推理。您需要为您的 IAM 用户生成它们并将它们设置为环境变量。

aws sts get-session-token --duration-seconds 3600

然后设置以下内容

import os

os.environ['AWS_ACCESS_KEY'] = "<AccessKeyId>"
os.environ['AWS_SECRET_ACCESS_KEY'] = "<SecretAccessKey>"
os.environ['AWS_SESSION_TOKEN'] = "<SessionToken>"

如上述代码所示，这些不会随模型一起记录，而只在 load_context 中设置。此方法在构建 ChatModel 时调用。更多详细信息请见此处

• Bedrock 代理 ID 和代理别名 ID 通过我们将在下面使用的 model_config 传递。

• boto3 模块已被排除在 pickle 之外。这是通过 __getstate__ 和 __setstate__ 完成的，我们分别排除和重置它

记录和加载 BedrockModel

import mlflow
from mlflow.models import infer_signature

input_example = [
    {
        "messages": [
            {
                "role": "user",
                "content": "When is the next launch window for Mars?",
            }
        ]
    }
]

output_example = {
    "choices": [
        {
            "index": 0,
            "finish_reason": "stop",
            "message": {"role": "assistant", "content": "test content"},
        }
    ]
}
signature = infer_signature(input_example, output_example)

with mlflow.start_run():

    model_config = {
        "agents": {
            "main": {
                "model": "anthropic.claude-v2",
                "aws_region": "us-east-1",
                "bedrock_agent_id": "O9KQSEVEFF",
                "bedrock_agent_alias_id": "3WHEEJKNUT",
                "instruction": (
                    "You have functions available at your disposal to use when anwering any questions about orbital mechanics."
                    "if you can't find a function to answer a question about orbital mechanics, simply reply "
                    "'I do not know'"
                ),
                "inference_configuration": {
                    "temperature": 0.5,
                    "maximumLength": 2000,
                },
            },
        },
    }

    # Input example for the model
    input_example = {
        "messages": [
            {
                "role": "user",
                "content": "When is the next launch window for Mars? My spacecraft's total mass is 50000, dry mass is 10000 and specific impulse is 2500. Mass in Kg.",
            }
        ]
    }

    # Log and load the model using MLflow
    logged_chain_info = mlflow.pyfunc.log_model(
        python_model=BedrockModel(),
        model_config=model_config,
        artifact_path="chain",  # This string is used as the path inside the MLflow model where artifacts are stored
        input_example=input_example,  # Must be a valid input to your chain
    )

loaded = mlflow.pyfunc.load_model(logged_chain_info.model_uri)

# Predict using the loaded model
response = loaded.predict(
    {
        "messages": [
            {
                "role": "user",
                "content": "When is the next launch window for Mars? My spacecraft's total mass is 50000, dry mass is 10000 and specific impulse is 2500. Mass in Kg.",
            }
        ]
    }
)
print(response)

将 Bedrock Agent 追踪数据映射到 MLflow Span 对象

在此步骤中，我们需要遍历 Bedrock Agent 响应追踪中返回的数据，以提供相关的映射来创建 MLflow span 对象。AWS Bedrock Agent 的响应是一个扁平列表，追踪事件通过 traceId 连接。以下是 Bedrock Agent 响应中发送的原始追踪：

展开以查看 AWS Bedrock Agent 的原始追踪

[
  {
    'trace': {
      'agentAliasId': '3WHEEJKNUT',
      'agentId': 'O9KQSEVEFF',
      'agentVersion': '1',
      'event_order': 0,
      'sessionId': '9566a6d78551434fb0409578ffed63c1',
      'trace': {
        'preProcessingTrace': {
          'modelInvocationInput': {
            'inferenceConfiguration': {
              ...
            },
            'text': '\n\nHuman: You are a classifying agent that filters user inputs into categories. Your job is to sort these inputs before they...<thinking> XML tags before providing only the category letter to sort the input into within <category> XML tags.\n\nAssistant:',
            'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-pre-0',
            'type': 'PRE_PROCESSING'
          }
        }
      }
    }
  },
  {
    'trace': {
      'agentAliasId': '3WHEEJKNUT',
      'agentId': 'O9KQSEVEFF',
      'agentVersion': '1',
      'event_order': 1,
      'sessionId': '9566a6d78551434fb0409578ffed63c1',
      'trace': {
        'preProcessingTrace': {
          'modelInvocationOutput': {
            'parsedResponse': {
              ...
            },
            'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-pre-0'
          }
        }
      }
    }
  },
  {
    'trace': {
      'agentAliasId': '3WHEEJKNUT',
      'agentId': 'O9KQSEVEFF',
      'agentVersion': '1',
      'event_order': 2,
      'sessionId': '9566a6d78551434fb0409578ffed63c1',
      'trace': {
        'orchestrationTrace': {
          'modelInvocationInput': {
            'inferenceConfiguration': {
              ...
            },
            'text': '\n\nHuman:\nYou are a research assistant AI that has been equipped with one or more functions to help you answer a <question>...\n\nAssistant: <scratchpad> I understand I cannot use functions that have not been provided to me to answer this question.\n\n',
            'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-0',
            'type': 'ORCHESTRATION'
          }
        }
      }
    }
  },
  {
    'trace': {
      'agentAliasId': '3WHEEJKNUT',
      'agentId': 'O9KQSEVEFF',
      'agentVersion': '1',
      'event_order': 3,
      'sessionId': '9566a6d78551434fb0409578ffed63c1',
      'trace': {
        'orchestrationTrace': {
          'modelInvocationOutput': {
            'metadata': {
              ...
            },
            'rawResponse': {
              ...
            },
            'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-0'
          }
        }
      }
    }
  },
  {
    'trace': {
      'agentAliasId': '3WHEEJKNUT',
      'agentId': 'O9KQSEVEFF',
      'agentVersion': '1',
      'event_order': 4,
      'sessionId': '9566a6d78551434fb0409578ffed63c1',
      'trace': {
        'orchestrationTrace': {
          'rationale': {
            'text': 'To answer this question about the next Mars launch window, I will:\n\n1. Call the GET::optimal_departure_window_mars::getNext...lse values.\n\nI have verified that I have access to the GET::optimal_departure_window_mars::getNextMarsLaunchWindow function.',
            'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-0'
          }
        }
      }
    }
  },
  {
    'trace': {
      'agentAliasId': '3WHEEJKNUT',
      'agentId': 'O9KQSEVEFF',
      'agentVersion': '1',
      'event_order': 5,
      'sessionId': '9566a6d78551434fb0409578ffed63c1',
      'trace': {
        'orchestrationTrace': {
          'invocationInput': {
            'actionGroupInvocationInput': {
              ...
            },
            'invocationType': 'ACTION_GROUP',
            'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-0'
          }
        }
      }
    }
  },
  {
    'trace': {
      'agentAliasId': '3WHEEJKNUT',
      'agentId': 'O9KQSEVEFF',
      'agentVersion': '1',
      'event_order': 6,
      'sessionId': '9566a6d78551434fb0409578ffed63c1',
      'trace': {
        'orchestrationTrace': {
          'observation': {
            'actionGroupInvocationOutput': {
              ...
            },
            'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-0',
            'type': 'ACTION_GROUP'
          }
        }
      }
    }
  },
  {
    'trace': {
      'agentAliasId': '3WHEEJKNUT',
      'agentId': 'O9KQSEVEFF',
      'agentVersion': '1',
      'event_order': 7,
      'sessionId': '9566a6d78551434fb0409578ffed63c1',
      'trace': {
        'orchestrationTrace': {
          'modelInvocationInput': {
            'inferenceConfiguration': {
              ...
            },
            'text': '\n\nHuman:\nYou are a research assistant AI that has been equipped with one or more functions to help you answer a <question>...lta_v_available_m_s": 39457.985759929674, "delta_v_required_m_s": 5595.997417810693, "is_feasible": true}}</function_result>\n',
            'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-1',
            'type': 'ORCHESTRATION'
          }
        }
      }
    }
  },
  {
    'trace': {
      'agentAliasId': '3WHEEJKNUT',
      'agentId': 'O9KQSEVEFF',
      'agentVersion': '1',
      'event_order': 8,
      'sessionId': '9566a6d78551434fb0409578ffed63c1',
      'trace': {
        'orchestrationTrace': {
          'modelInvocationOutput': {
            'metadata': {
              ...
            },
            'rawResponse': {
              ...
            },
            'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-1'
          }
        }
      }
    }
  },
  {
    'trace': {
      'agentAliasId': '3WHEEJKNUT',
      'agentId': 'O9KQSEVEFF',
      'agentVersion': '1',
      'event_order': 9,
      'sessionId': '9566a6d78551434fb0409578ffed63c1',
      'trace': {
        'orchestrationTrace': {
          'observation': {
            'finalResponse': {
              ...
            },
            'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-1',
            'type': 'FINISH'
          }
        }
      }
    }
  },
  {
    'chunk': {
      'bytes': b
      'Based on the provided spacecraft dry mass of 10000 kg, total mass of 50000 kg, and specific impulse of 2500 s, the next optimal launch window for a Hohmann transfer from Earth to Mars is on November 26, 2026 UTC. The transfer will take 259 days.'
    }
  }
]

为了将这种结构适配到 MLflow 的 span 中，我们首先需要遍历原始响应追踪并按 traceId 对事件进行分组。在按 traceId 对追踪事件进行分组后，结构如下所示：

展开以查看按 traceId 分组的追踪

{
  'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-0': [
    {
      'data': {
        'inferenceConfiguration': {
          'maximumLength': 2048,
          'stopSequences': [
            '</function_call>',
            '</answer>',
            '</error>'
          ],
          'temperature': 0.0,
          'topK': 250,
          'topP': 1.0
        },
        'text': '\n\nHuman:\nYou are a research assistant AI that has been equipped with one or more functions to help you answer a <question>...\n\nAssistant: <scratchpad> I understand I cannot use functions that have not been provided to me to answer this question.\n\n',
        'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-0',
        'type': 'ORCHESTRATION'
      },
      'event_order': 2,
      'type': 'modelInvocationInput'
    },
    {
      'data': {
        'metadata': {
          'usage': {
            'inputTokens': 5160,
            'outputTokens': 135
          }
        },
        'rawResponse': {
          'content': 'To answer this question about the next Mars launch window, I will:\n\n1. Call the GET::optimal_departure_window_mars::getNext...l>\nGET::optimal_departure_window_mars::getNextMarsLaunchWindow(specific_impulse="2500", dry_mass="10000", total_mass="50000")'
        },
        'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-0'
      },
      'event_order': 3,
      'type': 'modelInvocationOutput'
    },
    {
      'data': {
        'text': 'To answer this question about the next Mars launch window, I will:\n\n1. Call the GET::optimal_departure_window_mars::getNext...lse values.\n\nI have verified that I have access to the GET::optimal_departure_window_mars::getNextMarsLaunchWindow function.',
        'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-0'
      },
      'event_order': 4,
      'type': 'rationale'
    },
    {
      'data': {
        'actionGroupInvocationInput': {
          'actionGroupName': 'optimal_departure_window_mars',
          'apiPath': '/get-next-mars-launch-window',
          'executionType': 'LAMBDA',
          'parameters': [
            {
              ...
            },
            {
              ...
            },
            {
              ...
            }
          ],
          'verb': 'get'
        },
        'invocationType': 'ACTION_GROUP',
        'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-0'
      },
      'event_order': 5,
      'type': 'invocationInput'
    },
    {
      'data': {
        'actionGroupInvocationOutput': {
          'text': '{"next_launch_window": {"next_launch_date": "2026-11-26 00:00:00", "synodic_period_days": 779.9068939794238, "transfer_time_days": 259, "delta_v_available_m_s": 39457.985759929674, "delta_v_required_m_s": 5595.997417810693, "is_feasible": true}}'
        },
        'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-0',
        'type': 'ACTION_GROUP'
      },
      'event_order': 6,
      'type': 'observation'
    }
  ],
  'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-1': [
    {
      'data': {
        'inferenceConfiguration': {
          'maximumLength': 2048,
          'stopSequences': [
            '</function_call>',
            '</answer>',
            '</error>'
          ],
          'temperature': 0.0,
          'topK': 250,
          'topP': 1.0
        },
        'text': '\n\nHuman:\nYou are a research assistant AI that has been equipped with one or more functions to help you answer a <question>...lta_v_available_m_s": 39457.985759929674, "delta_v_required_m_s": 5595.997417810693, "is_feasible": true}}</function_result>\n',
        'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-1',
        'type': 'ORCHESTRATION'
      },
      'event_order': 7,
      'type': 'modelInvocationInput'
    },
    {
      'data': {
        'metadata': {
          'usage': {
            'inputTokens': 5405,
            'outputTokens': 64
          }
        },
        'rawResponse': {
          'content': '<answer>\nBased on the provided spacecraft dry mass of 10000 kg, total mass of 50000 kg, and specific impulse of 2500 s, the ... optimal launch window for a Hohmann transfer from Earth to Mars is on November 26, 2026 UTC. The transfer will take 259 days.'
        },
        'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-1'
      },
      'event_order': 8,
      'type': 'modelInvocationOutput'
    },
    {
      'data': {
        'finalResponse': {
          'text': 'Based on the provided spacecraft dry mass of 10000 kg, total mass of 50000 kg, and specific impulse of 2500 s, the next optimal launch window for a Hohmann transfer from Earth to Mars is on November 26, 2026 UTC. The transfer will take 259 days.'
        },
        'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-1',
        'type': 'FINISH'
      },
      'event_order': 9,
      'type': 'observation'
    }
  ],
  'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-pre-0': [
    {
      'data': {
        'inferenceConfiguration': {
          'maximumLength': 2048,
          'stopSequences': [
            '\n\nHuman:'
          ],
          'temperature': 0.0,
          'topK': 250,
          'topP': 1.0
        },
        'text': '\n\nHuman: You are a classifying agent that filters user inputs into categories. Your job is to sort these inputs before they...<thinking> XML tags before providing only the category letter to sort the input into within <category> XML tags.\n\nAssistant:',
        'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-pre-0',
        'type': 'PRE_PROCESSING'
      },
      'event_order': 0,
      'type': 'modelInvocationInput'
    },
    {
      'data': {
        'parsedResponse': {
          'isValid': True,
          'rationale': 'Based on the provided instructions, this input appears to be a question about orbital mechanics that can be answered using th...equired arguments for that function - specific impulse, dry mass, and total mass. Therefore, this input should be sorted into:'
        },
        'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-pre-0'
      },
      'event_order': 1,
      'type': 'modelInvocationOutput'
    }
  ]
}

每个具有相同 traceId 的事件组将至少包含两个事件：一个类型为 modelInvocationInput 的事件和一个类型为 modelInvocationOutput 的事件。涉及操作组追踪的组还将包括类型为 actionGroupInvocationInput 和 actionGroupInvocationOutput 的事件。类似地，使用知识库的组将有额外的类型为 knowledgeBaseLookupInput 和 knowledgeBaseLookupOutput 的事件。在上面提到的 BedrockModel 中，它实现了一种方法来将这些事件组解析为追踪节点。此方法允许追踪显示选择操作组/知识库来回答查询以及调用相应的 Lambda 函数调用的推理，如我们上面示例 OpenAPI 规范中所述。这种结构有助于清晰地显示信息流和 Bedrock 代理遵循的决策过程。

这是最终的 MLflow 追踪

{
  "spans": [
    {
      "name": "Bedrock Agent Runtime",
      "context": {
        "span_id": "0xb802165d133a33aa",
        "trace_id": "0x9b8bd0b2e018d77f936e48a09e54fd44"
      },
      "parent_id": null,
      "start_time": 1731388531754725000,
      "end_time": 1731388550226771000,
      "status_code": "OK",
      "status_message": "",
      "attributes": {
        "mlflow.traceRequestId": "\"1e036cc3a7f946ec995f7763b8dde51c\"",
        "mlflow.spanType": "\"CHAT_MODEL\"",
        "mlflow.spanFunctionName": "\"predict\"",
        "mlflow.spanInputs": "{\"context\": \"<mlflow.pyfunc.model.PythonModelContext object at 0x13397c530>\", \"messages\": [{\"role\": \"user\", \"content\": \"When is the next launch window for Mars? My spacecraft's total mass is 50000, dry mass is 10000 and specific impulse is 2500. Mass in Kg.\", \"name\": null}], \"params\": {\"temperature\": 1.0, \"max_tokens\": null, \"stop\": null, \"n\": 1, \"stream\": false, \"top_p\": null, \"top_k\": null, \"frequency_penalty\": null, \"presence_penalty\": null}}",
        "mlflow.spanOutputs": "{\"choices\": [{\"index\": 0, \"message\": {\"role\": \"user\", \"content\": \"Based on the provided spacecraft dry mass of 10000 kg, total mass of 50000 kg, and specific impulse of 2500 s, the next optimal launch window for a Hohmann transfer from Earth to Mars is on November 26, 2026 UTC. The transfer will take 259 days.\", \"name\": null}, \"finish_reason\": \"stop\", \"logprobs\": null}], \"usage\": {\"prompt_tokens\": null, \"completion_tokens\": null, \"total_tokens\": null}, \"id\": null, \"model\": \"anthropic.claude-v2\", \"object\": \"chat.completion\", \"created\": 1731388550}"
      },
      "events": []
    },
    {
      "name": "Bedrock Input Prompt",
      "context": {
        "span_id": "0x2e7cd730be70865b",
        "trace_id": "0x9b8bd0b2e018d77f936e48a09e54fd44"
      },
      "parent_id": "0xb802165d133a33aa",
      "start_time": 1731388531755172000,
      "end_time": 1731388531755252000,
      "status_code": "OK",
      "status_message": "",
      "attributes": {
        "mlflow.traceRequestId": "\"1e036cc3a7f946ec995f7763b8dde51c\"",
        "mlflow.spanType": "\"UNKNOWN\"",
        "mlflow.spanFunctionName": "\"_get_agent_prompt\"",
        "mlflow.spanInputs": "{\"raw_input_question\": \"When is the next launch window for Mars? My spacecraft's total mass is 50000, dry mass is 10000 and specific impulse is 2500. Mass in Kg.\"}",
        "mlflow.spanOutputs": "\"\\n        Answer the following question and pay strong attention to the prompt:\\n        <question>\\n        When is the next launch window for Mars? My spacecraft's total mass is 50000, dry mass is 10000 and specific impulse is 2500. Mass in Kg.\\n        </question>\\n        <instruction>\\n        You have functions available at your disposal to use when anwering any questions about orbital mechanics.if you can't find a function to answer a question about orbital mechanics, simply reply 'I do not know'\\n        </instruction>\\n        \""
      },
      "events": []
    },
    {
      "name": "ACTION GROUP DECISION -optimal_departure_window_mars",
      "context": {
        "span_id": "0x131e4e08cd5e95d9",
        "trace_id": "0x9b8bd0b2e018d77f936e48a09e54fd44"
      },
      "parent_id": "0xb802165d133a33aa",
      "start_time": 1731388550223219000,
      "end_time": 1731388550224592000,
      "status_code": "OK",
      "status_message": "",
      "attributes": {
        "mlflow.traceRequestId": "\"1e036cc3a7f946ec995f7763b8dde51c\"",
        "mlflow.spanType": "\"UNKNOWN\"",
        "trace_attributes": "[{\"type\": \"modelInvocationInput\", \"data\": {\"inferenceConfiguration\": {\"maximumLength\": 2048, \"stopSequences\": [\"</function_call>\", \"</answer>\", \"</error>\"], \"temperature\": 0.0, \"topK\": 250, \"topP\": 1.0}, \"text\": \"\\n\\nHuman:\\nYou are a research assistant AI that has been equipped with one or more functions to help you answer a <question>. Your goal is to answer the user's question to the best of your ability, using the function(s) to gather more information if necessary to better answer the question. If you choose to call a function, the result of the function call will be added to the conversation history in <function_results> tags (if the call succeeded) or <error> tags (if the function failed). \\nYou were created with these instructions to consider as well:\\n<auxiliary_instructions>\\n            You are a friendly chat bot. You have access to a function called that returns\\n            information about the Mars launch window. When responding with Mars launch window,\\n            please make sure to add the timezone UTC.\\n            </auxiliary_instructions>\\n\\nHere are some examples of correct action by other, different agents with access to functions that may or may not be similar to ones you are provided.\\n\\n<examples>\\n    <example_docstring> Here is an example of how you would correctly answer a question using a <function_call> and the corresponding <function_result>. Notice that you are free to think before deciding to make a <function_call> in the <scratchpad>.</example_docstring>\\n    <example>\\n        <functions>\\n            <function>\\n                <function_name>get::policyengineactions::getpolicyviolations</function_name>\\n                <function_description>Returns a list of policy engine violations for the specified alias within the specified date range.</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <required_argument>startDate (string): The start date of the range to filter violations. The format for startDate is MM/DD/YYYY.</required_argument>\\n                <required_argument>endDate (string): The end date of the range to filter violations</required_argument>\\n                <returns>array: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>post::policyengineactions::acknowledgeviolations</function_name>\\n                <function_description>Acknowledge policy engine violation. Generally used to acknowledge violation, once user notices a violation under their alias or their managers alias.</function_description>\\n                <required_argument>policyId (string): The ID of the policy violation</required_argument>\\n                <required_argument>expectedDateOfResolution (string): The date by when the violation will be addressed/resolved</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>get::activedirectoryactions::getmanager</function_name>\\n                <function_description>This API is used to identify the manager hierarchy above a given person. Every person could have a manager and the manager could have another manager to which they report to</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            \\n        </functions>\\n\\n        <question>Can you show me my policy engine violation from 1st january 2023 to 1st february 2023? My alias is jsmith.</question>\\n        <scratchpad>\\n            To answer this question, I will need to:\\n            1. I do not have knowledge to policy engine violations, so I should see if I can use any of the available functions to help. I have been equipped with get::policyengineactions::getpolicyviolations that gets the policy engine violations for a given alias, start date and end date. I will use this function to gather more information.\\n        </scratchpad>\\n        <function_call>get::policyengineactions::getpolicyviolations(alias=\\\"jsmith\\\", startDate=\\\"1st January 2023\\\", endDate=\\\"1st February 2023\\\")</function_call>\\n        <function_result>{response: [{creationDate: \\\"2023-06-01T09:30:00Z\\\", riskLevel: \\\"High\\\", policyId: \\\"POL-001\\\", policyUrl: \\\"https://example.com/policies/POL-001\\\", referenceUrl: \\\"https://example.com/violations/POL-001\\\"}, {creationDate: \\\"2023-06-02T14:45:00Z\\\", riskLevel: \\\"Medium\\\", policyId: \\\"POL-002\\\", policyUrl: \\\"https://example.com/policies/POL-002\\\", referenceUrl: \\\"https://example.com/violations/POL-002\\\"}]}</function_result>\\n        <answer>The policy engine violations between 1st january 2023 to 1st february 2023 for alias jsmith are - Policy ID: POL-001, Policy ID: POL-002</answer>\\n    </example>\\n\\n    <example_docstring>Here is another example that utilizes multiple function calls.</example_docstring>\\n    <example>\\n        <functions>\\n            <function>\\n                <function_name>get::policyengineactions::getpolicyviolations</function_name>\\n                <function_description>Returns a list of policy engine violations for the specified alias within the specified date range.</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <required_argument>startDate (string): The start date of the range to filter violations. The format for startDate is MM/DD/YYYY.</required_argument>\\n                <required_argument>endDate (string): The end date of the range to filter violations</required_argument>\\n                <returns>array: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>post::policyengineactions::acknowledgeviolations</function_name>\\n                <function_description>Acknowledge policy engine violation. Generally used to acknowledge violation, once user notices a violation under their alias or their managers alias.</function_description>\\n                <required_argument>policyId (string): The ID of the policy violation</required_argument>\\n                <required_argument>expectedDateOfResolution (string): The date by when the violation will be addressed/resolved</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>get::activedirectoryactions::getmanager</function_name>\\n                <function_description>This API is used to identify the manager hierarchy above a given person. Every person could have a manager and the manager could have another manager to which they report to</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            \\n        </functions>\\n        <question>Can you check the policy engine violations under my manager between 2nd May to 5th May? My alias is john.</question>\\n        <scratchpad>\\n            To answer this question, I will need to:\\n            1. Get the manager alias of the user using get::activedirectoryactions::getmanager function.\\n            2. Use the returned manager alias to get the policy engine violations using the get::policyengineactions::getpolicyviolations function.\\n\\n            I have double checked and made sure that I have been provided the get::activedirectoryactions::getmanager and the get::policyengineactions::getpolicyviolations functions.\\n        </scratchpad>\\n        <function_call>get::activedirectoryactions::getmanager(alias=\\\"john\\\")</function_call>\\n        <function_result>{response: {managerAlias: \\\"mark\\\", managerLevel: \\\"6\\\", teamName: \\\"Builder\\\", managerName: \\\"Mark Hunter\\\"}}}}</function_result>\\n        <scratchpad>\\n            1. I have the managerAlias from the function results as mark and I have the start and end date from the user input. I can use the function result to call get::policyengineactions::getpolicyviolations function.\\n            2. I will then return the get::policyengineactions::getpolicyviolations function result to the user.\\n\\n            I have double checked and made sure that I have been provided the get::policyengineactions::getpolicyviolations functions.\\n        </scratchpad>\\n        <function_call>get::policyengineactions::getpolicyviolations(alias=\\\"mark\\\", startDate=\\\"2nd May 2023\\\", endDate=\\\"5th May 2023\\\")</function_call>\\n        <function_result>{response: [{creationDate: \\\"2023-05-02T09:30:00Z\\\", riskLevel: \\\"High\\\", policyId: \\\"POL-001\\\", policyUrl: \\\"https://example.com/policies/POL-001\\\", referenceUrl: \\\"https://example.com/violations/POL-001\\\"}, {creationDate: \\\"2023-05-04T14:45:00Z\\\", riskLevel: \\\"Low\\\", policyId: \\\"POL-002\\\", policyUrl: \\\"https://example.com/policies/POL-002\\\", referenceUrl: \\\"https://example.com/violations/POL-002\\\"}]}</function_result>\\n        <answer>\\n            The policy engine violations between 2nd May 2023 to 5th May 2023 for your manager's alias mark are - Policy ID: POL-001, Policy ID: POL-002\\n        </answer>\\n    </example>\\n\\n    <example_docstring>Functions can also be search engine API's that issue a query to a knowledge base. Here is an example that utilizes regular function calls in combination with function calls to a search engine API. Please make sure to extract the source for the information within the final answer when using information returned from the search engine.</example_docstring>\\n    <example>\\n        <functions>\\n            <function>\\n                <function_name>get::benefitsaction::getbenefitplanname</function_name>\\n                <function_description>Get's the benefit plan name for a user. The API takes in a userName and a benefit type and returns the benefit name to the user (i.e. Aetna, Premera, Fidelity, etc.).</function_description>\\n                <optional_argument>userName (string): None</optional_argument>\\n                <optional_argument>benefitType (string): None</optional_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>post::benefitsaction::increase401klimit</function_name>\\n                <function_description>Increases the 401k limit for a generic user. The API takes in only the current 401k limit and returns the new limit.</function_description>\\n                <optional_argument>currentLimit (string): None</optional_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>get::x_amz_knowledgebase_dentalinsurance::search</function_name>\\n                <function_description>This is a search tool that provides information about Delta Dental benefits. It has information about covered dental benefits and other relevant information</function_description>\\n                <required_argument>query(string): A full sentence query that is fed to the search tool</required_argument>\\n                <returns>Returns string  related to the user query asked.</returns>\\n            </function>\\n            <function>\\n                <function_name>get::x_amz_knowledgebase_401kplan::search</function_name>\\n                <function_description>This is a search tool that provides information about Amazon 401k plan benefits. It can determine what a person's yearly 401k contribution limit is, based on their age.</function_description>\\n                <required_argument>query(string): A full sentence query that is fed to the search tool</required_argument>\\n                <returns>Returns string  related to the user query asked.</returns>\\n            </function>\\n            <function>\\n                <function_name>get::x_amz_knowledgebase_healthinsurance::search</function_name>\\n                <function_description>This is a search tool that provides information about Aetna and Premera health benefits. It has information about the savings plan and shared deductible plan, as well as others.</function_description>\\n                <required_argument>query(string): A full sentence query that is fed to the search tool</required_argument>\\n                <returns>Returns string  related to the user query asked.</returns>\\n            </function>\\n            \\n        </functions>\\n\\n        <question>What is my deductible? My username is Bob and my benefitType is Dental. Also, what is the 401k yearly contribution limit?</question>\\n        <scratchpad> I understand I cannot use functions that have not been provided to me to answer this question.\\n            To answer this question, I will:\\n            1. Call the get::benefitsaction::getbenefitplanname function to get the benefit plan name for the user Bob with benefit type Dental.\\n            2. Call the get::x_amz_knowledgebase_dentalinsurance::search function to search for information about deductibles for the plan name returned from step 1.\\n            3. Call the get::x_amz_knowledgebase_401k::search function to search for information about 401k yearly contribution limits.\\n            4. Return the deductible information from the search results to the user.\\n            I have checked that I have access to the get::benefitsaction::getbenefitplanname, x_amz_knowledgebase_dentalinsurance::search, and x_amz_knowledgebase_401k::search functions.\\n        </scratchpad>\\n        <function_call>get::benefitsaction::getbenefitplanname(userName=\\\"Bob\\\", benefitType=\\\"Dental\\\")</function_call>\\n        <function_result>{{'response': {{'planName': 'Delta Dental'}}}}</function_result>\\n        <scratchpad>\\n            I have received the plan name Delta Dental for the user Bob with Dental benefits. I will now call the x_amz_knowledgebase_dentalinsurance::search function to find deductible information for Delta Dental.\\n        </scratchpad>\\n        <function_call>get::x_amz_knowledgebase_dentalinsurance::search(searchQuery=\\\"What is the deductible for Delta Dental?\\\")</function_call>\\n        <function_result>{{'response': {{'responseCode': '200', 'responseBody': \\\"\\\"<answer>\\\\n<answer_part>\\\\n<text>The user's individual deductible is $50 per benefit period</text>\\\\n<source>dfe040f8-46ed-4a65-b3ea-529fa55f6b9e</source>\\\\n</answer_part>\\\\n<answer_part>\\\\n<text>If they are enrolled with dependents, the maximum family deductible is $150 per benefit period.</text>\\\\n<source>0e666064-31d8-4223-b7ba-8eecf40b7b47</source>\\\\n</answer_part>\\\\n</answer>\\\"}}}}</function_result> <scratchpad>\\n            I have found the deductible information for Dental benefits. I will now call the x_amz_knowledgebase_401k::search function to find yearly 401k contribution limits.\\n        </scratchpad>\\n        <function_call>get::x_amz_knowledgebase_401k::search(searchQuery=\\\"What is the yearly 401k contribution limit?\\\")</function_call>\\n        <function_result>{{'response': {{'responseCode': '200', 'responseBody': \\\"<answer>\\\\n<answer_part>\\\\n<text>The yearly 401k contribution limit is $20,500.</text>\\\\n<source>c546cbe8-07f6-45d1-90ca-74d87ab2885a</source>\\\\n</answer_part>\\\\n</answer>\\\"}}}}</function_result>\\n        <answer>\\n            <answer_part>\\n                <text>The deductible for your Delta Dental plan is $50 per benefit period.</text>\\n                <source>dfe040f8-46ed-4a65-b3ea-529fa55f6b9e</source>\\n            </answer_part>\\n            <answer_part>\\n                <text>If you have dependents enrolled, the maximum family deductible is $150 per benefit period.</text>\\n                <source>0e666064-31d8-4223-b7ba-8eecf40b7b47</source>\\n            </answer_part>\\n            <answer_part>\\n                <text>The yearly 401k contribution limit is $20,500.</text>\\n                <source>c546cbe8-07f6-45d1-90ca-74d87ab2885a</source>\\n            </answer_part>\\n        </answer>\\n    </example>\\n\\n    \\n\\n    <example_docstring>Here's a final example where the question asked could not be answered with information gathered from calling the provided functions. In this example, notice how you respond by telling the user you cannot answer, without using a function that was not provided to you.</example_docstring>\\n    <example>\\n        <functions>\\n            <function>\\n                <function_name>get::policyengineactions::getpolicyviolations</function_name>\\n                <function_description>Returns a list of policy engine violations for the specified alias within the specified date range.</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <required_argument>startDate (string): The start date of the range to filter violations. The format for startDate is MM/DD/YYYY.</required_argument>\\n                <required_argument>endDate (string): The end date of the range to filter violations</required_argument>\\n                <returns>array: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>post::policyengineactions::acknowledgeviolations</function_name>\\n                <function_description>Acknowledge policy engine violation. Generally used to acknowledge violation, once user notices a violation under their alias or their managers alias.</function_description>\\n                <required_argument>policyId (string): The ID of the policy violation</required_argument>\\n                <required_argument>expectedDateOfResolution (string): The date by when the violation will be addressed/resolved</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>get::activedirectoryactions::getmanager</function_name>\\n                <function_description>This API is used to identify the manager hierarchy above a given person. Every person could have a manager and the manager could have another manager to which they report to</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            \\n        </functions>\\n        <question>Who are the reportees of David?</question>\\n        <scratchpad>\\n            After reviewing the functions I was equipped with, I realize I am not able to accurately answer this question since I can't access reportees of David. Therefore, I should explain to the user I cannot answer this question.\\n        </scratchpad>\\n        <answer>\\n            Sorry, I am unable to assist you with this request.\\n        </answer>\\n    </example>\\n</examples>\\n\\nThe above examples have been provided to you to illustrate general guidelines and format for use of function calling for information retrieval, and how to use your scratchpad to plan your approach. IMPORTANT: the functions provided within the examples should not be assumed to have been provided to you to use UNLESS they are also explicitly given to you within <functions></functions> tags below. All of the values and information within the examples (the questions, function results, and answers) are strictly part of the examples and have not been provided to you.\\n\\nNow that you have read and understood the examples, I will define the functions that you have available to you to use. Here is a comprehensive list.\\n\\n<functions>\\n<function>\\n<function_name>GET::optimal_departure_window_mars::getNextMarsLaunchWindow</function_name>\\n<function_description>Gets the next optimal launch window to Mars.</function_description>\\n<required_argument>specific_impulse (string): Specific impulse of the propulsion system (s).</required_argument>\\n<required_argument>dry_mass (string): Mass of the spacecraft without fuel (kg).</required_argument>\\n<required_argument>total_mass (string): Total mass of the spacecraft including fuel (kg)</required_argument>\\n<returns>object: The next optimal departure date for a Hohmann transfer from Earth to Mars, based on the spacecraft's mass and specific impulse.</returns>\\n</function>\\n\\n\\n</functions>\\n\\nNote that the function arguments have been listed in the order that they should be passed into the function.\\n\\n\\n\\nDo not modify or extend the provided functions under any circumstances. For example, GET::optimal_departure_window_mars::getNextMarsLaunchWindow with additional parameters would be considered modifying the function which is not allowed. Please use the functions only as defined.\\n\\nDO NOT use any functions that I have not equipped you with.\\n\\n Do not make assumptions about inputs; instead, make sure you know the exact function and input to use before you call a function.\\n\\nTo call a function, output the name of the function in between <function_call> and </function_call> tags. You will receive a <function_result> in response to your call that contains information that you can use to better answer the question. Or, if the function call produced an error, you will receive an <error> in response.\\n\\n\\n\\nThe format for all other <function_call> MUST be: <function_call>$FUNCTION_NAME($FUNCTION_PARAMETER_NAME=$FUNCTION_PARAMETER_VALUE)</function_call>\\n\\nRemember, your goal is to answer the user's question to the best of your ability, using only the function(s) provided within the <functions></functions> tags to gather more information if necessary to better answer the question.\\n\\nDo not modify or extend the provided functions under any circumstances. For example, calling GET::optimal_departure_window_mars::getNextMarsLaunchWindow with additional parameters would be modifying the function which is not allowed. Please use the functions only as defined.\\n\\nBefore calling any functions, create a plan for performing actions to answer this question within the <scratchpad>. Double check your plan to make sure you don't call any functions that you haven't been provided with. Always return your final answer within <answer></answer> tags.\\n\\n\\n\\nThe user input is <question>Answer the following question and pay strong attention to the prompt:\\n        <question>\\n        When is the next launch window for Mars? My spacecraft's total mass is 50000, dry mass is 10000 and specific impulse is 2500. Mass in Kg.\\n        </question>\\n        <instruction>\\n        You have functions available at your disposal to use when anwering any questions about orbital mechanics.if you can't find a function to answer a question about orbital mechanics, simply reply 'I do not know'\\n        </instruction></question>\\n\\n\\nAssistant: <scratchpad> I understand I cannot use functions that have not been provided to me to answer this question.\\n\\n\", \"traceId\": \"e0b2b2c2-fb7c-4e17-8a1f-a3781100face-0\", \"type\": \"ORCHESTRATION\"}, \"event_order\": 2}, {\"type\": \"modelInvocationOutput\", \"data\": {\"metadata\": {\"usage\": {\"inputTokens\": 5160, \"outputTokens\": 135}}, \"rawResponse\": {\"content\": \"To answer this question about the next Mars launch window, I will:\\n\\n1. Call the GET::optimal_departure_window_mars::getNextMarsLaunchWindow function to get the next optimal launch window, passing in the provided spacecraft mass and specific impulse values.\\n\\nI have verified that I have access to the GET::optimal_departure_window_mars::getNextMarsLaunchWindow function.\\n\\n</scratchpad>\\n\\n<function_call>\\nGET::optimal_departure_window_mars::getNextMarsLaunchWindow(specific_impulse=\\\"2500\\\", dry_mass=\\\"10000\\\", total_mass=\\\"50000\\\")\"}, \"traceId\": \"e0b2b2c2-fb7c-4e17-8a1f-a3781100face-0\"}, \"event_order\": 3}, {\"type\": \"rationale\", \"data\": {\"text\": \"To answer this question about the next Mars launch window, I will:\\n\\n1. Call the GET::optimal_departure_window_mars::getNextMarsLaunchWindow function to get the next optimal launch window, passing in the provided spacecraft mass and specific impulse values.\\n\\nI have verified that I have access to the GET::optimal_departure_window_mars::getNextMarsLaunchWindow function.\", \"traceId\": \"e0b2b2c2-fb7c-4e17-8a1f-a3781100face-0\"}, \"event_order\": 4}, {\"type\": \"invocationInput\", \"data\": {\"actionGroupInvocationInput\": {\"actionGroupName\": \"optimal_departure_window_mars\", \"apiPath\": \"/get-next-mars-launch-window\", \"executionType\": \"LAMBDA\", \"parameters\": [{\"name\": \"total_mass\", \"type\": \"string\", \"value\": \"50000\"}, {\"name\": \"dry_mass\", \"type\": \"string\", \"value\": \"10000\"}, {\"name\": \"specific_impulse\", \"type\": \"string\", \"value\": \"2500\"}], \"verb\": \"get\"}, \"invocationType\": \"ACTION_GROUP\", \"traceId\": \"e0b2b2c2-fb7c-4e17-8a1f-a3781100face-0\"}, \"event_order\": 5}, {\"type\": \"observation\", \"data\": {\"actionGroupInvocationOutput\": {\"text\": \"{\\\"next_launch_window\\\": {\\\"next_launch_date\\\": \\\"2026-11-26 00:00:00\\\", \\\"synodic_period_days\\\": 779.9068939794238, \\\"transfer_time_days\\\": 259, \\\"delta_v_available_m_s\\\": 39457.985759929674, \\\"delta_v_required_m_s\\\": 5595.997417810693, \\\"is_feasible\\\": true}}\"}, \"traceId\": \"e0b2b2c2-fb7c-4e17-8a1f-a3781100face-0\", \"type\": \"ACTION_GROUP\"}, \"event_order\": 6}]",
        "mlflow.spanFunctionName": "\"_trace_agent_pre_context\"",
        "mlflow.spanInputs": "{\"inner_input_trace\": \"\\n\\nHuman:\\nYou are a research assistant AI that has been equipped with one or more functions to help you answer a <question>. Your goal is to answer the user's question to the best of your ability, using the function(s) to gather more information if necessary to better answer the question. If you choose to call a function, the result of the function call will be added to the conversation history in <function_results> tags (if the call succeeded) or <error> tags (if the function failed). \\nYou were created with these instructions to consider as well:\\n<auxiliary_instructions>\\n            You are a friendly chat bot. You have access to a function called that returns\\n            information about the Mars launch window. When responding with Mars launch window,\\n            please make sure to add the timezone UTC.\\n            </auxiliary_instructions>\\n\\nHere are some examples of correct action by other, different agents with access to functions that may or may not be similar to ones you are provided.\\n\\n<examples>\\n    <example_docstring> Here is an example of how you would correctly answer a question using a <function_call> and the corresponding <function_result>. Notice that you are free to think before deciding to make a <function_call> in the <scratchpad>.</example_docstring>\\n    <example>\\n        <functions>\\n            <function>\\n                <function_name>get::policyengineactions::getpolicyviolations</function_name>\\n                <function_description>Returns a list of policy engine violations for the specified alias within the specified date range.</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <required_argument>startDate (string): The start date of the range to filter violations. The format for startDate is MM/DD/YYYY.</required_argument>\\n                <required_argument>endDate (string): The end date of the range to filter violations</required_argument>\\n                <returns>array: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>post::policyengineactions::acknowledgeviolations</function_name>\\n                <function_description>Acknowledge policy engine violation. Generally used to acknowledge violation, once user notices a violation under their alias or their managers alias.</function_description>\\n                <required_argument>policyId (string): The ID of the policy violation</required_argument>\\n                <required_argument>expectedDateOfResolution (string): The date by when the violation will be addressed/resolved</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>get::activedirectoryactions::getmanager</function_name>\\n                <function_description>This API is used to identify the manager hierarchy above a given person. Every person could have a manager and the manager could have another manager to which they report to</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            \\n        </functions>\\n\\n        <question>Can you show me my policy engine violation from 1st january 2023 to 1st february 2023? My alias is jsmith.</question>\\n        <scratchpad>\\n            To answer this question, I will need to:\\n            1. I do not have knowledge to policy engine violations, so I should see if I can use any of the available functions to help. I have been equipped with get::policyengineactions::getpolicyviolations that gets the policy engine violations for a given alias, start date and end date. I will use this function to gather more information.\\n        </scratchpad>\\n        <function_call>get::policyengineactions::getpolicyviolations(alias=\\\"jsmith\\\", startDate=\\\"1st January 2023\\\", endDate=\\\"1st February 2023\\\")</function_call>\\n        <function_result>{response: [{creationDate: \\\"2023-06-01T09:30:00Z\\\", riskLevel: \\\"High\\\", policyId: \\\"POL-001\\\", policyUrl: \\\"https://example.com/policies/POL-001\\\", referenceUrl: \\\"https://example.com/violations/POL-001\\\"}, {creationDate: \\\"2023-06-02T14:45:00Z\\\", riskLevel: \\\"Medium\\\", policyId: \\\"POL-002\\\", policyUrl: \\\"https://example.com/policies/POL-002\\\", referenceUrl: \\\"https://example.com/violations/POL-002\\\"}]}</function_result>\\n        <answer>The policy engine violations between 1st january 2023 to 1st february 2023 for alias jsmith are - Policy ID: POL-001, Policy ID: POL-002</answer>\\n    </example>\\n\\n    <example_docstring>Here is another example that utilizes multiple function calls.</example_docstring>\\n    <example>\\n        <functions>\\n            <function>\\n                <function_name>get::policyengineactions::getpolicyviolations</function_name>\\n                <function_description>Returns a list of policy engine violations for the specified alias within the specified date range.</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <required_argument>startDate (string): The start date of the range to filter violations. The format for startDate is MM/DD/YYYY.</required_argument>\\n                <required_argument>endDate (string): The end date of the range to filter violations</required_argument>\\n                <returns>array: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>post::policyengineactions::acknowledgeviolations</function_name>\\n                <function_description>Acknowledge policy engine violation. Generally used to acknowledge violation, once user notices a violation under their alias or their managers alias.</function_description>\\n                <required_argument>policyId (string): The ID of the policy violation</required_argument>\\n                <required_argument>expectedDateOfResolution (string): The date by when the violation will be addressed/resolved</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>get::activedirectoryactions::getmanager</function_name>\\n                <function_description>This API is used to identify the manager hierarchy above a given person. Every person could have a manager and the manager could have another manager to which they report to</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            \\n        </functions>\\n        <question>Can you check the policy engine violations under my manager between 2nd May to 5th May? My alias is john.</question>\\n        <scratchpad>\\n            To answer this question, I will need to:\\n            1. Get the manager alias of the user using get::activedirectoryactions::getmanager function.\\n            2. Use the returned manager alias to get the policy engine violations using the get::policyengineactions::getpolicyviolations function.\\n\\n            I have double checked and made sure that I have been provided the get::activedirectoryactions::getmanager and the get::policyengineactions::getpolicyviolations functions.\\n        </scratchpad>\\n        <function_call>get::activedirectoryactions::getmanager(alias=\\\"john\\\")</function_call>\\n        <function_result>{response: {managerAlias: \\\"mark\\\", managerLevel: \\\"6\\\", teamName: \\\"Builder\\\", managerName: \\\"Mark Hunter\\\"}}}}</function_result>\\n        <scratchpad>\\n            1. I have the managerAlias from the function results as mark and I have the start and end date from the user input. I can use the function result to call get::policyengineactions::getpolicyviolations function.\\n            2. I will then return the get::policyengineactions::getpolicyviolations function result to the user.\\n\\n            I have double checked and made sure that I have been provided the get::policyengineactions::getpolicyviolations functions.\\n        </scratchpad>\\n        <function_call>get::policyengineactions::getpolicyviolations(alias=\\\"mark\\\", startDate=\\\"2nd May 2023\\\", endDate=\\\"5th May 2023\\\")</function_call>\\n        <function_result>{response: [{creationDate: \\\"2023-05-02T09:30:00Z\\\", riskLevel: \\\"High\\\", policyId: \\\"POL-001\\\", policyUrl: \\\"https://example.com/policies/POL-001\\\", referenceUrl: \\\"https://example.com/violations/POL-001\\\"}, {creationDate: \\\"2023-05-04T14:45:00Z\\\", riskLevel: \\\"Low\\\", policyId: \\\"POL-002\\\", policyUrl: \\\"https://example.com/policies/POL-002\\\", referenceUrl: \\\"https://example.com/violations/POL-002\\\"}]}</function_result>\\n        <answer>\\n            The policy engine violations between 2nd May 2023 to 5th May 2023 for your manager's alias mark are - Policy ID: POL-001, Policy ID: POL-002\\n        </answer>\\n    </example>\\n\\n    <example_docstring>Functions can also be search engine API's that issue a query to a knowledge base. Here is an example that utilizes regular function calls in combination with function calls to a search engine API. Please make sure to extract the source for the information within the final answer when using information returned from the search engine.</example_docstring>\\n    <example>\\n        <functions>\\n            <function>\\n                <function_name>get::benefitsaction::getbenefitplanname</function_name>\\n                <function_description>Get's the benefit plan name for a user. The API takes in a userName and a benefit type and returns the benefit name to the user (i.e. Aetna, Premera, Fidelity, etc.).</function_description>\\n                <optional_argument>userName (string): None</optional_argument>\\n                <optional_argument>benefitType (string): None</optional_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>post::benefitsaction::increase401klimit</function_name>\\n                <function_description>Increases the 401k limit for a generic user. The API takes in only the current 401k limit and returns the new limit.</function_description>\\n                <optional_argument>currentLimit (string): None</optional_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>get::x_amz_knowledgebase_dentalinsurance::search</function_name>\\n                <function_description>This is a search tool that provides information about Delta Dental benefits. It has information about covered dental benefits and other relevant information</function_description>\\n                <required_argument>query(string): A full sentence query that is fed to the search tool</required_argument>\\n                <returns>Returns string  related to the user query asked.</returns>\\n            </function>\\n            <function>\\n                <function_name>get::x_amz_knowledgebase_401kplan::search</function_name>\\n                <function_description>This is a search tool that provides information about Amazon 401k plan benefits. It can determine what a person's yearly 401k contribution limit is, based on their age.</function_description>\\n                <required_argument>query(string): A full sentence query that is fed to the search tool</required_argument>\\n                <returns>Returns string  related to the user query asked.</returns>\\n            </function>\\n            <function>\\n                <function_name>get::x_amz_knowledgebase_healthinsurance::search</function_name>\\n                <function_description>This is a search tool that provides information about Aetna and Premera health benefits. It has information about the savings plan and shared deductible plan, as well as others.</function_description>\\n                <required_argument>query(string): A full sentence query that is fed to the search tool</required_argument>\\n                <returns>Returns string  related to the user query asked.</returns>\\n            </function>\\n            \\n        </functions>\\n\\n        <question>What is my deductible? My username is Bob and my benefitType is Dental. Also, what is the 401k yearly contribution limit?</question>\\n        <scratchpad> I understand I cannot use functions that have not been provided to me to answer this question.\\n            To answer this question, I will:\\n            1. Call the get::benefitsaction::getbenefitplanname function to get the benefit plan name for the user Bob with benefit type Dental.\\n            2. Call the get::x_amz_knowledgebase_dentalinsurance::search function to search for information about deductibles for the plan name returned from step 1.\\n            3. Call the get::x_amz_knowledgebase_401k::search function to search for information about 401k yearly contribution limits.\\n            4. Return the deductible information from the search results to the user.\\n            I have checked that I have access to the get::benefitsaction::getbenefitplanname, x_amz_knowledgebase_dentalinsurance::search, and x_amz_knowledgebase_401k::search functions.\\n        </scratchpad>\\n        <function_call>get::benefitsaction::getbenefitplanname(userName=\\\"Bob\\\", benefitType=\\\"Dental\\\")</function_call>\\n        <function_result>{{'response': {{'planName': 'Delta Dental'}}}}</function_result>\\n        <scratchpad>\\n            I have received the plan name Delta Dental for the user Bob with Dental benefits. I will now call the x_amz_knowledgebase_dentalinsurance::search function to find deductible information for Delta Dental.\\n        </scratchpad>\\n        <function_call>get::x_amz_knowledgebase_dentalinsurance::search(searchQuery=\\\"What is the deductible for Delta Dental?\\\")</function_call>\\n        <function_result>{{'response': {{'responseCode': '200', 'responseBody': \\\"\\\"<answer>\\\\n<answer_part>\\\\n<text>The user's individual deductible is $50 per benefit period</text>\\\\n<source>dfe040f8-46ed-4a65-b3ea-529fa55f6b9e</source>\\\\n</answer_part>\\\\n<answer_part>\\\\n<text>If they are enrolled with dependents, the maximum family deductible is $150 per benefit period.</text>\\\\n<source>0e666064-31d8-4223-b7ba-8eecf40b7b47</source>\\\\n</answer_part>\\\\n</answer>\\\"}}}}</function_result> <scratchpad>\\n            I have found the deductible information for Dental benefits. I will now call the x_amz_knowledgebase_401k::search function to find yearly 401k contribution limits.\\n        </scratchpad>\\n        <function_call>get::x_amz_knowledgebase_401k::search(searchQuery=\\\"What is the yearly 401k contribution limit?\\\")</function_call>\\n        <function_result>{{'response': {{'responseCode': '200', 'responseBody': \\\"<answer>\\\\n<answer_part>\\\\n<text>The yearly 401k contribution limit is $20,500.</text>\\\\n<source>c546cbe8-07f6-45d1-90ca-74d87ab2885a</source>\\\\n</answer_part>\\\\n</answer>\\\"}}}}</function_result>\\n        <answer>\\n            <answer_part>\\n                <text>The deductible for your Delta Dental plan is $50 per benefit period.</text>\\n                <source>dfe040f8-46ed-4a65-b3ea-529fa55f6b9e</source>\\n            </answer_part>\\n            <answer_part>\\n                <text>If you have dependents enrolled, the maximum family deductible is $150 per benefit period.</text>\\n                <source>0e666064-31d8-4223-b7ba-8eecf40b7b47</source>\\n            </answer_part>\\n            <answer_part>\\n                <text>The yearly 401k contribution limit is $20,500.</text>\\n                <source>c546cbe8-07f6-45d1-90ca-74d87ab2885a</source>\\n            </answer_part>\\n        </answer>\\n    </example>\\n\\n    \\n\\n    <example_docstring>Here's a final example where the question asked could not be answered with information gathered from calling the provided functions. In this example, notice how you respond by telling the user you cannot answer, without using a function that was not provided to you.</example_docstring>\\n    <example>\\n        <functions>\\n            <function>\\n                <function_name>get::policyengineactions::getpolicyviolations</function_name>\\n                <function_description>Returns a list of policy engine violations for the specified alias within the specified date range.</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <required_argument>startDate (string): The start date of the range to filter violations. The format for startDate is MM/DD/YYYY.</required_argument>\\n                <required_argument>endDate (string): The end date of the range to filter violations</required_argument>\\n                <returns>array: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>post::policyengineactions::acknowledgeviolations</function_name>\\n                <function_description>Acknowledge policy engine violation. Generally used to acknowledge violation, once user notices a violation under their alias or their managers alias.</function_description>\\n                <required_argument>policyId (string): The ID of the policy violation</required_argument>\\n                <required_argument>expectedDateOfResolution (string): The date by when the violation will be addressed/resolved</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>get::activedirectoryactions::getmanager</function_name>\\n                <function_description>This API is used to identify the manager hierarchy above a given person. Every person could have a manager and the manager could have another manager to which they report to</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            \\n        </functions>\\n        <question>Who are the reportees of David?</question>\\n        <scratchpad>\\n            After reviewing the functions I was equipped with, I realize I am not able to accurately answer this question since I can't access reportees of David. Therefore, I should explain to the user I cannot answer this question.\\n        </scratchpad>\\n        <answer>\\n            Sorry, I am unable to assist you with this request.\\n        </answer>\\n    </example>\\n</examples>\\n\\nThe above examples have been provided to you to illustrate general guidelines and format for use of function calling for information retrieval, and how to use your scratchpad to plan your approach. IMPORTANT: the functions provided within the examples should not be assumed to have been provided to you to use UNLESS they are also explicitly given to you within <functions></functions> tags below. All of the values and information within the examples (the questions, function results, and answers) are strictly part of the examples and have not been provided to you.\\n\\nNow that you have read and understood the examples, I will define the functions that you have available to you to use. Here is a comprehensive list.\\n\\n<functions>\\n<function>\\n<function_name>GET::optimal_departure_window_mars::getNextMarsLaunchWindow</function_name>\\n<function_description>Gets the next optimal launch window to Mars.</function_description>\\n<required_argument>specific_impulse (string): Specific impulse of the propulsion system (s).</required_argument>\\n<required_argument>dry_mass (string): Mass of the spacecraft without fuel (kg).</required_argument>\\n<required_argument>total_mass (string): Total mass of the spacecraft including fuel (kg)</required_argument>\\n<returns>object: The next optimal departure date for a Hohmann transfer from Earth to Mars, based on the spacecraft's mass and specific impulse.</returns>\\n</function>\\n\\n\\n</functions>\\n\\nNote that the function arguments have been listed in the order that they should be passed into the function.\\n\\n\\n\\nDo not modify or extend the provided functions under any circumstances. For example, GET::optimal_departure_window_mars::getNextMarsLaunchWindow with additional parameters would be considered modifying the function which is not allowed. Please use the functions only as defined.\\n\\nDO NOT use any functions that I have not equipped you with.\\n\\n Do not make assumptions about inputs; instead, make sure you know the exact function and input to use before you call a function.\\n\\nTo call a function, output the name of the function in between <function_call> and </function_call> tags. You will receive a <function_result> in response to your call that contains information that you can use to better answer the question. Or, if the function call produced an error, you will receive an <error> in response.\\n\\n\\n\\nThe format for all other <function_call> MUST be: <function_call>$FUNCTION_NAME($FUNCTION_PARAMETER_NAME=$FUNCTION_PARAMETER_VALUE)</function_call>\\n\\nRemember, your goal is to answer the user's question to the best of your ability, using only the function(s) provided within the <functions></functions> tags to gather more information if necessary to better answer the question.\\n\\nDo not modify or extend the provided functions under any circumstances. For example, calling GET::optimal_departure_window_mars::getNextMarsLaunchWindow with additional parameters would be modifying the function which is not allowed. Please use the functions only as defined.\\n\\nBefore calling any functions, create a plan for performing actions to answer this question within the <scratchpad>. Double check your plan to make sure you don't call any functions that you haven't been provided with. Always return your final answer within <answer></answer> tags.\\n\\n\\n\\nThe user input is <question>Answer the following question and pay strong attention to the prompt:\\n        <question>\\n        When is the next launch window for Mars? My spacecraft's total mass is 50000, dry mass is 10000 and specific impulse is 2500. Mass in Kg.\\n        </question>\\n        <instruction>\\n        You have functions available at your disposal to use when anwering any questions about orbital mechanics.if you can't find a function to answer a question about orbital mechanics, simply reply 'I do not know'\\n        </instruction></question>\\n\\n\\nAssistant: <scratchpad> I understand I cannot use functions that have not been provided to me to answer this question.\\n\\n\"}",
        "mlflow.spanOutputs": "\"To answer this question about the next Mars launch window, I will:\\n\\n1. Call the GET::optimal_departure_window_mars::getNextMarsLaunchWindow function to get the next optimal launch window, passing in the provided spacecraft mass and specific impulse values.\\n\\nI have verified that I have access to the GET::optimal_departure_window_mars::getNextMarsLaunchWindow function.\""
      },
      "events": []
    },
    {
      "name": "Invoking Action Group",
      "context": {
        "span_id": "0x692bd6457647dc76",
        "trace_id": "0x9b8bd0b2e018d77f936e48a09e54fd44"
      },
      "parent_id": "0xb802165d133a33aa",
      "start_time": 1731388550224851000,
      "end_time": 1731388550225218000,
      "status_code": "OK",
      "status_message": "",
      "attributes": {
        "mlflow.traceRequestId": "\"1e036cc3a7f946ec995f7763b8dde51c\"",
        "mlflow.spanType": "\"UNKNOWN\"",
        "trace_attributes": "[{\"type\": \"modelInvocationInput\", \"data\": {\"inferenceConfiguration\": {\"maximumLength\": 2048, \"stopSequences\": [\"</function_call>\", \"</answer>\", \"</error>\"], \"temperature\": 0.0, \"topK\": 250, \"topP\": 1.0}, \"text\": \"\\n\\nHuman:\\nYou are a research assistant AI that has been equipped with one or more functions to help you answer a <question>. Your goal is to answer the user's question to the best of your ability, using the function(s) to gather more information if necessary to better answer the question. If you choose to call a function, the result of the function call will be added to the conversation history in <function_results> tags (if the call succeeded) or <error> tags (if the function failed). \\nYou were created with these instructions to consider as well:\\n<auxiliary_instructions>\\n            You are a friendly chat bot. You have access to a function called that returns\\n            information about the Mars launch window. When responding with Mars launch window,\\n            please make sure to add the timezone UTC.\\n            </auxiliary_instructions>\\n\\nHere are some examples of correct action by other, different agents with access to functions that may or may not be similar to ones you are provided.\\n\\n<examples>\\n    <example_docstring> Here is an example of how you would correctly answer a question using a <function_call> and the corresponding <function_result>. Notice that you are free to think before deciding to make a <function_call> in the <scratchpad>.</example_docstring>\\n    <example>\\n        <functions>\\n            <function>\\n                <function_name>get::policyengineactions::getpolicyviolations</function_name>\\n                <function_description>Returns a list of policy engine violations for the specified alias within the specified date range.</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <required_argument>startDate (string): The start date of the range to filter violations. The format for startDate is MM/DD/YYYY.</required_argument>\\n                <required_argument>endDate (string): The end date of the range to filter violations</required_argument>\\n                <returns>array: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>post::policyengineactions::acknowledgeviolations</function_name>\\n                <function_description>Acknowledge policy engine violation. Generally used to acknowledge violation, once user notices a violation under their alias or their managers alias.</function_description>\\n                <required_argument>policyId (string): The ID of the policy violation</required_argument>\\n                <required_argument>expectedDateOfResolution (string): The date by when the violation will be addressed/resolved</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>get::activedirectoryactions::getmanager</function_name>\\n                <function_description>This API is used to identify the manager hierarchy above a given person. Every person could have a manager and the manager could have another manager to which they report to</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            \\n        </functions>\\n\\n        <question>Can you show me my policy engine violation from 1st january 2023 to 1st february 2023? My alias is jsmith.</question>\\n        <scratchpad>\\n            To answer this question, I will need to:\\n            1. I do not have knowledge to policy engine violations, so I should see if I can use any of the available functions to help. I have been equipped with get::policyengineactions::getpolicyviolations that gets the policy engine violations for a given alias, start date and end date. I will use this function to gather more information.\\n        </scratchpad>\\n        <function_call>get::policyengineactions::getpolicyviolations(alias=\\\"jsmith\\\", startDate=\\\"1st January 2023\\\", endDate=\\\"1st February 2023\\\")</function_call>\\n        <function_result>{response: [{creationDate: \\\"2023-06-01T09:30:00Z\\\", riskLevel: \\\"High\\\", policyId: \\\"POL-001\\\", policyUrl: \\\"https://example.com/policies/POL-001\\\", referenceUrl: \\\"https://example.com/violations/POL-001\\\"}, {creationDate: \\\"2023-06-02T14:45:00Z\\\", riskLevel: \\\"Medium\\\", policyId: \\\"POL-002\\\", policyUrl: \\\"https://example.com/policies/POL-002\\\", referenceUrl: \\\"https://example.com/violations/POL-002\\\"}]}</function_result>\\n        <answer>The policy engine violations between 1st january 2023 to 1st february 2023 for alias jsmith are - Policy ID: POL-001, Policy ID: POL-002</answer>\\n    </example>\\n\\n    <example_docstring>Here is another example that utilizes multiple function calls.</example_docstring>\\n    <example>\\n        <functions>\\n            <function>\\n                <function_name>get::policyengineactions::getpolicyviolations</function_name>\\n                <function_description>Returns a list of policy engine violations for the specified alias within the specified date range.</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <required_argument>startDate (string): The start date of the range to filter violations. The format for startDate is MM/DD/YYYY.</required_argument>\\n                <required_argument>endDate (string): The end date of the range to filter violations</required_argument>\\n                <returns>array: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>post::policyengineactions::acknowledgeviolations</function_name>\\n                <function_description>Acknowledge policy engine violation. Generally used to acknowledge violation, once user notices a violation under their alias or their managers alias.</function_description>\\n                <required_argument>policyId (string): The ID of the policy violation</required_argument>\\n                <required_argument>expectedDateOfResolution (string): The date by when the violation will be addressed/resolved</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>get::activedirectoryactions::getmanager</function_name>\\n                <function_description>This API is used to identify the manager hierarchy above a given person. Every person could have a manager and the manager could have another manager to which they report to</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            \\n        </functions>\\n        <question>Can you check the policy engine violations under my manager between 2nd May to 5th May? My alias is john.</question>\\n        <scratchpad>\\n            To answer this question, I will need to:\\n            1. Get the manager alias of the user using get::activedirectoryactions::getmanager function.\\n            2. Use the returned manager alias to get the policy engine violations using the get::policyengineactions::getpolicyviolations function.\\n\\n            I have double checked and made sure that I have been provided the get::activedirectoryactions::getmanager and the get::policyengineactions::getpolicyviolations functions.\\n        </scratchpad>\\n        <function_call>get::activedirectoryactions::getmanager(alias=\\\"john\\\")</function_call>\\n        <function_result>{response: {managerAlias: \\\"mark\\\", managerLevel: \\\"6\\\", teamName: \\\"Builder\\\", managerName: \\\"Mark Hunter\\\"}}}}</function_result>\\n        <scratchpad>\\n            1. I have the managerAlias from the function results as mark and I have the start and end date from the user input. I can use the function result to call get::policyengineactions::getpolicyviolations function.\\n            2. I will then return the get::policyengineactions::getpolicyviolations function result to the user.\\n\\n            I have double checked and made sure that I have been provided the get::policyengineactions::getpolicyviolations functions.\\n        </scratchpad>\\n        <function_call>get::policyengineactions::getpolicyviolations(alias=\\\"mark\\\", startDate=\\\"2nd May 2023\\\", endDate=\\\"5th May 2023\\\")</function_call>\\n        <function_result>{response: [{creationDate: \\\"2023-05-02T09:30:00Z\\\", riskLevel: \\\"High\\\", policyId: \\\"POL-001\\\", policyUrl: \\\"https://example.com/policies/POL-001\\\", referenceUrl: \\\"https://example.com/violations/POL-001\\\"}, {creationDate: \\\"2023-05-04T14:45:00Z\\\", riskLevel: \\\"Low\\\", policyId: \\\"POL-002\\\", policyUrl: \\\"https://example.com/policies/POL-002\\\", referenceUrl: \\\"https://example.com/violations/POL-002\\\"}]}</function_result>\\n        <answer>\\n            The policy engine violations between 2nd May 2023 to 5th May 2023 for your manager's alias mark are - Policy ID: POL-001, Policy ID: POL-002\\n        </answer>\\n    </example>\\n\\n    <example_docstring>Functions can also be search engine API's that issue a query to a knowledge base. Here is an example that utilizes regular function calls in combination with function calls to a search engine API. Please make sure to extract the source for the information within the final answer when using information returned from the search engine.</example_docstring>\\n    <example>\\n        <functions>\\n            <function>\\n                <function_name>get::benefitsaction::getbenefitplanname</function_name>\\n                <function_description>Get's the benefit plan name for a user. The API takes in a userName and a benefit type and returns the benefit name to the user (i.e. Aetna, Premera, Fidelity, etc.).</function_description>\\n                <optional_argument>userName (string): None</optional_argument>\\n                <optional_argument>benefitType (string): None</optional_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>post::benefitsaction::increase401klimit</function_name>\\n                <function_description>Increases the 401k limit for a generic user. The API takes in only the current 401k limit and returns the new limit.</function_description>\\n                <optional_argument>currentLimit (string): None</optional_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>get::x_amz_knowledgebase_dentalinsurance::search</function_name>\\n                <function_description>This is a search tool that provides information about Delta Dental benefits. It has information about covered dental benefits and other relevant information</function_description>\\n                <required_argument>query(string): A full sentence query that is fed to the search tool</required_argument>\\n                <returns>Returns string  related to the user query asked.</returns>\\n            </function>\\n            <function>\\n                <function_name>get::x_amz_knowledgebase_401kplan::search</function_name>\\n                <function_description>This is a search tool that provides information about Amazon 401k plan benefits. It can determine what a person's yearly 401k contribution limit is, based on their age.</function_description>\\n                <required_argument>query(string): A full sentence query that is fed to the search tool</required_argument>\\n                <returns>Returns string  related to the user query asked.</returns>\\n            </function>\\n            <function>\\n                <function_name>get::x_amz_knowledgebase_healthinsurance::search</function_name>\\n                <function_description>This is a search tool that provides information about Aetna and Premera health benefits. It has information about the savings plan and shared deductible plan, as well as others.</function_description>\\n                <required_argument>query(string): A full sentence query that is fed to the search tool</required_argument>\\n                <returns>Returns string  related to the user query asked.</returns>\\n            </function>\\n            \\n        </functions>\\n\\n        <question>What is my deductible? My username is Bob and my benefitType is Dental. Also, what is the 401k yearly contribution limit?</question>\\n        <scratchpad> I understand I cannot use functions that have not been provided to me to answer this question.\\n            To answer this question, I will:\\n            1. Call the get::benefitsaction::getbenefitplanname function to get the benefit plan name for the user Bob with benefit type Dental.\\n            2. Call the get::x_amz_knowledgebase_dentalinsurance::search function to search for information about deductibles for the plan name returned from step 1.\\n            3. Call the get::x_amz_knowledgebase_401k::search function to search for information about 401k yearly contribution limits.\\n            4. Return the deductible information from the search results to the user.\\n            I have checked that I have access to the get::benefitsaction::getbenefitplanname, x_amz_knowledgebase_dentalinsurance::search, and x_amz_knowledgebase_401k::search functions.\\n        </scratchpad>\\n        <function_call>get::benefitsaction::getbenefitplanname(userName=\\\"Bob\\\", benefitType=\\\"Dental\\\")</function_call>\\n        <function_result>{{'response': {{'planName': 'Delta Dental'}}}}</function_result>\\n        <scratchpad>\\n            I have received the plan name Delta Dental for the user Bob with Dental benefits. I will now call the x_amz_knowledgebase_dentalinsurance::search function to find deductible information for Delta Dental.\\n        </scratchpad>\\n        <function_call>get::x_amz_knowledgebase_dentalinsurance::search(searchQuery=\\\"What is the deductible for Delta Dental?\\\")</function_call>\\n        <function_result>{{'response': {{'responseCode': '200', 'responseBody': \\\"\\\"<answer>\\\\n<answer_part>\\\\n<text>The user's individual deductible is $50 per benefit period</text>\\\\n<source>dfe040f8-46ed-4a65-b3ea-529fa55f6b9e</source>\\\\n</answer_part>\\\\n<answer_part>\\\\n<text>If they are enrolled with dependents, the maximum family deductible is $150 per benefit period.</text>\\\\n<source>0e666064-31d8-4223-b7ba-8eecf40b7b47</source>\\\\n</answer_part>\\\\n</answer>\\\"}}}}</function_result> <scratchpad>\\n            I have found the deductible information for Dental benefits. I will now call the x_amz_knowledgebase_401k::search function to find yearly 401k contribution limits.\\n        </scratchpad>\\n        <function_call>get::x_amz_knowledgebase_401k::search(searchQuery=\\\"What is the yearly 401k contribution limit?\\\")</function_call>\\n        <function_result>{{'response': {{'responseCode': '200', 'responseBody': \\\"<answer>\\\\n<answer_part>\\\\n<text>The yearly 401k contribution limit is $20,500.</text>\\\\n<source>c546cbe8-07f6-45d1-90ca-74d87ab2885a</source>\\\\n</answer_part>\\\\n</answer>\\\"}}}}</function_result>\\n        <answer>\\n            <answer_part>\\n                <text>The deductible for your Delta Dental plan is $50 per benefit period.</text>\\n                <source>dfe040f8-46ed-4a65-b3ea-529fa55f6b9e</source>\\n            </answer_part>\\n            <answer_part>\\n                <text>If you have dependents enrolled, the maximum family deductible is $150 per benefit period.</text>\\n                <source>0e666064-31d8-4223-b7ba-8eecf40b7b47</source>\\n            </answer_part>\\n            <answer_part>\\n                <text>The yearly 401k contribution limit is $20,500.</text>\\n                <source>c546cbe8-07f6-45d1-90ca-74d87ab2885a</source>\\n            </answer_part>\\n        </answer>\\n    </example>\\n\\n    \\n\\n    <example_docstring>Here's a final example where the question asked could not be answered with information gathered from calling the provided functions. In this example, notice how you respond by telling the user you cannot answer, without using a function that was not provided to you.</example_docstring>\\n    <example>\\n        <functions>\\n            <function>\\n                <function_name>get::policyengineactions::getpolicyviolations</function_name>\\n                <function_description>Returns a list of policy engine violations for the specified alias within the specified date range.</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <required_argument>startDate (string): The start date of the range to filter violations. The format for startDate is MM/DD/YYYY.</required_argument>\\n                <required_argument>endDate (string): The end date of the range to filter violations</required_argument>\\n                <returns>array: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>post::policyengineactions::acknowledgeviolations</function_name>\\n                <function_description>Acknowledge policy engine violation. Generally used to acknowledge violation, once user notices a violation under their alias or their managers alias.</function_description>\\n                <required_argument>policyId (string): The ID of the policy violation</required_argument>\\n                <required_argument>expectedDateOfResolution (string): The date by when the violation will be addressed/resolved</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>get::activedirectoryactions::getmanager</function_name>\\n                <function_description>This API is used to identify the manager hierarchy above a given person. Every person could have a manager and the manager could have another manager to which they report to</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            \\n        </functions>\\n        <question>Who are the reportees of David?</question>\\n        <scratchpad>\\n            After reviewing the functions I was equipped with, I realize I am not able to accurately answer this question since I can't access reportees of David. Therefore, I should explain to the user I cannot answer this question.\\n        </scratchpad>\\n        <answer>\\n            Sorry, I am unable to assist you with this request.\\n        </answer>\\n    </example>\\n</examples>\\n\\nThe above examples have been provided to you to illustrate general guidelines and format for use of function calling for information retrieval, and how to use your scratchpad to plan your approach. IMPORTANT: the functions provided within the examples should not be assumed to have been provided to you to use UNLESS they are also explicitly given to you within <functions></functions> tags below. All of the values and information within the examples (the questions, function results, and answers) are strictly part of the examples and have not been provided to you.\\n\\nNow that you have read and understood the examples, I will define the functions that you have available to you to use. Here is a comprehensive list.\\n\\n<functions>\\n<function>\\n<function_name>GET::optimal_departure_window_mars::getNextMarsLaunchWindow</function_name>\\n<function_description>Gets the next optimal launch window to Mars.</function_description>\\n<required_argument>specific_impulse (string): Specific impulse of the propulsion system (s).</required_argument>\\n<required_argument>dry_mass (string): Mass of the spacecraft without fuel (kg).</required_argument>\\n<required_argument>total_mass (string): Total mass of the spacecraft including fuel (kg)</required_argument>\\n<returns>object: The next optimal departure date for a Hohmann transfer from Earth to Mars, based on the spacecraft's mass and specific impulse.</returns>\\n</function>\\n\\n\\n</functions>\\n\\nNote that the function arguments have been listed in the order that they should be passed into the function.\\n\\n\\n\\nDo not modify or extend the provided functions under any circumstances. For example, GET::optimal_departure_window_mars::getNextMarsLaunchWindow with additional parameters would be considered modifying the function which is not allowed. Please use the functions only as defined.\\n\\nDO NOT use any functions that I have not equipped you with.\\n\\n Do not make assumptions about inputs; instead, make sure you know the exact function and input to use before you call a function.\\n\\nTo call a function, output the name of the function in between <function_call> and </function_call> tags. You will receive a <function_result> in response to your call that contains information that you can use to better answer the question. Or, if the function call produced an error, you will receive an <error> in response.\\n\\n\\n\\nThe format for all other <function_call> MUST be: <function_call>$FUNCTION_NAME($FUNCTION_PARAMETER_NAME=$FUNCTION_PARAMETER_VALUE)</function_call>\\n\\nRemember, your goal is to answer the user's question to the best of your ability, using only the function(s) provided within the <functions></functions> tags to gather more information if necessary to better answer the question.\\n\\nDo not modify or extend the provided functions under any circumstances. For example, calling GET::optimal_departure_window_mars::getNextMarsLaunchWindow with additional parameters would be modifying the function which is not allowed. Please use the functions only as defined.\\n\\nBefore calling any functions, create a plan for performing actions to answer this question within the <scratchpad>. Double check your plan to make sure you don't call any functions that you haven't been provided with. Always return your final answer within <answer></answer> tags.\\n\\n\\n\\nThe user input is <question>Answer the following question and pay strong attention to the prompt:\\n        <question>\\n        When is the next launch window for Mars? My spacecraft's total mass is 50000, dry mass is 10000 and specific impulse is 2500. Mass in Kg.\\n        </question>\\n        <instruction>\\n        You have functions available at your disposal to use when anwering any questions about orbital mechanics.if you can't find a function to answer a question about orbital mechanics, simply reply 'I do not know'\\n        </instruction></question>\\n\\n\\nAssistant: <scratchpad> I understand I cannot use functions that have not been provided to me to answer this question.\\n\\n\", \"traceId\": \"e0b2b2c2-fb7c-4e17-8a1f-a3781100face-0\", \"type\": \"ORCHESTRATION\"}, \"event_order\": 2}, {\"type\": \"modelInvocationOutput\", \"data\": {\"metadata\": {\"usage\": {\"inputTokens\": 5160, \"outputTokens\": 135}}, \"rawResponse\": {\"content\": \"To answer this question about the next Mars launch window, I will:\\n\\n1. Call the GET::optimal_departure_window_mars::getNextMarsLaunchWindow function to get the next optimal launch window, passing in the provided spacecraft mass and specific impulse values.\\n\\nI have verified that I have access to the GET::optimal_departure_window_mars::getNextMarsLaunchWindow function.\\n\\n</scratchpad>\\n\\n<function_call>\\nGET::optimal_departure_window_mars::getNextMarsLaunchWindow(specific_impulse=\\\"2500\\\", dry_mass=\\\"10000\\\", total_mass=\\\"50000\\\")\"}, \"traceId\": \"e0b2b2c2-fb7c-4e17-8a1f-a3781100face-0\"}, \"event_order\": 3}, {\"type\": \"rationale\", \"data\": {\"text\": \"To answer this question about the next Mars launch window, I will:\\n\\n1. Call the GET::optimal_departure_window_mars::getNextMarsLaunchWindow function to get the next optimal launch window, passing in the provided spacecraft mass and specific impulse values.\\n\\nI have verified that I have access to the GET::optimal_departure_window_mars::getNextMarsLaunchWindow function.\", \"traceId\": \"e0b2b2c2-fb7c-4e17-8a1f-a3781100face-0\"}, \"event_order\": 4}, {\"type\": \"invocationInput\", \"data\": {\"actionGroupInvocationInput\": {\"actionGroupName\": \"optimal_departure_window_mars\", \"apiPath\": \"/get-next-mars-launch-window\", \"executionType\": \"LAMBDA\", \"parameters\": [{\"name\": \"total_mass\", \"type\": \"string\", \"value\": \"50000\"}, {\"name\": \"dry_mass\", \"type\": \"string\", \"value\": \"10000\"}, {\"name\": \"specific_impulse\", \"type\": \"string\", \"value\": \"2500\"}], \"verb\": \"get\"}, \"invocationType\": \"ACTION_GROUP\", \"traceId\": \"e0b2b2c2-fb7c-4e17-8a1f-a3781100face-0\"}, \"event_order\": 5}, {\"type\": \"observation\", \"data\": {\"actionGroupInvocationOutput\": {\"text\": \"{\\\"next_launch_window\\\": {\\\"next_launch_date\\\": \\\"2026-11-26 00:00:00\\\", \\\"synodic_period_days\\\": 779.9068939794238, \\\"transfer_time_days\\\": 259, \\\"delta_v_available_m_s\\\": 39457.985759929674, \\\"delta_v_required_m_s\\\": 5595.997417810693, \\\"is_feasible\\\": true}}\"}, \"traceId\": \"e0b2b2c2-fb7c-4e17-8a1f-a3781100face-0\", \"type\": \"ACTION_GROUP\"}, \"event_order\": 6}]",
        "mlflow.spanFunctionName": "\"_action_group_trace\"",
        "mlflow.spanInputs": "{\"inner_trace_group\": \"{'actionGroupName': 'optimal_departure_window_mars', 'apiPath': '/get-next-mars-launch-window', 'executionType': 'LAMBDA', 'parameters': [{'name': 'total_mass', 'type': 'string', 'value': '50000'}, {'name': 'dry_mass', 'type': 'string', 'value': '10000'}, {'name': 'specific_impulse', 'type': 'string', 'value': '2500'}], 'verb': 'get'}\"}",
        "mlflow.spanOutputs": "\"{'action_group_name': 'optimal_departure_window_mars', 'api_path': '/get-next-mars-launch-window', 'execution_type': 'LAMBDA', 'execution_output': '{\\\"next_launch_window\\\": {\\\"next_launch_date\\\": \\\"2026-11-26 00:00:00\\\", \\\"synodic_period_days\\\": 779.9068939794238, \\\"transfer_time_days\\\": 259, \\\"delta_v_available_m_s\\\": 39457.985759929674, \\\"delta_v_required_m_s\\\": 5595.997417810693, \\\"is_feasible\\\": true}}'}\""
      },
      "events": []
    },
    {
      "name": "Retrieved Response",
      "context": {
        "span_id": "0xfe0b5f9149c39d7d",
        "trace_id": "0x9b8bd0b2e018d77f936e48a09e54fd44"
      },
      "parent_id": "0xb802165d133a33aa",
      "start_time": 1731388550225320000,
      "end_time": 1731388550226466000,
      "status_code": "OK",
      "status_message": "",
      "attributes": {
        "mlflow.traceRequestId": "\"1e036cc3a7f946ec995f7763b8dde51c\"",
        "mlflow.spanType": "\"AGENT\"",
        "mlflow.spanInputs": "[{\"role\": \"user\", \"content\": \"When is the next launch window for Mars? My spacecraft's total mass is 50000, dry mass is 10000 and specific impulse is 2500. Mass in Kg.\", \"name\": null}]",
        "mlflow.spanOutputs": "{\"choices\": [{\"index\": 0, \"message\": {\"role\": \"user\", \"content\": \"Based on the provided spacecraft dry mass of 10000 kg, total mass of 50000 kg, and specific impulse of 2500 s, the next optimal launch window for a Hohmann transfer from Earth to Mars is on November 26, 2026 UTC. The transfer will take 259 days.\", \"name\": null}, \"finish_reason\": \"stop\", \"logprobs\": null}], \"usage\": {\"prompt_tokens\": null, \"completion_tokens\": null, \"total_tokens\": null}, \"id\": null, \"model\": \"anthropic.claude-v2\", \"object\": \"chat.completion\", \"created\": 1731388550}"
      },
      "events": []
    }
  ],
  "request": "{\"context\": \"<mlflow.pyfunc.model.PythonModelContext object at 0x13397c530>\", \"messages\": [{\"role\": \"user\", \"content\": \"When is the next launch window for Mars? My spacecraft's total mass is 50000, dry mass is 10000 and specific impulse is 2500. Mass in Kg.\", \"name\": null}], \"params\": {\"temperature\": 1.0, \"max_tokens\": null, \"stop\": null, \"n\": 1, \"stream\": false, \"top_p\": null, \"top_k\": null, \"frequency_penalty\": null, \"presence_penalty\": null}}",
  "response": "{\"choices\": [{\"index\": 0, \"message\": {\"role\": \"user\", \"content\": \"Based on the provided spacecraft dry mass of 10000 kg, total mass of 50000 kg, and specific impulse of 2500 s, the next optimal launch window for a Hohmann transfer from Earth to Mars is on November 26, 2026 UTC. The transfer will take 259 days.\", \"name\": null}, \"finish_reason\": \"stop\", \"logprobs\": null}], \"usage\": {\"prompt_tokens\": null, \"completion_tokens\": null, \"total_tokens\": null}, \"id\": null, \"model\": \"anthropic.claude-v2\", \"object\": \"chat.completion\", \"created\": 1731388550}"
}

在 MLflow UI 中可视化追踪细分

1. 提交给 Bedrock Agent 的初始提示。

2. 在此追踪中，我们可以观察 Bedrock Agent 如何评估并选择最适合当前任务的操作组。

3. 一旦选择了操作组，就会对其调用进行追踪，显示与底层 Lambda 函数的输入和输出交互，如上述 OpenAPI 规范所概述。

4. 此外，Bedrock 的补充追踪包含在“属性”部分中，以及其他元数据，如下所示：

5. 随后，代理的最终响应被追踪，如下图所示。

注意：我们无法将 span 的持续时间分解为单个追踪持续时间，因为 Bedrock Agent 的追踪响应不包含每个追踪步骤的时间戳。

结论

在这篇博客中，我们探讨了如何将 AWS Bedrock Agent 作为 MLflow ChatModel 集成，重点关注操作组、知识库和追踪。我们展示了如何使用 MLflow 灵活而强大的 API 轻松构建自定义 ChatModel。这种方法使您能够利用 MLflow 的追踪和日志记录功能，即使对于 MLflow 不原生支持的模型或风格也是如此。

本博客的要点

• 使用操作组将 Bedrock Agent 部署为 AWS Lambda 函数
  • 我们介绍了如何设置 Bedrock Agent 并使用操作组中的 AWS Lambda 函数实现自定义操作。
• 将 AWS Bedrock Agent 的自定义追踪映射到 MLflow span/trace 对象
  • 我们演示了如何将代理的自定义追踪数据转换为 MLflow span 对象，以提高可观察性。
• 将 Bedrock Agent 作为 MLflow ChatModel 记录和加载
  • 我们展示了如何将 Bedrock Agent 记录到 MLflow 中作为 ChatModel，以及如何加载它以备将来使用。
• 外部化 AWS 客户端和 Bedrock 配置
  • 我们解释了如何外部化 AWS 客户端和 Bedrock 配置，以保护秘密并方便调整模型设置，而无需重新记录模型。

延伸阅读和参考资料

• Amazon Bedrock Agent 的工作原理
• Amazon Bedrock 追踪
• 创建自定义 GenAI 聊天代理
• AWS 代码示例库