将 Bedrock Agent 用作带有跟踪功能的 MLflow ChatModel

在这篇博文中，我们将深入探讨如何将 AWS Bedrock Agent 集成到 MLflow 中作为 ChatModel 使用，重点介绍如何利用 Bedrock 的 Action Groups 和 Knowledge Bases 构建会话式 AI 应用程序。博文将指导您设置 Bedrock Agent，配置 Action Groups 以通过 Lambda 实现自定义操作，并利用知识库进行上下文感知交互。文中特别强调了在 MLflow 中实现跟踪。阅读本文后，您将对如何将 AWS Bedrock 的高级功能与 MLflow 的能力（例如 Agent 请求跟踪、模型跟踪以及输入示例的一致签名）结合使用有深入的了解。

什么是 AWS Bedrock？

Amazon Bedrock 是 AWS 提供的一项托管服务，可简化生成式 AI 应用程序的开发。它通过单一 API 提供对来自领先 AI 提供商的各种基础模型 (FM) 的访问，使开发人员能够安全高效地构建和扩展 AI 解决方案。

与此集成相关的关键组件

Bedrock Agent：从宏观上看，Bedrock Agent 是 Bedrock 中的一种抽象，由基础模型、Action Groups 和 Knowledge Bases 组成。

Action Groups：这是一组可自定义的操作集合，定义了 Bedrock Agent 可以执行的任务。Action Groups 由一个 OpenAPI Schema 和用于执行工具调用的相应 Lambda 函数组成。OpenAPI Schema 用于定义 Agent 可以调用和完成任务的可用 API。

Knowledge Bases：Amazon Bedrock 支持创建 Knowledge Bases 来实现检索增强生成工作流。它包含数据源（在 S3 或网页上）和一个包含数据嵌入引用的向量存储。

Bedrock Agent 的执行过程和相应的 Agent 插装跟踪分组如下

预处理：此步骤验证用户输入，并对其进行上下文化和分类。

编排：此步骤处理用户输入的解释，决定何时执行哪些任务，并迭代地完善响应。

后处理（可选）：此步骤在将最终响应返回给用户之前对其进行格式化。

跟踪：上述每个步骤都有一个执行跟踪，其中包含 Agent 响应每个步骤的推理、操作、查询和观察。这包括 Action Groups 和 Knowledge Base 查询的输入和输出。

我们将在下面详细查看这些跟踪。

MLflow 中的 ChatModel 是什么？

ChatModel 类专门设计用于更轻松地实现与流行的大型语言模型 (LLM) 聊天 API 兼容的模型。它使您能够无缝地引入自己的模型或 Agent 并利用 MLflow 的功能，即使这些模型不作为 MLflow 中的原生 Flavor 受支持。此外，它还提供默认签名，与 PythonModel 不同，ChatModel 的签名是静态的。

在以下部分，我们将使用 ChatModel 封装 Bedrock Agent。

有关 ChatModel 的更多详细信息，您可以阅读 MLflow 文档 此处 和 此处

设置带有 Action Group 的 AWS Bedrock Agent

在本节中，我们将部署 Bedrock Agent 的所有组件，以便我们可以在 MLflow 中将其作为 ChatModel 调用。

前提条件

您需要设置以下各项（可通过 AWS 控制台或 SDKs）：

• 为 Agent 和 Lambda 函数设置角色。示例
• 创建/部署 Agent。示例
  • 重要：在此处保存 Agent ID，稍后我们将需要它。
• 创建 Lambda 函数。示例
• 配置 Agent-Lambda 交互的 IAM 权限。示例 和 示例
• 创建一个 Action Group 来链接 Agent 和 Lambda。示例
  • 重要：保存此处的 Agent 别名 ID，稍后我们将需要它。
• 使用别名部署 Bedrock Agent。示例

在我们的示例中，我们将部署以下示例 Action Group，它根据航天器的质量和比冲量计算从地球到火星霍曼转移的下一个最佳出发日期。

Action Groups 的 OpenAPI Schema

如上所述，这是我们的示例 Action Group 的 OpenAPI Schema

openapi: 3.0.0
info:
  title: Time API
  version: 1.0.0
  description: API to get the next optimal departure date for a Hohmann transfer from Earth to Mars.
paths:
  /get-next-mars-launch-window:
    get:
      summary: Gets the next optimal launch window to Mars.
      description: Gets the next optimal launch window to Mars.
      operationId: getNextMarsLaunchWindow
      parameters:
        - name: total_mass
          in: query
          description: Total mass of the spacecraft including fuel (kg)
          required: true
          schema:
            type: string
        - name: dry_mass
          in: query
          description: Mass of the spacecraft without fuel (kg).
          required: true
          schema:
            type: string
        - name: specific_impulse
          in: query
          description: Specific impulse of the propulsion system (s).
          required: true
          schema:
            type: string
      responses:
        "200":
          description: The next optimal departure date for a Hohmann transfer from Earth to Mars, based on the spacecraft's mass and specific impulse.
          content:
            "application/json":
              schema:
                type: object
                properties:
                  next_launch_window:
                    type: string
                    description: Next Mars Launch Window

Action Groups - Lambda 函数

这是 Action Group 示例 Lambda 的代码部署

import json
import math
from datetime import datetime, timedelta


def lambda_handler(event, context):
    def _calculate_optimal_departure_window(
        total_mass, dry_mass, specific_impulse
    ):
        """
        Calculate the next optimal departure date for a Hohmann transfer from Earth to Mars,
        based on the spacecraft's mass and specific impulse.

        Parameters:
        - total_mass (float): Total mass of the spacecraft including fuel (kg).
        - dry_mass (float): Mass of the spacecraft without fuel (kg).
        - specific_impulse (float): Specific impulse of the propulsion system (s).

        Returns:
        - dict: {
            'next_launch_date': datetime,
            'synodic_period_days': float,
            'transfer_time_days': float,
            'delta_v_available_m_s': float,
            'delta_v_required_m_s': float,
            'is_feasible': bool
          }
        """
        current_date = None
        # Constants
        G0 = 9.80665  # m/s^2, standard gravity
        MU_SUN = (
            1.32712440018e20  # m^3/s^2, standard gravitational parameter for the Sun
        )
        AU = 1.496e11  # meters, astronomical unit
        EARTH_ORBITAL_PERIOD = 365.25  # days
        MARS_ORBITAL_PERIOD = 686.98  # days
        SYNODIC_PERIOD = 1 / abs((1 / EARTH_ORBITAL_PERIOD) - (1 / MARS_ORBITAL_PERIOD))
        TRANSFER_TIME = 259  # days, approximate duration of Hohmann transfer
        BASE_LAUNCH_DATE = datetime(2020, 7, 1)  # A reference past launch window date

        # Orbital Radii (assuming circular orbits for simplicity)
        r1 = AU  # Earth's orbital radius in meters
        r2 = 1.524 * AU  # Mars' orbital radius in meters

        # Calculate Required Delta-V for Hohmann Transfer
        # Using vis-viva equation for Hohmann transfer
        def calculate_hohmann_delta_v(mu, r_start, r_end):
            # Velocity of departure orbit (Earth)
            v_start = math.sqrt(mu / r_start)
            # Velocity of transfer orbit at departure
            a_transfer = (r_start + r_end) / 2
            v_transfer_start = math.sqrt(mu * (2 / r_start - 1 / a_transfer))
            delta_v1 = v_transfer_start - v_start

            # Velocity of arrival orbit (Mars)
            v_end = math.sqrt(mu / r_end)
            # Velocity of transfer orbit at arrival
            v_transfer_end = math.sqrt(mu * (2 / r_end - 1 / a_transfer))
            delta_v2 = v_end - v_transfer_end

            return delta_v1, delta_v2

        delta_v1, delta_v2 = calculate_hohmann_delta_v(MU_SUN, r1, r2)
        delta_v_required = abs(delta_v1) + abs(delta_v2)  # Total delta-v in m/s

        # Delta-V using Tsiolkovsky Rocket Equation
        if dry_mass <= 0 or total_mass <= dry_mass:
            raise ValueError("Total mass must be greater than dry mass.")

        delta_v_available = (
            specific_impulse * G0 * math.log(total_mass / dry_mass)
        )  # m/s

        is_feasible = delta_v_available >= delta_v_required

        if current_date is None:
            current_date = datetime.now()

        days_since_base = (current_date - BASE_LAUNCH_DATE).days
        if days_since_base < 0:
            # Current date is before the base launch date
            next_launch_date = BASE_LAUNCH_DATE
        else:
            synodic_periods_passed = days_since_base / SYNODIC_PERIOD
            synodic_periods_passed_int = math.floor(synodic_periods_passed)
            next_launch_date = BASE_LAUNCH_DATE + timedelta(
                days=(synodic_periods_passed_int + 1) * SYNODIC_PERIOD
            )

        next_launch_date = next_launch_date.replace(
            hour=0, minute=0, second=0, microsecond=0
        )

        return {
            "next_launch_date": next_launch_date,
            "synodic_period_days": SYNODIC_PERIOD,
            "transfer_time_days": TRANSFER_TIME,
            "delta_v_available_m_s": delta_v_available,
            "delta_v_required_m_s": delta_v_required,
            "is_feasible": is_feasible,
        }

    query_params = {
        event["name"]: event["value"] for event in event.get("parameters", [])
    }

    total_mass = float(query_params.get("total_mass"))
    dry_mass = float(query_params.get("dry_mass"))
    specific_impulse = float(query_params.get("specific_impulse"))

    response = {
        "next_launch_window": _calculate_optimal_departure_window(
            total_mass, dry_mass, specific_impulse
        )
    }

    response_body = {"application/json": {"body": json.dumps(response)}}

    action_response = {
        "actionGroup": event["actionGroup"],
        "apiPath": event["apiPath"],
        "httpMethod": event["httpMethod"],
        "httpStatusCode": 200,
        "responseBody": response_body,
    }

    session_attributes = event["sessionAttributes"]
    prompt_session_attributes = event["promptSessionAttributes"]

    return {
        "messageVersion": "1.0",
        "response": action_response,
        "sessionAttributes": session_attributes,
        "promptSessionAttributes": prompt_session_attributes,
    }

接下来，我们将把 Bedrock Agent 封装成 ChatModel，以便我们可以注册并加载它进行推理。

为 Bedrock Agent 编写 ChatModel

以下是在 Python 3.12.7 中本地运行以下示例使用的顶级包

boto3==1.35.31
mlflow==2.16.2

将 Bedrock Agent 实现为带有跟踪功能的 MLflow ChatModel

import copy
import os
import uuid
from typing import List, Optional

import boto3
import mlflow
from botocore.config import Config
from mlflow.entities import SpanType
from mlflow.pyfunc import ChatModel
from mlflow.types.llm import ChatResponse, ChatMessage, ChatParams, ChatChoice


class BedrockModel(ChatModel):
    def __init__(self):
        """
        Initializes the BedrockModel instance with placeholder values.

        Note:
            The `load_context` method cannot create new instance variables; it can only modify existing ones.
            Therefore, all instance variables should be defined in the `__init__` method with placeholder values.
        """
        self.brt = None
        self._main_bedrock_agent = None
        self._bedrock_agent_id = None
        self._bedrock_agent_alias_id = None
        self._inference_configuration = None
        self._agent_instruction = None
        self._model = None
        self._aws_region = None

    def __getstate__(self):
        """
        Prepares the instance state for pickling.

        This method is needed because the `boto3` client (`self.brt`) cannot be pickled.
        By excluding `self.brt` from the state, we ensure that the model can be serialized and deserialized properly.
        """
        # Create a dictionary of the instance's state, excluding the boto3 client
        state = self.__dict__.copy()
        del state["brt"]
        return state

    def __setstate__(self, state):
        """
        Restores the instance state during unpickling.

        This method is needed to reinitialize the `boto3` client (`self.brt`) after the instance is unpickled,
        because the client was excluded during pickling.
        """
        self.__dict__.update(state)
        self.brt = None

    def load_context(self, context):
        """
        Initializes the Bedrock client with AWS credentials.

        Args:
            context: The MLflow context containing model configuration.

        Note:
            Dependent secret variables must be in the execution environment prior to loading the model;
            else they will not be available during model initialization.
        """
        self._main_bedrock_agent = context.model_config.get("agents", {}).get(
            "main", {}
        )
        self._bedrock_agent_id = self._main_bedrock_agent.get("bedrock_agent_id")
        self._bedrock_agent_alias_id = self._main_bedrock_agent.get(
            "bedrock_agent_alias_id"
        )
        self._inference_configuration = self._main_bedrock_agent.get(
            "inference_configuration"
        )
        self._agent_instruction = self._main_bedrock_agent.get("instruction")
        self._model = self._main_bedrock_agent.get("model")
        self._aws_region = self._main_bedrock_agent.get("aws_region")

        # Initialize the Bedrock client
        self.brt = boto3.client(
            service_name="bedrock-agent-runtime",
            config=Config(region_name=self._aws_region),
            aws_access_key_id=os.environ["AWS_ACCESS_KEY"],
            aws_secret_access_key=os.environ["AWS_SECRET_ACCESS_KEY"],
            aws_session_token=os.environ["AWS_SESSION_TOKEN"],
            region_name=self._aws_region,
        )

    @staticmethod
    def _extract_trace_groups(events):
        """
        Extracts trace groups from a list of events based on their trace IDs.

        Args:
            events (list): A list of event dictionaries.

        Returns:
            dict: A dictionary where keys are trace IDs and values are lists of trace items.
        """
        from collections import defaultdict

        trace_groups = defaultdict(list)

        def find_trace_ids(obj, original_trace, depth=0, parent_key=None):
            if depth > 5:
                return  # Stop recursion after 5 levels if no traceId has been found
            if isinstance(obj, dict):
                trace_id = obj.get("traceId")
                if trace_id:
                    # Include the parent key as the 'type'
                    item = {
                        "type": parent_key,
                        "data": obj,
                        "event_order": original_trace.get("trace", {}).get(
                            "event_order"
                        ),
                    }
                    trace_groups[trace_id].append(item)
                else:
                    for key, value in obj.items():
                        find_trace_ids(
                            value, original_trace, depth=depth + 1, parent_key=key
                        )
            elif isinstance(obj, list):
                for item in obj:
                    find_trace_ids(item, item, depth=depth + 1, parent_key=parent_key)

        find_trace_ids(events, {})
        return dict(trace_groups)

    @staticmethod
    def _get_final_response_with_trace(trace_id_groups: dict[str, list[dict]]):
        """
        Processes trace groups to extract the final response and create relevant MLflow spans.

        Args:
            trace_id_groups (dict): A dictionary of trace groups keyed by trace IDs.

        Returns:
            str: The final response text extracted from the trace groups.
        """
        trace_id_groups_copy = copy.deepcopy(trace_id_groups)
        model_invocation_input_key = "modelInvocationInput"

        def _create_trace_by_type(
            trace_name, _trace_id, context_input, optional_rationale_subtrace
        ):
            @mlflow.trace(
                name=trace_name,
                attributes={"trace_attributes": trace_id_groups[_trace_id]},
            )
            def _trace_agent_pre_context(inner_input_trace):
                return optional_rationale_subtrace.get("data", {}).get("text")

            trace_id_groups_copy[_trace_id].remove(context_input)
            _trace_agent_pre_context(context_input.get("data", {}).get("text"))

        def _extract_action_group_trace(
            _trace_id, trace_group, action_group_invocation_input: dict
        ):
            @mlflow.trace(
                name="action-group-invocation",
                attributes={"trace_attributes": trace_id_groups[_trace_id]},
            )
            def _action_group_trace(inner_trace_group):
                for _trace in trace_group:
                    action_group_invocation_output = _trace.get("data", {}).get(
                        "actionGroupInvocationOutput"
                    )
                    if action_group_invocation_output is not None:
                        action_group_response = str(
                            {
                                "action_group_name": action_group_invocation_input.get(
                                    "actionGroupName"
                                ),
                                "api_path": action_group_invocation_input.get(
                                    "apiPath"
                                ),
                                "execution_type": action_group_invocation_input.get(
                                    "executionType"
                                ),
                                "execution_output": action_group_invocation_output.get(
                                    "text"
                                ),
                            }
                        )
                        trace_group.remove(_trace)
                        return action_group_response

            _action_group_trace(str(action_group_invocation_input))

        def _extract_knowledge_base_trace(
            _trace_id, trace_group, knowledge_base_lookup_input
        ):
            @mlflow.trace(
                name="knowledge-base-lookup",
                attributes={"trace_attributes": trace_id_groups[_trace_id]},
            )
            def _knowledge_base_trace(inner_trace_group):
                for _trace in trace_group:
                    knowledge_base_lookup_output = _trace.get("data", {}).get(
                        "knowledgeBaseLookupOutput"
                    )
                    if knowledge_base_lookup_output is not None:
                        knowledge_base_response = str(
                            {
                                "knowledge_base_id": knowledge_base_lookup_input.get(
                                    "knowledgeBaseId"
                                ),
                                "text": knowledge_base_lookup_input.get("text"),
                                "retrieved_references": knowledge_base_lookup_output.get(
                                    "retrievedReferences"
                                ),
                            }
                        )
                        trace_group.remove(_trace)
                        return knowledge_base_response

            _knowledge_base_trace(str(trace_group))

        def _trace_group_type(
            _trace_id, trace_group, _trace, optional_rationale_subtrace
        ):
            trace_name = "observation"
            pre_processing_trace_id_suffix = "-pre"
            if pre_processing_trace_id_suffix in _trace_id:
                trace_name = "agent-initial-context"
            else:
                for _inner_trace in trace_group:
                    action_group_invocation_input = _inner_trace.get("data", {}).get(
                        "actionGroupInvocationInput"
                    )
                    if action_group_invocation_input is not None:
                        action_group_name = action_group_invocation_input.get(
                            "actionGroupName"
                        )
                        trace_name = f"ACTION-GROUP-{action_group_name}"
                        _create_trace_by_type(
                            trace_name, _trace_id, _trace, optional_rationale_subtrace
                        )
                        _extract_action_group_trace(
                            _trace_id, trace_group, action_group_invocation_input
                        )
                        trace_group.remove(_trace)
                    knowledge_base_lookup_input = _inner_trace.get("data", {}).get(
                        "knowledgeBaseLookupInput"
                    )
                    if knowledge_base_lookup_input is not None:
                        knowledge_base_id = knowledge_base_lookup_input.get(
                            "knowledgeBaseId"
                        )
                        trace_name = f"KNOWLEDGE_BASE_{knowledge_base_id}"
                        _create_trace_by_type(
                            trace_name, _trace_id, _trace, optional_rationale_subtrace
                        )
                        _extract_knowledge_base_trace(
                            _trace_id, trace_group, knowledge_base_lookup_input
                        )
                        trace_group.remove(_trace)
            return trace_name

        for _trace_id, _trace_group in trace_id_groups_copy.items():
            trace_group = sorted(_trace_group, key=lambda tg: tg["event_order"])
            model_invocation_input_subtrace = None
            optional_rationale_subtrace = None
            for _trace in _trace_group:
                if model_invocation_input_key == _trace.get("type", ""):
                    model_invocation_input_subtrace = _trace
                elif "rationale" == _trace.get("type", ""):
                    optional_rationale_subtrace = _trace
            _trace_group_type(
                _trace_id,
                trace_group,
                model_invocation_input_subtrace,
                optional_rationale_subtrace,
            )

        final_response = (
            list(trace_id_groups_copy.values())[-1][-1]
            .get("data", {})
            .get("finalResponse", {})
            .get("text")
        )
        return final_response

    @mlflow.trace(name="Bedrock Input Prompt")
    def _get_agent_prompt(self, raw_input_question):
        """
        Constructs the agent prompt by combining the input question and the agent instruction.

        Args:
            raw_input_question (str): The user's input question.

        Returns:
            str: The formatted agent prompt.
        """
        return f"""
        Answer the following question and pay strong attention to the prompt:
        <question>
        {raw_input_question}
        </question>
        <instruction>
        {self._agent_instruction}
        </instruction>
        """

    @mlflow.trace(name="bedrock-agent", span_type=SpanType.CHAT_MODEL)
    def predict(
        self, context, messages: List[ChatMessage], params: Optional[ChatParams]
    ) -> ChatResponse:
        """
        Makes a prediction using the Bedrock agent and processes the response.

        Args:
            context: The MLflow context.
            messages (List[ChatMessage]): A list of chat messages.
            params (Optional[ChatParams]): Optional parameters for the chat.

        Returns:
            ChatResponse: The response from the Bedrock agent.
        """
        formatted_input = messages[-1].content
        session_id = uuid.uuid4().hex

        response = self.brt.invoke_agent(
            agentId=self._bedrock_agent_id,
            agentAliasId=self._bedrock_agent_alias_id,
            inputText=self._get_agent_prompt(formatted_input),
            enableTrace=True,
            sessionId=session_id,
            endSession=False,
        )

        # Since this provider's output doesn't match the OpenAI specification,
        # we need to go through the returned trace data and map it appropriately
        # to create the MLflow span object.
        events = []
        for index, event in enumerate(response.get("completion", [])):
            if "trace" in event:
                event["trace"]["event_order"] = index
            events.append(event)
        trace_id_groups = self._extract_trace_groups(events)
        final_response = self._get_final_response_with_trace(trace_id_groups)
        with mlflow.start_span(
            name="retrieved-response", span_type=SpanType.AGENT
        ) as span:
            span.set_inputs(messages)
            span.set_attributes({})

            output = ChatResponse(
                choices=[
                    ChatChoice(
                        index=0,
                        message=ChatMessage(role="user", content=final_response),
                    )
                ],
                usage={},
                model=self._model,
            )

            span.set_outputs(output)

        return output

关于这个 BedrockModel 实现的一些重要说明

• AWS 访问密钥 ID、秘密访问密钥和会话令牌在此处进行了外部化。在运行推理之前，这些信息需要在环境中存在。您需要为您的 IAM 用户生成它们并将其设置为环境变量。

aws sts get-session-token --duration-seconds 3600

然后设置以下内容

import os

os.environ['AWS_ACCESS_KEY'] = "<AccessKeyId>"
os.environ['AWS_SECRET_ACCESS_KEY'] = "<SecretAccessKey>"
os.environ['AWS_SESSION_TOKEN'] = "<SessionToken>"

正如上面的代码所示，这些信息不会随模型一起记录，只在 load_context 中设置。此方法在构造 ChatModel 时调用。更多详细信息请见 此处

• Bedrock Agent ID 和 Agent 别名 ID 通过 model_config 传递，我们将在下面使用它。

• boto3 模块已被排除在 pickle 之外。这是通过 __getstate__ 和 __setstate__ 方法实现的，我们分别在其中排除和重置它。

记录并加载 BedrockModel

import mlflow
from mlflow.models import infer_signature

input_example = [
    {
        "messages": [
            {
                "role": "user",
                "content": "When is the next launch window for Mars?",
            }
        ]
    }
]

output_example = {
    "choices": [
        {
            "index": 0,
            "finish_reason": "stop",
            "message": {"role": "assistant", "content": "test content"},
        }
    ]
}
signature = infer_signature(input_example, output_example)

with mlflow.start_run():

    model_config = {
        "agents": {
            "main": {
                "model": "anthropic.claude-v2",
                "aws_region": "us-east-1",
                "bedrock_agent_id": "O9KQSEVEFF",
                "bedrock_agent_alias_id": "3WHEEJKNUT",
                "instruction": (
                    "You have functions available at your disposal to use when anwering any questions about orbital mechanics."
                    "if you can't find a function to answer a question about orbital mechanics, simply reply "
                    "'I do not know'"
                ),
                "inference_configuration": {
                    "temperature": 0.5,
                    "maximumLength": 2000,
                },
            },
        },
    }

    # Input example for the model
    input_example = {
        "messages": [
            {
                "role": "user",
                "content": "When is the next launch window for Mars? My spacecraft's total mass is 50000, dry mass is 10000 and specific impulse is 2500. Mass in Kg.",
            }
        ]
    }

    # Log and load the model using MLflow
    logged_chain_info = mlflow.pyfunc.log_model(
        python_model=BedrockModel(),
        model_config=model_config,
        artifact_path="chain",  # This string is used as the path inside the MLflow model where artifacts are stored
        input_example=input_example,  # Must be a valid input to your chain
    )

loaded = mlflow.pyfunc.load_model(logged_chain_info.model_uri)

# Predict using the loaded model
response = loaded.predict(
    {
        "messages": [
            {
                "role": "user",
                "content": "When is the next launch window for Mars? My spacecraft's total mass is 50000, dry mass is 10000 and specific impulse is 2500. Mass in Kg.",
            }
        ]
    }
)
print(response)

将 Bedrock Agent 跟踪数据映射到 MLflow Span 对象

在此步骤中，我们需要遍历 Bedrock Agent 响应跟踪中返回的数据，以提供相关的映射来创建 MLflow span 对象。AWS Bedrock Agent 的响应是一个平面列表，其中跟踪事件通过 traceId 连接。以下是 Bedrock Agent 响应中发送的原始跟踪。

展开查看 AWS Bedrock Agent 的原始跟踪

[
  {
    'trace': {
      'agentAliasId': '3WHEEJKNUT',
      'agentId': 'O9KQSEVEFF',
      'agentVersion': '1',
      'event_order': 0,
      'sessionId': '9566a6d78551434fb0409578ffed63c1',
      'trace': {
        'preProcessingTrace': {
          'modelInvocationInput': {
            'inferenceConfiguration': {
              ...
            },
            'text': '\n\nHuman: You are a classifying agent that filters user inputs into categories. Your job is to sort these inputs before they...<thinking> XML tags before providing only the category letter to sort the input into within <category> XML tags.\n\nAssistant:',
            'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-pre-0',
            'type': 'PRE_PROCESSING'
          }
        }
      }
    }
  },
  {
    'trace': {
      'agentAliasId': '3WHEEJKNUT',
      'agentId': 'O9KQSEVEFF',
      'agentVersion': '1',
      'event_order': 1,
      'sessionId': '9566a6d78551434fb0409578ffed63c1',
      'trace': {
        'preProcessingTrace': {
          'modelInvocationOutput': {
            'parsedResponse': {
              ...
            },
            'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-pre-0'
          }
        }
      }
    }
  },
  {
    'trace': {
      'agentAliasId': '3WHEEJKNUT',
      'agentId': 'O9KQSEVEFF',
      'agentVersion': '1',
      'event_order': 2,
      'sessionId': '9566a6d78551434fb0409578ffed63c1',
      'trace': {
        'orchestrationTrace': {
          'modelInvocationInput': {
            'inferenceConfiguration': {
              ...
            },
            'text': '\n\nHuman:\nYou are a research assistant AI that has been equipped with one or more functions to help you answer a <question>...\n\nAssistant: <scratchpad> I understand I cannot use functions that have not been provided to me to answer this question.\n\n',
            'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-0',
            'type': 'ORCHESTRATION'
          }
        }
      }
    }
  },
  {
    'trace': {
      'agentAliasId': '3WHEEJKNUT',
      'agentId': 'O9KQSEVEFF',
      'agentVersion': '1',
      'event_order': 3,
      'sessionId': '9566a6d78551434fb0409578ffed63c1',
      'trace': {
        'orchestrationTrace': {
          'modelInvocationOutput': {
            'metadata': {
              ...
            },
            'rawResponse': {
              ...
            },
            'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-0'
          }
        }
      }
    }
  },
  {
    'trace': {
      'agentAliasId': '3WHEEJKNUT',
      'agentId': 'O9KQSEVEFF',
      'agentVersion': '1',
      'event_order': 4,
      'sessionId': '9566a6d78551434fb0409578ffed63c1',
      'trace': {
        'orchestrationTrace': {
          'rationale': {
            'text': 'To answer this question about the next Mars launch window, I will:\n\n1. Call the GET::optimal_departure_window_mars::getNext...lse values.\n\nI have verified that I have access to the GET::optimal_departure_window_mars::getNextMarsLaunchWindow function.',
            'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-0'
          }
        }
      }
    }
  },
  {
    'trace': {
      'agentAliasId': '3WHEEJKNUT',
      'agentId': 'O9KQSEVEFF',
      'agentVersion': '1',
      'event_order': 5,
      'sessionId': '9566a6d78551434fb0409578ffed63c1',
      'trace': {
        'orchestrationTrace': {
          'invocationInput': {
            'actionGroupInvocationInput': {
              ...
            },
            'invocationType': 'ACTION_GROUP',
            'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-0'
          }
        }
      }
    }
  },
  {
    'trace': {
      'agentAliasId': '3WHEEJKNUT',
      'agentId': 'O9KQSEVEFF',
      'agentVersion': '1',
      'event_order': 6,
      'sessionId': '9566a6d78551434fb0409578ffed63c1',
      'trace': {
        'orchestrationTrace': {
          'observation': {
            'actionGroupInvocationOutput': {
              ...
            },
            'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-0',
            'type': 'ACTION_GROUP'
          }
        }
      }
    }
  },
  {
    'trace': {
      'agentAliasId': '3WHEEJKNUT',
      'agentId': 'O9KQSEVEFF',
      'agentVersion': '1',
      'event_order': 7,
      'sessionId': '9566a6d78551434fb0409578ffed63c1',
      'trace': {
        'orchestrationTrace': {
          'modelInvocationInput': {
            'inferenceConfiguration': {
              ...
            },
            'text': '\n\nHuman:\nYou are a research assistant AI that has been equipped with one or more functions to help you answer a <question>...lta_v_available_m_s": 39457.985759929674, "delta_v_required_m_s": 5595.997417810693, "is_feasible": true}}</function_result>\n',
            'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-1',
            'type': 'ORCHESTRATION'
          }
        }
      }
    }
  },
  {
    'trace': {
      'agentAliasId': '3WHEEJKNUT',
      'agentId': 'O9KQSEVEFF',
      'agentVersion': '1',
      'event_order': 8,
      'sessionId': '9566a6d78551434fb0409578ffed63c1',
      'trace': {
        'orchestrationTrace': {
          'modelInvocationOutput': {
            'metadata': {
              ...
            },
            'rawResponse': {
              ...
            },
            'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-1'
          }
        }
      }
    }
  },
  {
    'trace': {
      'agentAliasId': '3WHEEJKNUT',
      'agentId': 'O9KQSEVEFF',
      'agentVersion': '1',
      'event_order': 9,
      'sessionId': '9566a6d78551434fb0409578ffed63c1',
      'trace': {
        'orchestrationTrace': {
          'observation': {
            'finalResponse': {
              ...
            },
            'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-1',
            'type': 'FINISH'
          }
        }
      }
    }
  },
  {
    'chunk': {
      'bytes': b
      'Based on the provided spacecraft dry mass of 10000 kg, total mass of 50000 kg, and specific impulse of 2500 s, the next optimal launch window for a Hohmann transfer from Earth to Mars is on November 26, 2026 UTC. The transfer will take 259 days.'
    }
  }
]

为了将此结构适配到 MLflow 的 span 中，我们首先需要遍历原始响应跟踪，并按 traceId 对事件进行分组。按 traceId 分组后，结构如下所示：

展开查看按 traceId 分组的跟踪

{
  'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-0': [
    {
      'data': {
        'inferenceConfiguration': {
          'maximumLength': 2048,
          'stopSequences': [
            '</function_call>',
            '</answer>',
            '</error>'
          ],
          'temperature': 0.0,
          'topK': 250,
          'topP': 1.0
        },
        'text': '\n\nHuman:\nYou are a research assistant AI that has been equipped with one or more functions to help you answer a <question>...\n\nAssistant: <scratchpad> I understand I cannot use functions that have not been provided to me to answer this question.\n\n',
        'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-0',
        'type': 'ORCHESTRATION'
      },
      'event_order': 2,
      'type': 'modelInvocationInput'
    },
    {
      'data': {
        'metadata': {
          'usage': {
            'inputTokens': 5160,
            'outputTokens': 135
          }
        },
        'rawResponse': {
          'content': 'To answer this question about the next Mars launch window, I will:\n\n1. Call the GET::optimal_departure_window_mars::getNext...l>\nGET::optimal_departure_window_mars::getNextMarsLaunchWindow(specific_impulse="2500", dry_mass="10000", total_mass="50000")'
        },
        'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-0'
      },
      'event_order': 3,
      'type': 'modelInvocationOutput'
    },
    {
      'data': {
        'text': 'To answer this question about the next Mars launch window, I will:\n\n1. Call the GET::optimal_departure_window_mars::getNext...lse values.\n\nI have verified that I have access to the GET::optimal_departure_window_mars::getNextMarsLaunchWindow function.',
        'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-0'
      },
      'event_order': 4,
      'type': 'rationale'
    },
    {
      'data': {
        'actionGroupInvocationInput': {
          'actionGroupName': 'optimal_departure_window_mars',
          'apiPath': '/get-next-mars-launch-window',
          'executionType': 'LAMBDA',
          'parameters': [
            {
              ...
            },
            {
              ...
            },
            {
              ...
            }
          ],
          'verb': 'get'
        },
        'invocationType': 'ACTION_GROUP',
        'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-0'
      },
      'event_order': 5,
      'type': 'invocationInput'
    },
    {
      'data': {
        'actionGroupInvocationOutput': {
          'text': '{"next_launch_window": {"next_launch_date": "2026-11-26 00:00:00", "synodic_period_days": 779.9068939794238, "transfer_time_days": 259, "delta_v_available_m_s": 39457.985759929674, "delta_v_required_m_s": 5595.997417810693, "is_feasible": true}}'
        },
        'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-0',
        'type': 'ACTION_GROUP'
      },
      'event_order': 6,
      'type': 'observation'
    }
  ],
  'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-1': [
    {
      'data': {
        'inferenceConfiguration': {
          'maximumLength': 2048,
          'stopSequences': [
            '</function_call>',
            '</answer>',
            '</error>'
          ],
          'temperature': 0.0,
          'topK': 250,
          'topP': 1.0
        },
        'text': '\n\nHuman:\nYou are a research assistant AI that has been equipped with one or more functions to help you answer a <question>...lta_v_available_m_s": 39457.985759929674, "delta_v_required_m_s": 5595.997417810693, "is_feasible": true}}</function_result>\n',
        'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-1',
        'type': 'ORCHESTRATION'
      },
      'event_order': 7,
      'type': 'modelInvocationInput'
    },
    {
      'data': {
        'metadata': {
          'usage': {
            'inputTokens': 5405,
            'outputTokens': 64
          }
        },
        'rawResponse': {
          'content': '<answer>\nBased on the provided spacecraft dry mass of 10000 kg, total mass of 50000 kg, and specific impulse of 2500 s, the ... optimal launch window for a Hohmann transfer from Earth to Mars is on November 26, 2026 UTC. The transfer will take 259 days.'
        },
        'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-1'
      },
      'event_order': 8,
      'type': 'modelInvocationOutput'
    },
    {
      'data': {
        'finalResponse': {
          'text': 'Based on the provided spacecraft dry mass of 10000 kg, total mass of 50000 kg, and specific impulse of 2500 s, the next optimal launch window for a Hohmann transfer from Earth to Mars is on November 26, 2026 UTC. The transfer will take 259 days.'
        },
        'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-1',
        'type': 'FINISH'
      },
      'event_order': 9,
      'type': 'observation'
    }
  ],
  'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-pre-0': [
    {
      'data': {
        'inferenceConfiguration': {
          'maximumLength': 2048,
          'stopSequences': [
            '\n\nHuman:'
          ],
          'temperature': 0.0,
          'topK': 250,
          'topP': 1.0
        },
        'text': '\n\nHuman: You are a classifying agent that filters user inputs into categories. Your job is to sort these inputs before they...<thinking> XML tags before providing only the category letter to sort the input into within <category> XML tags.\n\nAssistant:',
        'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-pre-0',
        'type': 'PRE_PROCESSING'
      },
      'event_order': 0,
      'type': 'modelInvocationInput'
    },
    {
      'data': {
        'parsedResponse': {
          'isValid': True,
          'rationale': 'Based on the provided instructions, this input appears to be a question about orbital mechanics that can be answered using th...equired arguments for that function - specific impulse, dry mass, and total mass. Therefore, this input should be sorted into:'
        },
        'traceId': 'ca9880a2-dae7-46ac-a480-f38ca7e2d99f-pre-0'
      },
      'event_order': 1,
      'type': 'modelInvocationOutput'
    }
  ]
}

每个具有相同 traceId 的事件组将包含至少两个事件：一个类型为 modelInvocationInput，另一个类型为 modelInvocationOutput。涉及 Action Group 跟踪的组还将包括类型为 actionGroupInvocationInput 和 actionGroupInvocationOutput 的事件。同样，使用 Knowledge Bases 的组将有额外类型为 knowledgeBaseLookupInput 和 knowledgeBaseLookupOutput 的事件。在上面提到的 BedrockModel 中，它实现了一种方法来将这些事件组解析为跟踪节点。这种方法允许跟踪显示选择 Action Groups/Knowledge Bases 来回答查询以及调用相应的 Lambda 函数（如我们上面的示例 OpenAPI 规范所定义）背后的推理。这种结构有助于清晰地展示 Bedrock Agent 遵循的信息流和决策过程。

这是最终的 MLflow 跟踪

{
  "spans": [
    {
      "name": "Bedrock Agent Runtime",
      "context": {
        "span_id": "0xb802165d133a33aa",
        "trace_id": "0x9b8bd0b2e018d77f936e48a09e54fd44"
      },
      "parent_id": null,
      "start_time": 1731388531754725000,
      "end_time": 1731388550226771000,
      "status_code": "OK",
      "status_message": "",
      "attributes": {
        "mlflow.traceRequestId": "\"1e036cc3a7f946ec995f7763b8dde51c\"",
        "mlflow.spanType": "\"CHAT_MODEL\"",
        "mlflow.spanFunctionName": "\"predict\"",
        "mlflow.spanInputs": "{\"context\": \"<mlflow.pyfunc.model.PythonModelContext object at 0x13397c530>\", \"messages\": [{\"role\": \"user\", \"content\": \"When is the next launch window for Mars? My spacecraft's total mass is 50000, dry mass is 10000 and specific impulse is 2500. Mass in Kg.\", \"name\": null}], \"params\": {\"temperature\": 1.0, \"max_tokens\": null, \"stop\": null, \"n\": 1, \"stream\": false, \"top_p\": null, \"top_k\": null, \"frequency_penalty\": null, \"presence_penalty\": null}}",
        "mlflow.spanOutputs": "{\"choices\": [{\"index\": 0, \"message\": {\"role\": \"user\", \"content\": \"Based on the provided spacecraft dry mass of 10000 kg, total mass of 50000 kg, and specific impulse of 2500 s, the next optimal launch window for a Hohmann transfer from Earth to Mars is on November 26, 2026 UTC. The transfer will take 259 days.\", \"name\": null}, \"finish_reason\": \"stop\", \"logprobs\": null}], \"usage\": {\"prompt_tokens\": null, \"completion_tokens\": null, \"total_tokens\": null}, \"id\": null, \"model\": \"anthropic.claude-v2\", \"object\": \"chat.completion\", \"created\": 1731388550}"
      },
      "events": []
    },
    {
      "name": "Bedrock Input Prompt",
      "context": {
        "span_id": "0x2e7cd730be70865b",
        "trace_id": "0x9b8bd0b2e018d77f936e48a09e54fd44"
      },
      "parent_id": "0xb802165d133a33aa",
      "start_time": 1731388531755172000,
      "end_time": 1731388531755252000,
      "status_code": "OK",
      "status_message": "",
      "attributes": {
        "mlflow.traceRequestId": "\"1e036cc3a7f946ec995f7763b8dde51c\"",
        "mlflow.spanType": "\"UNKNOWN\"",
        "mlflow.spanFunctionName": "\"_get_agent_prompt\"",
        "mlflow.spanInputs": "{\"raw_input_question\": \"When is the next launch window for Mars? My spacecraft's total mass is 50000, dry mass is 10000 and specific impulse is 2500. Mass in Kg.\"}",
        "mlflow.spanOutputs": "\"\\n        Answer the following question and pay strong attention to the prompt:\\n        <question>\\n        When is the next launch window for Mars? My spacecraft's total mass is 50000, dry mass is 10000 and specific impulse is 2500. Mass in Kg.\\n        </question>\\n        <instruction>\\n        You have functions available at your disposal to use when anwering any questions about orbital mechanics.if you can't find a function to answer a question about orbital mechanics, simply reply 'I do not know'\\n        </instruction>\\n        \""
      },
      "events": []
    },
    {
      "name": "ACTION GROUP DECISION -optimal_departure_window_mars",
      "context": {
        "span_id": "0x131e4e08cd5e95d9",
        "trace_id": "0x9b8bd0b2e018d77f936e48a09e54fd44"
      },
      "parent_id": "0xb802165d133a33aa",
      "start_time": 1731388550223219000,
      "end_time": 1731388550224592000,
      "status_code": "OK",
      "status_message": "",
      "attributes": {
        "mlflow.traceRequestId": "\"1e036cc3a7f946ec995f7763b8dde51c\"",
        "mlflow.spanType": "\"UNKNOWN\"",
        "trace_attributes": "[{\"type\": \"modelInvocationInput\", \"data\": {\"inferenceConfiguration\": {\"maximumLength\": 2048, \"stopSequences\": [\"</function_call>\", \"</answer>\", \"</error>\"], \"temperature\": 0.0, \"topK\": 250, \"topP\": 1.0}, \"text\": \"\\n\\nHuman:\\nYou are a research assistant AI that has been equipped with one or more functions to help you answer a <question>. Your goal is to answer the user's question to the best of your ability, using the function(s) to gather more information if necessary to better answer the question. If you choose to call a function, the result of the function call will be added to the conversation history in <function_results> tags (if the call succeeded) or <error> tags (if the function failed). \\nYou were created with these instructions to consider as well:\\n<auxiliary_instructions>\\n            You are a friendly chat bot. You have access to a function called that returns\\n            information about the Mars launch window. When responding with Mars launch window,\\n            please make sure to add the timezone UTC.\\n            </auxiliary_instructions>\\n\\nHere are some examples of correct action by other, different agents with access to functions that may or may not be similar to ones you are provided.\\n\\n<examples>\\n    <example_docstring> Here is an example of how you would correctly answer a question using a <function_call> and the corresponding <function_result>. Notice that you are free to think before deciding to make a <function_call> in the <scratchpad>.</example_docstring>\\n    <example>\\n        <functions>\\n            <function>\\n                <function_name>get::policyengineactions::getpolicyviolations</function_name>\\n                <function_description>Returns a list of policy engine violations for the specified alias within the specified date range.</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <required_argument>startDate (string): The start date of the range to filter violations. The format for startDate is MM/DD/YYYY.</required_argument>\\n                <required_argument>endDate (string): The end date of the range to filter violations</required_argument>\\n                <returns>array: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>post::policyengineactions::acknowledgeviolations</function_name>\\n                <function_description>Acknowledge policy engine violation. Generally used to acknowledge violation, once user notices a violation under their alias or their managers alias.</function_description>\\n                <required_argument>policyId (string): The ID of the policy violation</required_argument>\\n                <required_argument>expectedDateOfResolution (string): The date by when the violation will be addressed/resolved</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>get::activedirectoryactions::getmanager</function_name>\\n                <function_description>This API is used to identify the manager hierarchy above a given person. Every person could have a manager and the manager could have another manager to which they report to</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            \\n        </functions>\\n\\n        <question>Can you show me my policy engine violation from 1st january 2023 to 1st february 2023? My alias is jsmith.</question>\\n        <scratchpad>\\n            To answer this question, I will need to:\\n            1. I do not have knowledge to policy engine violations, so I should see if I can use any of the available functions to help. I have been equipped with get::policyengineactions::getpolicyviolations that gets the policy engine violations for a given alias, start date and end date. I will use this function to gather more information.\\n        </scratchpad>\\n        <function_call>get::policyengineactions::getpolicyviolations(alias=\\\"jsmith\\\", startDate=\\\"1st January 2023\\\", endDate=\\\"1st February 2023\\\")</function_call>\\n        <function_result>{response: [{creationDate: \\\"2023-06-01T09:30:00Z\\\", riskLevel: \\\"High\\\", policyId: \\\"POL-001\\\", policyUrl: \\\"https://example.com/policies/POL-001\\\", referenceUrl: \\\"https://example.com/violations/POL-001\\\"}, {creationDate: \\\"2023-06-02T14:45:00Z\\\", riskLevel: \\\"Medium\\\", policyId: \\\"POL-002\\\", policyUrl: \\\"https://example.com/policies/POL-002\\\", referenceUrl: \\\"https://example.com/violations/POL-002\\\"}]}</function_result>\\n        <answer>The policy engine violations between 1st january 2023 to 1st february 2023 for alias jsmith are - Policy ID: POL-001, Policy ID: POL-002</answer>\\n    </example>\\n\\n    <example_docstring>Here is another example that utilizes multiple function calls.</example_docstring>\\n    <example>\\n        <functions>\\n            <function>\\n                <function_name>get::policyengineactions::getpolicyviolations</function_name>\\n                <function_description>Returns a list of policy engine violations for the specified alias within the specified date range.</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <required_argument>startDate (string): The start date of the range to filter violations. The format for startDate is MM/DD/YYYY.</required_argument>\\n                <required_argument>endDate (string): The end date of the range to filter violations</required_argument>\\n                <returns>array: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>post::policyengineactions::acknowledgeviolations</function_name>\\n                <function_description>Acknowledge policy engine violation. Generally used to acknowledge violation, once user notices a violation under their alias or their managers alias.</function_description>\\n                <required_argument>policyId (string): The ID of the policy violation</required_argument>\\n                <required_argument>expectedDateOfResolution (string): The date by when the violation will be addressed/resolved</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>get::activedirectoryactions::getmanager</function_name>\\n                <function_description>This API is used to identify the manager hierarchy above a given person. Every person could have a manager and the manager could have another manager to which they report to</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            \\n        </functions>\\n        <question>Can you check the policy engine violations under my manager between 2nd May to 5th May? My alias is john.</question>\\n        <scratchpad>\\n            To answer this question, I will need to:\\n            1. Get the manager alias of the user using get::activedirectoryactions::getmanager function.\\n            2. Use the returned manager alias to get the policy engine violations using the get::policyengineactions::getpolicyviolations function.\\n\\n            I have double checked and made sure that I have been provided the get::activedirectoryactions::getmanager and the get::policyengineactions::getpolicyviolations functions.\\n        </scratchpad>\\n        <function_call>get::activedirectoryactions::getmanager(alias=\\\"john\\\")</function_call>\\n        <function_result>{response: {managerAlias: \\\"mark\\\", managerLevel: \\\"6\\\", teamName: \\\"Builder\\\", managerName: \\\"Mark Hunter\\\"}}}}</function_result>\\n        <scratchpad>\\n            1. I have the managerAlias from the function results as mark and I have the start and end date from the user input. I can use the function result to call get::policyengineactions::getpolicyviolations function.\\n            2. I will then return the get::policyengineactions::getpolicyviolations function result to the user.\\n\\n            I have double checked and made sure that I have been provided the get::policyengineactions::getpolicyviolations functions.\\n        </scratchpad>\\n        <function_call>get::policyengineactions::getpolicyviolations(alias=\\\"mark\\\", startDate=\\\"2nd May 2023\\\", endDate=\\\"5th May 2023\\\")</function_call>\\n        <function_result>{response: [{creationDate: \\\"2023-05-02T09:30:00Z\\\", riskLevel: \\\"High\\\", policyId: \\\"POL-001\\\", policyUrl: \\\"https://example.com/policies/POL-001\\\", referenceUrl: \\\"https://example.com/violations/POL-001\\\"}, {creationDate: \\\"2023-05-04T14:45:00Z\\\", riskLevel: \\\"Low\\\", policyId: \\\"POL-002\\\", policyUrl: \\\"https://example.com/policies/POL-002\\\", referenceUrl: \\\"https://example.com/violations/POL-002\\\"}]}</function_result>\\n        <answer>\\n            The policy engine violations between 2nd May 2023 to 5th May 2023 for your manager's alias mark are - Policy ID: POL-001, Policy ID: POL-002\\n        </answer>\\n    </example>\\n\\n    <example_docstring>Functions can also be search engine API's that issue a query to a knowledge base. Here is an example that utilizes regular function calls in combination with function calls to a search engine API. Please make sure to extract the source for the information within the final answer when using information returned from the search engine.</example_docstring>\\n    <example>\\n        <functions>\\n            <function>\\n                <function_name>get::benefitsaction::getbenefitplanname</function_name>\\n                <function_description>Get's the benefit plan name for a user. The API takes in a userName and a benefit type and returns the benefit name to the user (i.e. Aetna, Premera, Fidelity, etc.).</function_description>\\n                <optional_argument>userName (string): None</optional_argument>\\n                <optional_argument>benefitType (string): None</optional_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>post::benefitsaction::increase401klimit</function_name>\\n                <function_description>Increases the 401k limit for a generic user. The API takes in only the current 401k limit and returns the new limit.</function_description>\\n                <optional_argument>currentLimit (string): None</optional_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>get::x_amz_knowledgebase_dentalinsurance::search</function_name>\\n                <function_description>This is a search tool that provides information about Delta Dental benefits. It has information about covered dental benefits and other relevant information</function_description>\\n                <required_argument>query(string): A full sentence query that is fed to the search tool</required_argument>\\n                <returns>Returns string  related to the user query asked.</returns>\\n            </function>\\n            <function>\\n                <function_name>get::x_amz_knowledgebase_401kplan::search</function_name>\\n                <function_description>This is a search tool that provides information about Amazon 401k plan benefits. It can determine what a person's yearly 401k contribution limit is, based on their age.</function_description>\\n                <required_argument>query(string): A full sentence query that is fed to the search tool</required_argument>\\n                <returns>Returns string  related to the user query asked.</returns>\\n            </function>\\n            <function>\\n                <function_name>get::x_amz_knowledgebase_healthinsurance::search</function_name>\\n                <function_description>This is a search tool that provides information about Aetna and Premera health benefits. It has information about the savings plan and shared deductible plan, as well as others.</function_description>\\n                <required_argument>query(string): A full sentence query that is fed to the search tool</required_argument>\\n                <returns>Returns string  related to the user query asked.</returns>\\n            </function>\\n            \\n        </functions>\\n\\n        <question>What is my deductible? My username is Bob and my benefitType is Dental. Also, what is the 401k yearly contribution limit?</question>\\n        <scratchpad> I understand I cannot use functions that have not been provided to me to answer this question.\\n            To answer this question, I will:\\n            1. Call the get::benefitsaction::getbenefitplanname function to get the benefit plan name for the user Bob with benefit type Dental.\\n            2. Call the get::x_amz_knowledgebase_dentalinsurance::search function to search for information about deductibles for the plan name returned from step 1.\\n            3. Call the get::x_amz_knowledgebase_401k::search function to search for information about 401k yearly contribution limits.\\n            4. Return the deductible information from the search results to the user.\\n            I have checked that I have access to the get::benefitsaction::getbenefitplanname, x_amz_knowledgebase_dentalinsurance::search, and x_amz_knowledgebase_401k::search functions.\\n        </scratchpad>\\n        <function_call>get::benefitsaction::getbenefitplanname(userName=\\\"Bob\\\", benefitType=\\\"Dental\\\")</function_call>\\n        <function_result>{{'response': {{'planName': 'Delta Dental'}}}}</function_result>\\n        <scratchpad>\\n            I have received the plan name Delta Dental for the user Bob with Dental benefits. I will now call the x_amz_knowledgebase_dentalinsurance::search function to find deductible information for Delta Dental.\\n        </scratchpad>\\n        <function_call>get::x_amz_knowledgebase_dentalinsurance::search(searchQuery=\\\"What is the deductible for Delta Dental?\\\")</function_call>\\n        <function_result>{{'response': {{'responseCode': '200', 'responseBody': \\\"\\\"<answer>\\\\n<answer_part>\\\\n<text>The user's individual deductible is $50 per benefit period</text>\\\\n<source>dfe040f8-46ed-4a65-b3ea-529fa55f6b9e</source>\\\\n</answer_part>\\\\n<answer_part>\\\\n<text>If they are enrolled with dependents, the maximum family deductible is $150 per benefit period.</text>\\\\n<source>0e666064-31d8-4223-b7ba-8eecf40b7b47</source>\\\\n</answer_part>\\\\n</answer>\\\"}}}}</function_result> <scratchpad>\\n            I have found the deductible information for Dental benefits. I will now call the x_amz_knowledgebase_401k::search function to find yearly 401k contribution limits.\\n        </scratchpad>\\n        <function_call>get::x_amz_knowledgebase_401k::search(searchQuery=\\\"What is the yearly 401k contribution limit?\\\")</function_call>\\n        <function_result>{{'response': {{'responseCode': '200', 'responseBody': \\\"<answer>\\\\n<answer_part>\\\\n<text>The yearly 401k contribution limit is $20,500.</text>\\\\n<source>c546cbe8-07f6-45d1-90ca-74d87ab2885a</source>\\\\n</answer_part>\\\\n</answer>\\\"}}}}</function_result>\\n        <answer>\\n            <answer_part>\\n                <text>The deductible for your Delta Dental plan is $50 per benefit period.</text>\\n                <source>dfe040f8-46ed-4a65-b3ea-529fa55f6b9e</source>\\n            </answer_part>\\n            <answer_part>\\n                <text>If you have dependents enrolled, the maximum family deductible is $150 per benefit period.</text>\\n                <source>0e666064-31d8-4223-b7ba-8eecf40b7b47</source>\\n            </answer_part>\\n            <answer_part>\\n                <text>The yearly 401k contribution limit is $20,500.</text>\\n                <source>c546cbe8-07f6-45d1-90ca-74d87ab2885a</source>\\n            </answer_part>\\n        </answer>\\n    </example>\\n\\n    \\n\\n    <example_docstring>Here's a final example where the question asked could not be answered with information gathered from calling the provided functions. In this example, notice how you respond by telling the user you cannot answer, without using a function that was not provided to you.</example_docstring>\\n    <example>\\n        <functions>\\n            <function>\\n                <function_name>get::policyengineactions::getpolicyviolations</function_name>\\n                <function_description>Returns a list of policy engine violations for the specified alias within the specified date range.</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <required_argument>startDate (string): The start date of the range to filter violations. The format for startDate is MM/DD/YYYY.</required_argument>\\n                <required_argument>endDate (string): The end date of the range to filter violations</required_argument>\\n                <returns>array: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>post::policyengineactions::acknowledgeviolations</function_name>\\n                <function_description>Acknowledge policy engine violation. Generally used to acknowledge violation, once user notices a violation under their alias or their managers alias.</function_description>\\n                <required_argument>policyId (string): The ID of the policy violation</required_argument>\\n                <required_argument>expectedDateOfResolution (string): The date by when the violation will be addressed/resolved</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>get::activedirectoryactions::getmanager</function_name>\\n                <function_description>This API is used to identify the manager hierarchy above a given person. Every person could have a manager and the manager could have another manager to which they report to</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            \\n        </functions>\\n        <question>Who are the reportees of David?</question>\\n        <scratchpad>\\n            After reviewing the functions I was equipped with, I realize I am not able to accurately answer this question since I can't access reportees of David. Therefore, I should explain to the user I cannot answer this question.\\n        </scratchpad>\\n        <answer>\\n            Sorry, I am unable to assist you with this request.\\n        </answer>\\n    </example>\\n</examples>\\n\\nThe above examples have been provided to you to illustrate general guidelines and format for use of function calling for information retrieval, and how to use your scratchpad to plan your approach. IMPORTANT: the functions provided within the examples should not be assumed to have been provided to you to use UNLESS they are also explicitly given to you within <functions></functions> tags below. All of the values and information within the examples (the questions, function results, and answers) are strictly part of the examples and have not been provided to you.\\n\\nNow that you have read and understood the examples, I will define the functions that you have available to you to use. Here is a comprehensive list.\\n\\n<functions>\\n<function>\\n<function_name>GET::optimal_departure_window_mars::getNextMarsLaunchWindow</function_name>\\n<function_description>Gets the next optimal launch window to Mars.</function_description>\\n<required_argument>specific_impulse (string): Specific impulse of the propulsion system (s).</required_argument>\\n<required_argument>dry_mass (string): Mass of the spacecraft without fuel (kg).</required_argument>\\n<required_argument>total_mass (string): Total mass of the spacecraft including fuel (kg)</required_argument>\\n<returns>object: The next optimal departure date for a Hohmann transfer from Earth to Mars, based on the spacecraft's mass and specific impulse.</returns>\\n</function>\\n\\n\\n</functions>\\n\\nNote that the function arguments have been listed in the order that they should be passed into the function.\\n\\n\\n\\nDo not modify or extend the provided functions under any circumstances. For example, GET::optimal_departure_window_mars::getNextMarsLaunchWindow with additional parameters would be considered modifying the function which is not allowed. Please use the functions only as defined.\\n\\nDO NOT use any functions that I have not equipped you with.\\n\\n Do not make assumptions about inputs; instead, make sure you know the exact function and input to use before you call a function.\\n\\nTo call a function, output the name of the function in between <function_call> and </function_call> tags. You will receive a <function_result> in response to your call that contains information that you can use to better answer the question. Or, if the function call produced an error, you will receive an <error> in response.\\n\\n\\n\\nThe format for all other <function_call> MUST be: <function_call>$FUNCTION_NAME($FUNCTION_PARAMETER_NAME=$FUNCTION_PARAMETER_VALUE)</function_call>\\n\\nRemember, your goal is to answer the user's question to the best of your ability, using only the function(s) provided within the <functions></functions> tags to gather more information if necessary to better answer the question.\\n\\nDo not modify or extend the provided functions under any circumstances. For example, calling GET::optimal_departure_window_mars::getNextMarsLaunchWindow with additional parameters would be modifying the function which is not allowed. Please use the functions only as defined.\\n\\nBefore calling any functions, create a plan for performing actions to answer this question within the <scratchpad>. Double check your plan to make sure you don't call any functions that you haven't been provided with. Always return your final answer within <answer></answer> tags.\\n\\n\\n\\nThe user input is <question>Answer the following question and pay strong attention to the prompt:\\n        <question>\\n        When is the next launch window for Mars? My spacecraft's total mass is 50000, dry mass is 10000 and specific impulse is 2500. Mass in Kg.\\n        </question>\\n        <instruction>\\n        You have functions available at your disposal to use when anwering any questions about orbital mechanics.if you can't find a function to answer a question about orbital mechanics, simply reply 'I do not know'\\n        </instruction></question>\\n\\n\\nAssistant: <scratchpad> I understand I cannot use functions that have not been provided to me to answer this question.\\n\\n\", \"traceId\": \"e0b2b2c2-fb7c-4e17-8a1f-a3781100face-0\", \"type\": \"ORCHESTRATION\"}, \"event_order\": 2}, {\"type\": \"modelInvocationOutput\", \"data\": {\"metadata\": {\"usage\": {\"inputTokens\": 5160, \"outputTokens\": 135}}, \"rawResponse\": {\"content\": \"To answer this question about the next Mars launch window, I will:\\n\\n1. Call the GET::optimal_departure_window_mars::getNextMarsLaunchWindow function to get the next optimal launch window, passing in the provided spacecraft mass and specific impulse values.\\n\\nI have verified that I have access to the GET::optimal_departure_window_mars::getNextMarsLaunchWindow function.\\n\\n</scratchpad>\\n\\n<function_call>\\nGET::optimal_departure_window_mars::getNextMarsLaunchWindow(specific_impulse=\\\"2500\\\", dry_mass=\\\"10000\\\", total_mass=\\\"50000\\\")\"}, \"traceId\": \"e0b2b2c2-fb7c-4e17-8a1f-a3781100face-0\"}, \"event_order\": 3}, {\"type\": \"rationale\", \"data\": {\"text\": \"To answer this question about the next Mars launch window, I will:\\n\\n1. Call the GET::optimal_departure_window_mars::getNextMarsLaunchWindow function to get the next optimal launch window, passing in the provided spacecraft mass and specific impulse values.\\n\\nI have verified that I have access to the GET::optimal_departure_window_mars::getNextMarsLaunchWindow function.\", \"traceId\": \"e0b2b2c2-fb7c-4e17-8a1f-a3781100face-0\"}, \"event_order\": 4}, {\"type\": \"invocationInput\", \"data\": {\"actionGroupInvocationInput\": {\"actionGroupName\": \"optimal_departure_window_mars\", \"apiPath\": \"/get-next-mars-launch-window\", \"executionType\": \"LAMBDA\", \"parameters\": [{\"name\": \"total_mass\", \"type\": \"string\", \"value\": \"50000\"}, {\"name\": \"dry_mass\", \"type\": \"string\", \"value\": \"10000\"}, {\"name\": \"specific_impulse\", \"type\": \"string\", \"value\": \"2500\"}], \"verb\": \"get\"}, \"invocationType\": \"ACTION_GROUP\", \"traceId\": \"e0b2b2c2-fb7c-4e17-8a1f-a3781100face-0\"}, \"event_order\": 5}, {\"type\": \"observation\", \"data\": {\"actionGroupInvocationOutput\": {\"text\": \"{\\\"next_launch_window\\\": {\\\"next_launch_date\\\": \\\"2026-11-26 00:00:00\\\", \\\"synodic_period_days\\\": 779.9068939794238, \\\"transfer_time_days\\\": 259, \\\"delta_v_available_m_s\\\": 39457.985759929674, \\\"delta_v_required_m_s\\\": 5595.997417810693, \\\"is_feasible\\\": true}}\"}, \"traceId\": \"e0b2b2c2-fb7c-4e17-8a1f-a3781100face-0\", \"type\": \"ACTION_GROUP\"}, \"event_order\": 6}]",
        "mlflow.spanFunctionName": "\"_trace_agent_pre_context\"",
        "mlflow.spanInputs": "{\"inner_input_trace\": \"\\n\\nHuman:\\nYou are a research assistant AI that has been equipped with one or more functions to help you answer a <question>. Your goal is to answer the user's question to the best of your ability, using the function(s) to gather more information if necessary to better answer the question. If you choose to call a function, the result of the function call will be added to the conversation history in <function_results> tags (if the call succeeded) or <error> tags (if the function failed). \\nYou were created with these instructions to consider as well:\\n<auxiliary_instructions>\\n            You are a friendly chat bot. You have access to a function called that returns\\n            information about the Mars launch window. When responding with Mars launch window,\\n            please make sure to add the timezone UTC.\\n            </auxiliary_instructions>\\n\\nHere are some examples of correct action by other, different agents with access to functions that may or may not be similar to ones you are provided.\\n\\n<examples>\\n    <example_docstring> Here is an example of how you would correctly answer a question using a <function_call> and the corresponding <function_result>. Notice that you are free to think before deciding to make a <function_call> in the <scratchpad>.</example_docstring>\\n    <example>\\n        <functions>\\n            <function>\\n                <function_name>get::policyengineactions::getpolicyviolations</function_name>\\n                <function_description>Returns a list of policy engine violations for the specified alias within the specified date range.</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <required_argument>startDate (string): The start date of the range to filter violations. The format for startDate is MM/DD/YYYY.</required_argument>\\n                <required_argument>endDate (string): The end date of the range to filter violations</required_argument>\\n                <returns>array: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>post::policyengineactions::acknowledgeviolations</function_name>\\n                <function_description>Acknowledge policy engine violation. Generally used to acknowledge violation, once user notices a violation under their alias or their managers alias.</function_description>\\n                <required_argument>policyId (string): The ID of the policy violation</required_argument>\\n                <required_argument>expectedDateOfResolution (string): The date by when the violation will be addressed/resolved</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>get::activedirectoryactions::getmanager</function_name>\\n                <function_description>This API is used to identify the manager hierarchy above a given person. Every person could have a manager and the manager could have another manager to which they report to</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            \\n        </functions>\\n\\n        <question>Can you show me my policy engine violation from 1st january 2023 to 1st february 2023? My alias is jsmith.</question>\\n        <scratchpad>\\n            To answer this question, I will need to:\\n            1. I do not have knowledge to policy engine violations, so I should see if I can use any of the available functions to help. I have been equipped with get::policyengineactions::getpolicyviolations that gets the policy engine violations for a given alias, start date and end date. I will use this function to gather more information.\\n        </scratchpad>\\n        <function_call>get::policyengineactions::getpolicyviolations(alias=\\\"jsmith\\\", startDate=\\\"1st January 2023\\\", endDate=\\\"1st February 2023\\\")</function_call>\\n        <function_result>{response: [{creationDate: \\\"2023-06-01T09:30:00Z\\\", riskLevel: \\\"High\\\", policyId: \\\"POL-001\\\", policyUrl: \\\"https://example.com/policies/POL-001\\\", referenceUrl: \\\"https://example.com/violations/POL-001\\\"}, {creationDate: \\\"2023-06-02T14:45:00Z\\\", riskLevel: \\\"Medium\\\", policyId: \\\"POL-002\\\", policyUrl: \\\"https://example.com/policies/POL-002\\\", referenceUrl: \\\"https://example.com/violations/POL-002\\\"}]}</function_result>\\n        <answer>The policy engine violations between 1st january 2023 to 1st february 2023 for alias jsmith are - Policy ID: POL-001, Policy ID: POL-002</answer>\\n    </example>\\n\\n    <example_docstring>Here is another example that utilizes multiple function calls.</example_docstring>\\n    <example>\\n        <functions>\\n            <function>\\n                <function_name>get::policyengineactions::getpolicyviolations</function_name>\\n                <function_description>Returns a list of policy engine violations for the specified alias within the specified date range.</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <required_argument>startDate (string): The start date of the range to filter violations. The format for startDate is MM/DD/YYYY.</required_argument>\\n                <required_argument>endDate (string): The end date of the range to filter violations</required_argument>\\n                <returns>array: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>post::policyengineactions::acknowledgeviolations</function_name>\\n                <function_description>Acknowledge policy engine violation. Generally used to acknowledge violation, once user notices a violation under their alias or their managers alias.</function_description>\\n                <required_argument>policyId (string): The ID of the policy violation</required_argument>\\n                <required_argument>expectedDateOfResolution (string): The date by when the violation will be addressed/resolved</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>get::activedirectoryactions::getmanager</function_name>\\n                <function_description>This API is used to identify the manager hierarchy above a given person. Every person could have a manager and the manager could have another manager to which they report to</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            \\n        </functions>\\n        <question>Can you check the policy engine violations under my manager between 2nd May to 5th May? My alias is john.</question>\\n        <scratchpad>\\n            To answer this question, I will need to:\\n            1. Get the manager alias of the user using get::activedirectoryactions::getmanager function.\\n            2. Use the returned manager alias to get the policy engine violations using the get::policyengineactions::getpolicyviolations function.\\n\\n            I have double checked and made sure that I have been provided the get::activedirectoryactions::getmanager and the get::policyengineactions::getpolicyviolations functions.\\n        </scratchpad>\\n        <function_call>get::activedirectoryactions::getmanager(alias=\\\"john\\\")</function_call>\\n        <function_result>{response: {managerAlias: \\\"mark\\\", managerLevel: \\\"6\\\", teamName: \\\"Builder\\\", managerName: \\\"Mark Hunter\\\"}}}}</function_result>\\n        <scratchpad>\\n            1. I have the managerAlias from the function results as mark and I have the start and end date from the user input. I can use the function result to call get::policyengineactions::getpolicyviolations function.\\n            2. I will then return the get::policyengineactions::getpolicyviolations function result to the user.\\n\\n            I have double checked and made sure that I have been provided the get::policyengineactions::getpolicyviolations functions.\\n        </scratchpad>\\n        <function_call>get::policyengineactions::getpolicyviolations(alias=\\\"mark\\\", startDate=\\\"2nd May 2023\\\", endDate=\\\"5th May 2023\\\")</function_call>\\n        <function_result>{response: [{creationDate: \\\"2023-05-02T09:30:00Z\\\", riskLevel: \\\"High\\\", policyId: \\\"POL-001\\\", policyUrl: \\\"https://example.com/policies/POL-001\\\", referenceUrl: \\\"https://example.com/violations/POL-001\\\"}, {creationDate: \\\"2023-05-04T14:45:00Z\\\", riskLevel: \\\"Low\\\", policyId: \\\"POL-002\\\", policyUrl: \\\"https://example.com/policies/POL-002\\\", referenceUrl: \\\"https://example.com/violations/POL-002\\\"}]}</function_result>\\n        <answer>\\n            The policy engine violations between 2nd May 2023 to 5th May 2023 for your manager's alias mark are - Policy ID: POL-001, Policy ID: POL-002\\n        </answer>\\n    </example>\\n\\n    <example_docstring>Functions can also be search engine API's that issue a query to a knowledge base. Here is an example that utilizes regular function calls in combination with function calls to a search engine API. Please make sure to extract the source for the information within the final answer when using information returned from the search engine.</example_docstring>\\n    <example>\\n        <functions>\\n            <function>\\n                <function_name>get::benefitsaction::getbenefitplanname</function_name>\\n                <function_description>Get's the benefit plan name for a user. The API takes in a userName and a benefit type and returns the benefit name to the user (i.e. Aetna, Premera, Fidelity, etc.).</function_description>\\n                <optional_argument>userName (string): None</optional_argument>\\n                <optional_argument>benefitType (string): None</optional_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>post::benefitsaction::increase401klimit</function_name>\\n                <function_description>Increases the 401k limit for a generic user. The API takes in only the current 401k limit and returns the new limit.</function_description>\\n                <optional_argument>currentLimit (string): None</optional_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>get::x_amz_knowledgebase_dentalinsurance::search</function_name>\\n                <function_description>This is a search tool that provides information about Delta Dental benefits. It has information about covered dental benefits and other relevant information</function_description>\\n                <required_argument>query(string): A full sentence query that is fed to the search tool</required_argument>\\n                <returns>Returns string  related to the user query asked.</returns>\\n            </function>\\n            <function>\\n                <function_name>get::x_amz_knowledgebase_401kplan::search</function_name>\\n                <function_description>This is a search tool that provides information about Amazon 401k plan benefits. It can determine what a person's yearly 401k contribution limit is, based on their age.</function_description>\\n                <required_argument>query(string): A full sentence query that is fed to the search tool</required_argument>\\n                <returns>Returns string  related to the user query asked.</returns>\\n            </function>\\n            <function>\\n                <function_name>get::x_amz_knowledgebase_healthinsurance::search</function_name>\\n                <function_description>This is a search tool that provides information about Aetna and Premera health benefits. It has information about the savings plan and shared deductible plan, as well as others.</function_description>\\n                <required_argument>query(string): A full sentence query that is fed to the search tool</required_argument>\\n                <returns>Returns string  related to the user query asked.</returns>\\n            </function>\\n            \\n        </functions>\\n\\n        <question>What is my deductible? My username is Bob and my benefitType is Dental. Also, what is the 401k yearly contribution limit?</question>\\n        <scratchpad> I understand I cannot use functions that have not been provided to me to answer this question.\\n            To answer this question, I will:\\n            1. Call the get::benefitsaction::getbenefitplanname function to get the benefit plan name for the user Bob with benefit type Dental.\\n            2. Call the get::x_amz_knowledgebase_dentalinsurance::search function to search for information about deductibles for the plan name returned from step 1.\\n            3. Call the get::x_amz_knowledgebase_401k::search function to search for information about 401k yearly contribution limits.\\n            4. Return the deductible information from the search results to the user.\\n            I have checked that I have access to the get::benefitsaction::getbenefitplanname, x_amz_knowledgebase_dentalinsurance::search, and x_amz_knowledgebase_401k::search functions.\\n        </scratchpad>\\n        <function_call>get::benefitsaction::getbenefitplanname(userName=\\\"Bob\\\", benefitType=\\\"Dental\\\")</function_call>\\n        <function_result>{{'response': {{'planName': 'Delta Dental'}}}}</function_result>\\n        <scratchpad>\\n            I have received the plan name Delta Dental for the user Bob with Dental benefits. I will now call the x_amz_knowledgebase_dentalinsurance::search function to find deductible information for Delta Dental.\\n        </scratchpad>\\n        <function_call>get::x_amz_knowledgebase_dentalinsurance::search(searchQuery=\\\"What is the deductible for Delta Dental?\\\")</function_call>\\n        <function_result>{{'response': {{'responseCode': '200', 'responseBody': \\\"\\\"<answer>\\\\n<answer_part>\\\\n<text>The user's individual deductible is $50 per benefit period</text>\\\\n<source>dfe040f8-46ed-4a65-b3ea-529fa55f6b9e</source>\\\\n</answer_part>\\\\n<answer_part>\\\\n<text>If they are enrolled with dependents, the maximum family deductible is $150 per benefit period.</text>\\\\n<source>0e666064-31d8-4223-b7ba-8eecf40b7b47</source>\\\\n</answer_part>\\\\n</answer>\\\"}}}}</function_result> <scratchpad>\\n            I have found the deductible information for Dental benefits. I will now call the x_amz_knowledgebase_401k::search function to find yearly 401k contribution limits.\\n        </scratchpad>\\n        <function_call>get::x_amz_knowledgebase_401k::search(searchQuery=\\\"What is the yearly 401k contribution limit?\\\")</function_call>\\n        <function_result>{{'response': {{'responseCode': '200', 'responseBody': \\\"<answer>\\\\n<answer_part>\\\\n<text>The yearly 401k contribution limit is $20,500.</text>\\\\n<source>c546cbe8-07f6-45d1-90ca-74d87ab2885a</source>\\\\n</answer_part>\\\\n</answer>\\\"}}}}</function_result>\\n        <answer>\\n            <answer_part>\\n                <text>The deductible for your Delta Dental plan is $50 per benefit period.</text>\\n                <source>dfe040f8-46ed-4a65-b3ea-529fa55f6b9e</source>\\n            </answer_part>\\n            <answer_part>\\n                <text>If you have dependents enrolled, the maximum family deductible is $150 per benefit period.</text>\\n                <source>0e666064-31d8-4223-b7ba-8eecf40b7b47</source>\\n            </answer_part>\\n            <answer_part>\\n                <text>The yearly 401k contribution limit is $20,500.</text>\\n                <source>c546cbe8-07f6-45d1-90ca-74d87ab2885a</source>\\n            </answer_part>\\n        </answer>\\n    </example>\\n\\n    \\n\\n    <example_docstring>Here's a final example where the question asked could not be answered with information gathered from calling the provided functions. In this example, notice how you respond by telling the user you cannot answer, without using a function that was not provided to you.</example_docstring>\\n    <example>\\n        <functions>\\n            <function>\\n                <function_name>get::policyengineactions::getpolicyviolations</function_name>\\n                <function_description>Returns a list of policy engine violations for the specified alias within the specified date range.</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <required_argument>startDate (string): The start date of the range to filter violations. The format for startDate is MM/DD/YYYY.</required_argument>\\n                <required_argument>endDate (string): The end date of the range to filter violations</required_argument>\\n                <returns>array: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>post::policyengineactions::acknowledgeviolations</function_name>\\n                <function_description>Acknowledge policy engine violation. Generally used to acknowledge violation, once user notices a violation under their alias or their managers alias.</function_description>\\n                <required_argument>policyId (string): The ID of the policy violation</required_argument>\\n                <required_argument>expectedDateOfResolution (string): The date by when the violation will be addressed/resolved</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>get::activedirectoryactions::getmanager</function_name>\\n                <function_description>This API is used to identify the manager hierarchy above a given person. Every person could have a manager and the manager could have another manager to which they report to</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            \\n        </functions>\\n        <question>Who are the reportees of David?</question>\\n        <scratchpad>\\n            After reviewing the functions I was equipped with, I realize I am not able to accurately answer this question since I can't access reportees of David. Therefore, I should explain to the user I cannot answer this question.\\n        </scratchpad>\\n        <answer>\\n            Sorry, I am unable to assist you with this request.\\n        </answer>\\n    </example>\\n</examples>\\n\\nThe above examples have been provided to you to illustrate general guidelines and format for use of function calling for information retrieval, and how to use your scratchpad to plan your approach. IMPORTANT: the functions provided within the examples should not be assumed to have been provided to you to use UNLESS they are also explicitly given to you within <functions></functions> tags below. All of the values and information within the examples (the questions, function results, and answers) are strictly part of the examples and have not been provided to you.\\n\\nNow that you have read and understood the examples, I will define the functions that you have available to you to use. Here is a comprehensive list.\\n\\n<functions>\\n<function>\\n<function_name>GET::optimal_departure_window_mars::getNextMarsLaunchWindow</function_name>\\n<function_description>Gets the next optimal launch window to Mars.</function_description>\\n<required_argument>specific_impulse (string): Specific impulse of the propulsion system (s).</required_argument>\\n<required_argument>dry_mass (string): Mass of the spacecraft without fuel (kg).</required_argument>\\n<required_argument>total_mass (string): Total mass of the spacecraft including fuel (kg)</required_argument>\\n<returns>object: The next optimal departure date for a Hohmann transfer from Earth to Mars, based on the spacecraft's mass and specific impulse.</returns>\\n</function>\\n\\n\\n</functions>\\n\\nNote that the function arguments have been listed in the order that they should be passed into the function.\\n\\n\\n\\nDo not modify or extend the provided functions under any circumstances. For example, GET::optimal_departure_window_mars::getNextMarsLaunchWindow with additional parameters would be considered modifying the function which is not allowed. Please use the functions only as defined.\\n\\nDO NOT use any functions that I have not equipped you with.\\n\\n Do not make assumptions about inputs; instead, make sure you know the exact function and input to use before you call a function.\\n\\nTo call a function, output the name of the function in between <function_call> and </function_call> tags. You will receive a <function_result> in response to your call that contains information that you can use to better answer the question. Or, if the function call produced an error, you will receive an <error> in response.\\n\\n\\n\\nThe format for all other <function_call> MUST be: <function_call>$FUNCTION_NAME($FUNCTION_PARAMETER_NAME=$FUNCTION_PARAMETER_VALUE)</function_call>\\n\\nRemember, your goal is to answer the user's question to the best of your ability, using only the function(s) provided within the <functions></functions> tags to gather more information if necessary to better answer the question.\\n\\nDo not modify or extend the provided functions under any circumstances. For example, calling GET::optimal_departure_window_mars::getNextMarsLaunchWindow with additional parameters would be modifying the function which is not allowed. Please use the functions only as defined.\\n\\nBefore calling any functions, create a plan for performing actions to answer this question within the <scratchpad>. Double check your plan to make sure you don't call any functions that you haven't been provided with. Always return your final answer within <answer></answer> tags.\\n\\n\\n\\nThe user input is <question>Answer the following question and pay strong attention to the prompt:\\n        <question>\\n        When is the next launch window for Mars? My spacecraft's total mass is 50000, dry mass is 10000 and specific impulse is 2500. Mass in Kg.\\n        </question>\\n        <instruction>\\n        You have functions available at your disposal to use when anwering any questions about orbital mechanics.if you can't find a function to answer a question about orbital mechanics, simply reply 'I do not know'\\n        </instruction></question>\\n\\n\\nAssistant: <scratchpad> I understand I cannot use functions that have not been provided to me to answer this question.\\n\\n\"}",
        "mlflow.spanOutputs": "\"To answer this question about the next Mars launch window, I will:\\n\\n1. Call the GET::optimal_departure_window_mars::getNextMarsLaunchWindow function to get the next optimal launch window, passing in the provided spacecraft mass and specific impulse values.\\n\\nI have verified that I have access to the GET::optimal_departure_window_mars::getNextMarsLaunchWindow function.\""
      },
      "events": []
    },
    {
      "name": "Invoking Action Group",
      "context": {
        "span_id": "0x692bd6457647dc76",
        "trace_id": "0x9b8bd0b2e018d77f936e48a09e54fd44"
      },
      "parent_id": "0xb802165d133a33aa",
      "start_time": 1731388550224851000,
      "end_time": 1731388550225218000,
      "status_code": "OK",
      "status_message": "",
      "attributes": {
        "mlflow.traceRequestId": "\"1e036cc3a7f946ec995f7763b8dde51c\"",
        "mlflow.spanType": "\"UNKNOWN\"",
        "trace_attributes": "[{\"type\": \"modelInvocationInput\", \"data\": {\"inferenceConfiguration\": {\"maximumLength\": 2048, \"stopSequences\": [\"</function_call>\", \"</answer>\", \"</error>\"], \"temperature\": 0.0, \"topK\": 250, \"topP\": 1.0}, \"text\": \"\\n\\nHuman:\\nYou are a research assistant AI that has been equipped with one or more functions to help you answer a <question>. Your goal is to answer the user's question to the best of your ability, using the function(s) to gather more information if necessary to better answer the question. If you choose to call a function, the result of the function call will be added to the conversation history in <function_results> tags (if the call succeeded) or <error> tags (if the function failed). \\nYou were created with these instructions to consider as well:\\n<auxiliary_instructions>\\n            You are a friendly chat bot. You have access to a function called that returns\\n            information about the Mars launch window. When responding with Mars launch window,\\n            please make sure to add the timezone UTC.\\n            </auxiliary_instructions>\\n\\nHere are some examples of correct action by other, different agents with access to functions that may or may not be similar to ones you are provided.\\n\\n<examples>\\n    <example_docstring> Here is an example of how you would correctly answer a question using a <function_call> and the corresponding <function_result>. Notice that you are free to think before deciding to make a <function_call> in the <scratchpad>.</example_docstring>\\n    <example>\\n        <functions>\\n            <function>\\n                <function_name>get::policyengineactions::getpolicyviolations</function_name>\\n                <function_description>Returns a list of policy engine violations for the specified alias within the specified date range.</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <required_argument>startDate (string): The start date of the range to filter violations. The format for startDate is MM/DD/YYYY.</required_argument>\\n                <required_argument>endDate (string): The end date of the range to filter violations</required_argument>\\n                <returns>array: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>post::policyengineactions::acknowledgeviolations</function_name>\\n                <function_description>Acknowledge policy engine violation. Generally used to acknowledge violation, once user notices a violation under their alias or their managers alias.</function_description>\\n                <required_argument>policyId (string): The ID of the policy violation</required_argument>\\n                <required_argument>expectedDateOfResolution (string): The date by when the violation will be addressed/resolved</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>get::activedirectoryactions::getmanager</function_name>\\n                <function_description>This API is used to identify the manager hierarchy above a given person. Every person could have a manager and the manager could have another manager to which they report to</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            \\n        </functions>\\n\\n        <question>Can you show me my policy engine violation from 1st january 2023 to 1st february 2023? My alias is jsmith.</question>\\n        <scratchpad>\\n            To answer this question, I will need to:\\n            1. I do not have knowledge to policy engine violations, so I should see if I can use any of the available functions to help. I have been equipped with get::policyengineactions::getpolicyviolations that gets the policy engine violations for a given alias, start date and end date. I will use this function to gather more information.\\n        </scratchpad>\\n        <function_call>get::policyengineactions::getpolicyviolations(alias=\\\"jsmith\\\", startDate=\\\"1st January 2023\\\", endDate=\\\"1st February 2023\\\")</function_call>\\n        <function_result>{response: [{creationDate: \\\"2023-06-01T09:30:00Z\\\", riskLevel: \\\"High\\\", policyId: \\\"POL-001\\\", policyUrl: \\\"https://example.com/policies/POL-001\\\", referenceUrl: \\\"https://example.com/violations/POL-001\\\"}, {creationDate: \\\"2023-06-02T14:45:00Z\\\", riskLevel: \\\"Medium\\\", policyId: \\\"POL-002\\\", policyUrl: \\\"https://example.com/policies/POL-002\\\", referenceUrl: \\\"https://example.com/violations/POL-002\\\"}]}</function_result>\\n        <answer>The policy engine violations between 1st january 2023 to 1st february 2023 for alias jsmith are - Policy ID: POL-001, Policy ID: POL-002</answer>\\n    </example>\\n\\n    <example_docstring>Here is another example that utilizes multiple function calls.</example_docstring>\\n    <example>\\n        <functions>\\n            <function>\\n                <function_name>get::policyengineactions::getpolicyviolations</function_name>\\n                <function_description>Returns a list of policy engine violations for the specified alias within the specified date range.</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <required_argument>startDate (string): The start date of the range to filter violations. The format for startDate is MM/DD/YYYY.</required_argument>\\n                <required_argument>endDate (string): The end date of the range to filter violations</required_argument>\\n                <returns>array: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>post::policyengineactions::acknowledgeviolations</function_name>\\n                <function_description>Acknowledge policy engine violation. Generally used to acknowledge violation, once user notices a violation under their alias or their managers alias.</function_description>\\n                <required_argument>policyId (string): The ID of the policy violation</required_argument>\\n                <required_argument>expectedDateOfResolution (string): The date by when the violation will be addressed/resolved</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>get::activedirectoryactions::getmanager</function_name>\\n                <function_description>This API is used to identify the manager hierarchy above a given person. Every person could have a manager and the manager could have another manager to which they report to</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            \\n        </functions>\\n        <question>Can you check the policy engine violations under my manager between 2nd May to 5th May? My alias is john.</question>\\n        <scratchpad>\\n            To answer this question, I will need to:\\n            1. Get the manager alias of the user using get::activedirectoryactions::getmanager function.\\n            2. Use the returned manager alias to get the policy engine violations using the get::policyengineactions::getpolicyviolations function.\\n\\n            I have double checked and made sure that I have been provided the get::activedirectoryactions::getmanager and the get::policyengineactions::getpolicyviolations functions.\\n        </scratchpad>\\n        <function_call>get::activedirectoryactions::getmanager(alias=\\\"john\\\")</function_call>\\n        <function_result>{response: {managerAlias: \\\"mark\\\", managerLevel: \\\"6\\\", teamName: \\\"Builder\\\", managerName: \\\"Mark Hunter\\\"}}}}</function_result>\\n        <scratchpad>\\n            1. I have the managerAlias from the function results as mark and I have the start and end date from the user input. I can use the function result to call get::policyengineactions::getpolicyviolations function.\\n            2. I will then return the get::policyengineactions::getpolicyviolations function result to the user.\\n\\n            I have double checked and made sure that I have been provided the get::policyengineactions::getpolicyviolations functions.\\n        </scratchpad>\\n        <function_call>get::policyengineactions::getpolicyviolations(alias=\\\"mark\\\", startDate=\\\"2nd May 2023\\\", endDate=\\\"5th May 2023\\\")</function_call>\\n        <function_result>{response: [{creationDate: \\\"2023-05-02T09:30:00Z\\\", riskLevel: \\\"High\\\", policyId: \\\"POL-001\\\", policyUrl: \\\"https://example.com/policies/POL-001\\\", referenceUrl: \\\"https://example.com/violations/POL-001\\\"}, {creationDate: \\\"2023-05-04T14:45:00Z\\\", riskLevel: \\\"Low\\\", policyId: \\\"POL-002\\\", policyUrl: \\\"https://example.com/policies/POL-002\\\", referenceUrl: \\\"https://example.com/violations/POL-002\\\"}]}</function_result>\\n        <answer>\\n            The policy engine violations between 2nd May 2023 to 5th May 2023 for your manager's alias mark are - Policy ID: POL-001, Policy ID: POL-002\\n        </answer>\\n    </example>\\n\\n    <example_docstring>Functions can also be search engine API's that issue a query to a knowledge base. Here is an example that utilizes regular function calls in combination with function calls to a search engine API. Please make sure to extract the source for the information within the final answer when using information returned from the search engine.</example_docstring>\\n    <example>\\n        <functions>\\n            <function>\\n                <function_name>get::benefitsaction::getbenefitplanname</function_name>\\n                <function_description>Get's the benefit plan name for a user. The API takes in a userName and a benefit type and returns the benefit name to the user (i.e. Aetna, Premera, Fidelity, etc.).</function_description>\\n                <optional_argument>userName (string): None</optional_argument>\\n                <optional_argument>benefitType (string): None</optional_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>post::benefitsaction::increase401klimit</function_name>\\n                <function_description>Increases the 401k limit for a generic user. The API takes in only the current 401k limit and returns the new limit.</function_description>\\n                <optional_argument>currentLimit (string): None</optional_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>get::x_amz_knowledgebase_dentalinsurance::search</function_name>\\n                <function_description>This is a search tool that provides information about Delta Dental benefits. It has information about covered dental benefits and other relevant information</function_description>\\n                <required_argument>query(string): A full sentence query that is fed to the search tool</required_argument>\\n                <returns>Returns string  related to the user query asked.</returns>\\n            </function>\\n            <function>\\n                <function_name>get::x_amz_knowledgebase_401kplan::search</function_name>\\n                <function_description>This is a search tool that provides information about Amazon 401k plan benefits. It can determine what a person's yearly 401k contribution limit is, based on their age.</function_description>\\n                <required_argument>query(string): A full sentence query that is fed to the search tool</required_argument>\\n                <returns>Returns string  related to the user query asked.</returns>\\n            </function>\\n            <function>\\n                <function_name>get::x_amz_knowledgebase_healthinsurance::search</function_name>\\n                <function_description>This is a search tool that provides information about Aetna and Premera health benefits. It has information about the savings plan and shared deductible plan, as well as others.</function_description>\\n                <required_argument>query(string): A full sentence query that is fed to the search tool</required_argument>\\n                <returns>Returns string  related to the user query asked.</returns>\\n            </function>\\n            \\n        </functions>\\n\\n        <question>What is my deductible? My username is Bob and my benefitType is Dental. Also, what is the 401k yearly contribution limit?</question>\\n        <scratchpad> I understand I cannot use functions that have not been provided to me to answer this question.\\n            To answer this question, I will:\\n            1. Call the get::benefitsaction::getbenefitplanname function to get the benefit plan name for the user Bob with benefit type Dental.\\n            2. Call the get::x_amz_knowledgebase_dentalinsurance::search function to search for information about deductibles for the plan name returned from step 1.\\n            3. Call the get::x_amz_knowledgebase_401k::search function to search for information about 401k yearly contribution limits.\\n            4. Return the deductible information from the search results to the user.\\n            I have checked that I have access to the get::benefitsaction::getbenefitplanname, x_amz_knowledgebase_dentalinsurance::search, and x_amz_knowledgebase_401k::search functions.\\n        </scratchpad>\\n        <function_call>get::benefitsaction::getbenefitplanname(userName=\\\"Bob\\\", benefitType=\\\"Dental\\\")</function_call>\\n        <function_result>{{'response': {{'planName': 'Delta Dental'}}}}</function_result>\\n        <scratchpad>\\n            I have received the plan name Delta Dental for the user Bob with Dental benefits. I will now call the x_amz_knowledgebase_dentalinsurance::search function to find deductible information for Delta Dental.\\n        </scratchpad>\\n        <function_call>get::x_amz_knowledgebase_dentalinsurance::search(searchQuery=\\\"What is the deductible for Delta Dental?\\\")</function_call>\\n        <function_result>{{'response': {{'responseCode': '200', 'responseBody': \\\"\\\"<answer>\\\\n<answer_part>\\\\n<text>The user's individual deductible is $50 per benefit period</text>\\\\n<source>dfe040f8-46ed-4a65-b3ea-529fa55f6b9e</source>\\\\n</answer_part>\\\\n<answer_part>\\\\n<text>If they are enrolled with dependents, the maximum family deductible is $150 per benefit period.</text>\\\\n<source>0e666064-31d8-4223-b7ba-8eecf40b7b47</source>\\\\n</answer_part>\\\\n</answer>\\\"}}}}</function_result> <scratchpad>\\n            I have found the deductible information for Dental benefits. I will now call the x_amz_knowledgebase_401k::search function to find yearly 401k contribution limits.\\n        </scratchpad>\\n        <function_call>get::x_amz_knowledgebase_401k::search(searchQuery=\\\"What is the yearly 401k contribution limit?\\\")</function_call>\\n        <function_result>{{'response': {{'responseCode': '200', 'responseBody': \\\"<answer>\\\\n<answer_part>\\\\n<text>The yearly 401k contribution limit is $20,500.</text>\\\\n<source>c546cbe8-07f6-45d1-90ca-74d87ab2885a</source>\\\\n</answer_part>\\\\n</answer>\\\"}}}}</function_result>\\n        <answer>\\n            <answer_part>\\n                <text>The deductible for your Delta Dental plan is $50 per benefit period.</text>\\n                <source>dfe040f8-46ed-4a65-b3ea-529fa55f6b9e</source>\\n            </answer_part>\\n            <answer_part>\\n                <text>If you have dependents enrolled, the maximum family deductible is $150 per benefit period.</text>\\n                <source>0e666064-31d8-4223-b7ba-8eecf40b7b47</source>\\n            </answer_part>\\n            <answer_part>\\n                <text>The yearly 401k contribution limit is $20,500.</text>\\n                <source>c546cbe8-07f6-45d1-90ca-74d87ab2885a</source>\\n            </answer_part>\\n        </answer>\\n    </example>\\n\\n    \\n\\n    <example_docstring>Here's a final example where the question asked could not be answered with information gathered from calling the provided functions. In this example, notice how you respond by telling the user you cannot answer, without using a function that was not provided to you.</example_docstring>\\n    <example>\\n        <functions>\\n            <function>\\n                <function_name>get::policyengineactions::getpolicyviolations</function_name>\\n                <function_description>Returns a list of policy engine violations for the specified alias within the specified date range.</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <required_argument>startDate (string): The start date of the range to filter violations. The format for startDate is MM/DD/YYYY.</required_argument>\\n                <required_argument>endDate (string): The end date of the range to filter violations</required_argument>\\n                <returns>array: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>post::policyengineactions::acknowledgeviolations</function_name>\\n                <function_description>Acknowledge policy engine violation. Generally used to acknowledge violation, once user notices a violation under their alias or their managers alias.</function_description>\\n                <required_argument>policyId (string): The ID of the policy violation</required_argument>\\n                <required_argument>expectedDateOfResolution (string): The date by when the violation will be addressed/resolved</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            <function>\\n                <function_name>get::activedirectoryactions::getmanager</function_name>\\n                <function_description>This API is used to identify the manager hierarchy above a given person. Every person could have a manager and the manager could have another manager to which they report to</function_description>\\n                <required_argument>alias (string): The alias of the employee under whose name current violations needs to be listed</required_argument>\\n                <returns>object: Successful response</returns>\\n                <raises>object: Invalid request</raises>\\n            </function>\\n            \\n        </functions>\\n        <question>Who are the reportees of David?</question>\\n        <scratchpad>\\n            After reviewing the functions I was equipped with, I realize I am not able to accurately answer this question since I can't access reportees of David. Therefore, I should explain to the user I cannot answer this question.\\n        </scratchpad>\\n        <answer>\\n            Sorry, I am unable to assist you with this request.\\n        </answer>\\n    </example>\\n</examples>\\n\\nThe above examples have been provided to you to illustrate general guidelines and format for use of function calling for information retrieval, and how to use your scratchpad to plan your approach. IMPORTANT: the functions provided within the examples should not be assumed to have been provided to you to use UNLESS they are also explicitly given to you within <functions></functions> tags below. All of the values and information within the examples (the questions, function results, and answers) are strictly part of the examples and have not been provided to you.\\n\\nNow that you have read and understood the examples, I will define the functions that you have available to you to use. Here is a comprehensive list.\\n\\n<functions>\\n<function>\\n<function_name>GET::optimal_departure_window_mars::getNextMarsLaunchWindow</function_name>\\n<function_description>Gets the next optimal launch window to Mars.</function_description>\\n<required_argument>specific_impulse (string): Specific impulse of the propulsion system (s).</required_argument>\\n<required_argument>dry_mass (string): Mass of the spacecraft without fuel (kg).</required_argument>\\n<required_argument>total_mass (string): Total mass of the spacecraft including fuel (kg)</required_argument>\\n<returns>object: The next optimal departure date for a Hohmann transfer from Earth to Mars, based on the spacecraft's mass and specific impulse.</returns>\\n</function>\\n\\n\\n</functions>\\n\\nNote that the function arguments have been listed in the order that they should be passed into the function.\\n\\n\\n\\nDo not modify or extend the provided functions under any circumstances. For example, GET::optimal_departure_window_mars::getNextMarsLaunchWindow with additional parameters would be considered modifying the function which is not allowed. Please use the functions only as defined.\\n\\nDO NOT use any functions that I have not equipped you with.\\n\\n Do not make assumptions about inputs; instead, make sure you know the exact function and input to use before you call a function.\\n\\nTo call a function, output the name of the function in between <function_call> and </function_call> tags. You will receive a <function_result> in response to your call that contains information that you can use to better answer the question. Or, if the function call produced an error, you will receive an <error> in response.\\n\\n\\n\\nThe format for all other <function_call> MUST be: <function_call>$FUNCTION_NAME($FUNCTION_PARAMETER_NAME=$FUNCTION_PARAMETER_VALUE)</function_call>\\n\\nRemember, your goal is to answer the user's question to the best of your ability, using only the function(s) provided within the <functions></functions> tags to gather more information if necessary to better answer the question.\\n\\nDo not modify or extend the provided functions under any circumstances. For example, calling GET::optimal_departure_window_mars::getNextMarsLaunchWindow with additional parameters would be modifying the function which is not allowed. Please use the functions only as defined.\\n\\nBefore calling any functions, create a plan for performing actions to answer this question within the <scratchpad>. Double check your plan to make sure you don't call any functions that you haven't been provided with. Always return your final answer within <answer></answer> tags.\\n\\n\\n\\nThe user input is <question>Answer the following question and pay strong attention to the prompt:\\n        <question>\\n        When is the next launch window for Mars? My spacecraft's total mass is 50000, dry mass is 10000 and specific impulse is 2500. Mass in Kg.\\n        </question>\\n        <instruction>\\n        You have functions available at your disposal to use when anwering any questions about orbital mechanics.if you can't find a function to answer a question about orbital mechanics, simply reply 'I do not know'\\n        </instruction></question>\\n\\n\\nAssistant: <scratchpad> I understand I cannot use functions that have not been provided to me to answer this question.\\n\\n\", \"traceId\": \"e0b2b2c2-fb7c-4e17-8a1f-a3781100face-0\", \"type\": \"ORCHESTRATION\"}, \"event_order\": 2}, {\"type\": \"modelInvocationOutput\", \"data\": {\"metadata\": {\"usage\": {\"inputTokens\": 5160, \"outputTokens\": 135}}, \"rawResponse\": {\"content\": \"To answer this question about the next Mars launch window, I will:\\n\\n1. Call the GET::optimal_departure_window_mars::getNextMarsLaunchWindow function to get the next optimal launch window, passing in the provided spacecraft mass and specific impulse values.\\n\\nI have verified that I have access to the GET::optimal_departure_window_mars::getNextMarsLaunchWindow function.\\n\\n</scratchpad>\\n\\n<function_call>\\nGET::optimal_departure_window_mars::getNextMarsLaunchWindow(specific_impulse=\\\"2500\\\", dry_mass=\\\"10000\\\", total_mass=\\\"50000\\\")\"}, \"traceId\": \"e0b2b2c2-fb7c-4e17-8a1f-a3781100face-0\"}, \"event_order\": 3}, {\"type\": \"rationale\", \"data\": {\"text\": \"To answer this question about the next Mars launch window, I will:\\n\\n1. Call the GET::optimal_departure_window_mars::getNextMarsLaunchWindow function to get the next optimal launch window, passing in the provided spacecraft mass and specific impulse values.\\n\\nI have verified that I have access to the GET::optimal_departure_window_mars::getNextMarsLaunchWindow function.\", \"traceId\": \"e0b2b2c2-fb7c-4e17-8a1f-a3781100face-0\"}, \"event_order\": 4}, {\"type\": \"invocationInput\", \"data\": {\"actionGroupInvocationInput\": {\"actionGroupName\": \"optimal_departure_window_mars\", \"apiPath\": \"/get-next-mars-launch-window\", \"executionType\": \"LAMBDA\", \"parameters\": [{\"name\": \"total_mass\", \"type\": \"string\", \"value\": \"50000\"}, {\"name\": \"dry_mass\", \"type\": \"string\", \"value\": \"10000\"}, {\"name\": \"specific_impulse\", \"type\": \"string\", \"value\": \"2500\"}], \"verb\": \"get\"}, \"invocationType\": \"ACTION_GROUP\", \"traceId\": \"e0b2b2c2-fb7c-4e17-8a1f-a3781100face-0\"}, \"event_order\": 5}, {\"type\": \"observation\", \"data\": {\"actionGroupInvocationOutput\": {\"text\": \"{\\\"next_launch_window\\\": {\\\"next_launch_date\\\": \\\"2026-11-26 00:00:00\\\", \\\"synodic_period_days\\\": 779.9068939794238, \\\"transfer_time_days\\\": 259, \\\"delta_v_available_m_s\\\": 39457.985759929674, \\\"delta_v_required_m_s\\\": 5595.997417810693, \\\"is_feasible\\\": true}}\"}, \"traceId\": \"e0b2b2c2-fb7c-4e17-8a1f-a3781100face-0\", \"type\": \"ACTION_GROUP\"}, \"event_order\": 6}]",
        "mlflow.spanFunctionName": "\"_action_group_trace\"",
        "mlflow.spanInputs": "{\"inner_trace_group\": \"{'actionGroupName': 'optimal_departure_window_mars', 'apiPath': '/get-next-mars-launch-window', 'executionType': 'LAMBDA', 'parameters': [{'name': 'total_mass', 'type': 'string', 'value': '50000'}, {'name': 'dry_mass', 'type': 'string', 'value': '10000'}, {'name': 'specific_impulse', 'type': 'string', 'value': '2500'}], 'verb': 'get'}\"}",
        "mlflow.spanOutputs": "\"{'action_group_name': 'optimal_departure_window_mars', 'api_path': '/get-next-mars-launch-window', 'execution_type': 'LAMBDA', 'execution_output': '{\\\"next_launch_window\\\": {\\\"next_launch_date\\\": \\\"2026-11-26 00:00:00\\\", \\\"synodic_period_days\\\": 779.9068939794238, \\\"transfer_time_days\\\": 259, \\\"delta_v_available_m_s\\\": 39457.985759929674, \\\"delta_v_required_m_s\\\": 5595.997417810693, \\\"is_feasible\\\": true}}'}\""
      },
      "events": []
    },
    {
      "name": "Retrieved Response",
      "context": {
        "span_id": "0xfe0b5f9149c39d7d",
        "trace_id": "0x9b8bd0b2e018d77f936e48a09e54fd44"
      },
      "parent_id": "0xb802165d133a33aa",
      "start_time": 1731388550225320000,
      "end_time": 1731388550226466000,
      "status_code": "OK",
      "status_message": "",
      "attributes": {
        "mlflow.traceRequestId": "\"1e036cc3a7f946ec995f7763b8dde51c\"",
        "mlflow.spanType": "\"AGENT\"",
        "mlflow.spanInputs": "[{\"role\": \"user\", \"content\": \"When is the next launch window for Mars? My spacecraft's total mass is 50000, dry mass is 10000 and specific impulse is 2500. Mass in Kg.\", \"name\": null}]",
        "mlflow.spanOutputs": "{\"choices\": [{\"index\": 0, \"message\": {\"role\": \"user\", \"content\": \"Based on the provided spacecraft dry mass of 10000 kg, total mass of 50000 kg, and specific impulse of 2500 s, the next optimal launch window for a Hohmann transfer from Earth to Mars is on November 26, 2026 UTC. The transfer will take 259 days.\", \"name\": null}, \"finish_reason\": \"stop\", \"logprobs\": null}], \"usage\": {\"prompt_tokens\": null, \"completion_tokens\": null, \"total_tokens\": null}, \"id\": null, \"model\": \"anthropic.claude-v2\", \"object\": \"chat.completion\", \"created\": 1731388550}"
      },
      "events": []
    }
  ],
  "request": "{\"context\": \"<mlflow.pyfunc.model.PythonModelContext object at 0x13397c530>\", \"messages\": [{\"role\": \"user\", \"content\": \"When is the next launch window for Mars? My spacecraft's total mass is 50000, dry mass is 10000 and specific impulse is 2500. Mass in Kg.\", \"name\": null}], \"params\": {\"temperature\": 1.0, \"max_tokens\": null, \"stop\": null, \"n\": 1, \"stream\": false, \"top_p\": null, \"top_k\": null, \"frequency_penalty\": null, \"presence_penalty\": null}}",
  "response": "{\"choices\": [{\"index\": 0, \"message\": {\"role\": \"user\", \"content\": \"Based on the provided spacecraft dry mass of 10000 kg, total mass of 50000 kg, and specific impulse of 2500 s, the next optimal launch window for a Hohmann transfer from Earth to Mars is on November 26, 2026 UTC. The transfer will take 259 days.\", \"name\": null}, \"finish_reason\": \"stop\", \"logprobs\": null}], \"usage\": {\"prompt_tokens\": null, \"completion_tokens\": null, \"total_tokens\": null}, \"id\": null, \"model\": \"anthropic.claude-v2\", \"object\": \"chat.completion\", \"created\": 1731388550}"
}

在 MLflow UI 中可视化跟踪细分

1. 提交给 Bedrock Agent 的初始提示。

2. 在此跟踪中，我们可以观察到 Bedrock Agent 如何评估并选择最适合当前任务的 Action Group。

3. 选择 Action Group 后，将跟踪其调用，显示与底层 Lambda 函数的输入和输出交互，如上文 OpenAPI Spec 中所述。

4. 此外，Bedrock 的补充跟踪包含在“Attributes”部分下，以及如下图所示的其他元数据

5. 随后，Agent 的最终响应被跟踪，如下图所示。

注意：我们无法将 span 的持续时间分解为单个跟踪持续时间，因为 Bedrock Agent 的跟踪响应不包含每个跟踪步骤的时间戳。

总结

在这篇博文中，我们探讨了如何将 AWS Bedrock Agent 集成到 MLflow 中作为 ChatModel 使用，重点关注 Action Groups、Knowledge Bases 和 Tracing。我们展示了如何利用 MLflow 灵活而强大的 API 轻松构建自定义 ChatModel。这种方法使您能够利用 MLflow 的跟踪和日志记录功能，即使对于 MLflow 原生不支持的模型或 Flavor 也是如此。

本博文的关键要点

• 将带有 Action Groups 的 Bedrock Agent 部署为 AWS Lambda 函数
  • 我们介绍了如何设置 Bedrock Agent，并使用 Action Groups 中的 AWS Lambda 函数实现自定义操作。
• 将 AWS Bedrock Agent 的自定义跟踪映射到 MLflow span/trace 对象
  • 我们演示了如何将 Agent 的自定义跟踪数据转换为 MLflow span 对象，以提高可观察性。
• 将 Bedrock Agent 记录并加载为 MLflow ChatModel
  • 我们展示了如何将 Bedrock Agent 作为 ChatModel 记录到 MLflow 中，以及如何加载它以备将来使用。
• 外部化 AWS Client 和 Bedrock 配置
  • 我们解释了如何外部化 AWS 客户端和 Bedrock 配置，以保护秘密信息，并使其易于调整模型设置而无需重新记录模型。

延伸阅读与参考资料

• Amazon Bedrock Agent 的工作原理
• Amazon Bedrock 跟踪
• 创建自定义 GenAI 聊天 Agent
• AWS 代码示例存储库