Prompts

Prompts are the initial instructions given to agents at the start of an episode. They provide the context and goal for the task, forming the agent’s initial observation (s₀ in RL terminology).

What is a Prompt?

A prompt is a sequence of blocks (text and/or images) that describes the task to the agent:

type Prompt = Blocks  // Array of TextBlock | ImageBlock

Purpose: -Tell the agent what problem to solve -Provide initial context -Set expectations for the episode

Accessing Prompts

Prompts are retrieved via the API after creating a session:

GET /gsm8k/prompt
X-Session-ID: abc-123

Response:

[
  {
    "text": "What is 2+2?",
    "detail": null,
    "type": "text"
  }
]

Typical flow:

Create session with task
Get prompt for that task
Agent reads prompt to understand what to do
Agent begins calling tools to solve task

Prompt Structure

Text Prompts

Simple text instructions:

[
  {
    "text": "Solve this math problem: If x + 5 = 12, what is x?",
    "detail": null,
    "type": "text"
  }
]

Multi-Line Prompts

Complex instructions:

[
  {
    "text": "You are given a Python programming challenge.\n\nWrite a function that reverses a string.\n\nRequirements:\n-Function name: reverse_string\n-Input: string\n-Output: reversed string\n\nYou have access to a Python REPL. Submit your function when ready.",
    "detail": null,
    "type": "text"
  }
]

Text + images:

[
  {
    "text": "What object is shown in this image?",
    "detail": null,
    "type": "text"
  },
  {
    "data": "iVBORw0KGgoAAAANSUhEUgA...",
    "mimeType": "image/jpeg",
    "detail": null,
    "type": "image"
  }
]

Generating Prompts

Prompts are generated by the environment’s get_prompt() method:

from openreward.environments import Environment, tool
from openreward.environments.types import Blocks, TextBlock, ToolOutput

class MathEnvironment(Environment):
    def get_prompt(self) -> Blocks:
        """Generate prompt from task"""
        question = self.task_spec["question"]

        return [
            TextBlock(
                text=f"Solve this math problem: {question}",
                type="text"
            )
        ]

Key points: -Prompts are dynamically generated per task -Use self.task_spec to access task data -Return list of blocks (even for single text)

Prompt Design Patterns

Pattern 1: Simple Question

Direct question from task:

def get_prompt(self) -> Blocks:
    return [TextBlock(text=self.task_spec["question"])]

Example:

“What is the capital of France?”

Pattern 2: Contextual Instructions

Add context and instructions:

def get_prompt(self) -> Blocks:
    problem = self.task_spec["problem"]

    prompt_text = f"""You are a coding assistant. Solve the following problem:

{problem}

You have access to a Python REPL. Use the 'python' tool to write and test code.
When you have a solution, use the 'submit' tool.
"""

    return [TextBlock(text=prompt_text)]

Pattern 3: Role-Playing

Set agent persona:

def get_prompt(self) -> Blocks:
    scenario = self.task_spec["scenario"]

    prompt_text = f"""You are a customer service agent.

A customer has the following issue:
{scenario}

Your goal is to resolve their issue professionally and efficiently.
Use the 'respond' tool to communicate with the customer.
"""

    return [TextBlock(text=prompt_text)]

Include images:

import base64

def get_prompt(self) -> Blocks:
    question = self.task_spec["question"]
    image_path = self.task_spec["image_path"]

    # Load and encode image
    with open(image_path, "rb") as f:
        image_data = base64.b64encode(f.read()).decode()

    return [
        TextBlock(text=question),
        ImageBlock(
            data=image_data,
            mimeType="image/png",
            type="image"
        )
    ]

Pattern 5: Few-Shot Examples

Provide examples in prompt:

def get_prompt(self) -> Blocks:
    problem = self.task_spec["problem"]

    prompt_text = f"""Solve the following math problem.

Examples:
-Problem: "2 + 2" → Answer: 4
-Problem: "10 - 3" → Answer: 7
-Problem: "5 * 6" → Answer: 30

Now solve:
Problem: "{problem}"

Use the 'submit' tool with your answer.
"""

    return [TextBlock(text=prompt_text)]

Pattern 6: Tool Instructions

Explain available tools:

def get_prompt(self) -> Blocks:
    goal = self.task_spec["goal"]

    prompt_text = f"""Goal: {goal}

Available tools:
- bash: Execute shell commands to explore the file system
- read_file: Read file contents
- submit: Submit your final answer

Start by exploring the environment, then submit your answer.
"""

    return [TextBlock(text=prompt_text)]

Best Practices

1. Be Specific

# Good - clear instructions
"Solve this equation for x: 2x + 5 = 13"

# No Bad - vague
"Do the math thing"

2. Include Tool Guidance

# Good - explains tools
"""Find the answer in /data/answers.txt

Use the 'read_file' tool to read files.
When you find the answer, use the 'submit' tool.
"""

# No Bad - no guidance
"Find the answer in /data/answers.txt"

3. Set Clear Success Criteria

# Good - defines success
"""Write a function that reverses a string.

Success criteria:
-Function name: reverse_string
-Passes all test cases
-No syntax errors
"""

# No Bad - unclear goal
"Write some code about strings"

4. Provide Context

# Good - gives context
"""You are debugging a web application.

Error log:

TypeError: Cannot read property ‘name’ of undefined

Find and fix the bug. Use 'bash' to explore files.
"""

# No Bad - no context
"Fix the bug"

5. Format for Readability

# Good - well-formatted
prompt_text = """
Task: Analyze customer feedback

Input: {feedback}

Steps:
1. Read the feedback carefully
2. Identify sentiment (positive/negative/neutral)
3. Extract key themes
4. Submit your analysis

Use the 'submit' tool with your results.
""".strip()

# No Bad - wall of text
prompt_text = "Task: Analyze customer feedback Input: {feedback} Steps: 1. Read the feedback carefully 2. Identify sentiment..."

Dynamic Prompts

Prompts can be customized based on task properties:

def get_prompt(self) -> Blocks:
    difficulty = self.task_spec.get("difficulty", "medium")

    if difficulty == "easy":
        instructions = "This is a simple problem. Take your time."
    elif difficulty == "hard":
        instructions = "This is a challenging problem. Think carefully and use all available tools."
    else:
        instructions = "Solve this problem step by step."

    problem = self.task_spec["problem"]

    return [TextBlock(text=f"{instructions}\n\nProblem: {problem}")]

Prompt Templates

Reusable prompt structures:

PROMPT_TEMPLATE = """
{role_context}

Task: {task_description}

{tool_instructions}

{success_criteria}
"""

class Environment(Environment):
    def get_prompt(self) -> Blocks:
        prompt_text = PROMPT_TEMPLATE.format(
            role_context="You are a helpful coding assistant.",
            task_description=self.task_spec["description"],
            tool_instructions="Use 'python' to run code, 'submit' to submit solution.",
            success_criteria="Your code must pass all test cases."
        )

        return [TextBlock(text=prompt_text)]

Multi-Turn Prompts

For environments with dialogue or multi-step interactions, prompts can evolve:

class DialogueEnvironment(Environment):
    def __init__(self, task_spec, **kwargs):
        super().__init__(task_spec, **kwargs)
        self.conversation_history = []

    def get_prompt(self) -> Blocks:
        # Initial prompt
        if not self.conversation_history:
            return [TextBlock(text=self.task_spec["initial_message"])]

        # Subsequent prompts include history
        history_text = "\n".join(self.conversation_history)
        return [TextBlock(text=f"Conversation so far:\n{history_text}\n\nYour turn:")]

    @tool
    def respond(self, params: RespondParams) -> ToolOutput:
        self.conversation_history.append(f"You: {params.message}")

        # Generate response...
        response = generate_response(params.message)
        self.conversation_history.append(f"Other: {response}")

        return ToolOutput(
            blocks=[TextBlock(text=response)],
            reward=0.0,
            finished=False
        )

Note: In typical ORS usage, get_prompt() is called once at episode start. For evolving context, include context in tool outputs rather than regenerating prompts.

Debugging Prompts

View Actual Prompt

# Test what prompt looks like
curl http://localhost:8080/gsm8k/prompt \
  -H "X-Session-ID: abc-123"

Common Issues

Issue: Prompt is empty

Cause: get_prompt() returns empty list
Fix: Ensure returning non-empty Blocks

Issue: Prompt is unclear to agent

Cause: Ambiguous instructions
Fix: Add more context and tool guidance

Issue: Prompt too long

Cause: Including too much information
Fix: Focus on essentials, use tools for additional info

Next Steps

Tools

Design tools agents use after reading prompts

Tasks & Splits

Organize tasks that prompts are generated from

Implementing a Server

Implement get_prompt() in your environment

Data Types

See prompt data structure (Blocks)

Key Takeaway: Prompts are the agent’s starting point for each episode. Design them to be clear, specific, and informative. Good prompts guide agents toward successful task completion by setting context and explaining available tools.

Getting Started

Specification

Core Concepts

Implementation Guides

Comparison

Prompts

Prompts

What is a Prompt?

Accessing Prompts

Prompt Structure

Text Prompts

Multi-Line Prompts

Generating Prompts

Prompt Design Patterns

Pattern 1: Simple Question

Pattern 2: Contextual Instructions

Pattern 3: Role-Playing

Pattern 5: Few-Shot Examples

Pattern 6: Tool Instructions

Best Practices

1. Be Specific

2. Include Tool Guidance

3. Set Clear Success Criteria

4. Provide Context

5. Format for Readability

Dynamic Prompts

Prompt Templates

Multi-Turn Prompts

Debugging Prompts

View Actual Prompt

Common Issues

Next Steps

Tools

Tasks & Splits

Implementing a Server

Data Types

Getting Started

Specification

Core Concepts

Implementation Guides

Comparison

​Prompts

​What is a Prompt?

​Accessing Prompts

​Prompt Structure

​Text Prompts

​Multi-Line Prompts

​Multi-Modal Prompts

​Generating Prompts

​Prompt Design Patterns

​Pattern 1: Simple Question

​Pattern 2: Contextual Instructions

​Pattern 3: Role-Playing

​Pattern 4: Multi-Modal

​Pattern 5: Few-Shot Examples

​Pattern 6: Tool Instructions

​Best Practices

​1. Be Specific

​2. Include Tool Guidance

​3. Set Clear Success Criteria

​4. Provide Context

​5. Format for Readability

​Dynamic Prompts

​Prompt Templates

​Multi-Turn Prompts

​Debugging Prompts

​View Actual Prompt

​Common Issues

​Next Steps

Tools

Tasks & Splits

Implementing a Server

Data Types

Prompts

What is a Prompt?

Accessing Prompts

Prompt Structure

Text Prompts

Multi-Line Prompts

Multi-Modal Prompts

Generating Prompts

Prompt Design Patterns

Pattern 1: Simple Question

Pattern 2: Contextual Instructions

Pattern 3: Role-Playing

Pattern 4: Multi-Modal

Pattern 5: Few-Shot Examples

Pattern 6: Tool Instructions

Best Practices

1. Be Specific

2. Include Tool Guidance

3. Set Clear Success Criteria

4. Provide Context

5. Format for Readability

Dynamic Prompts

Prompt Templates

Multi-Turn Prompts

Debugging Prompts

View Actual Prompt

Common Issues

Next Steps