Empromptu LogoEmpromptu

Input Optimization

Manage the data your AI applications use for testing and learning. Create manual test inputs and analyze real user data to improve performance.

Manual Inputs vs End User Inputs

📝Manual Inputs

Test data you create to optimize and validate your AI application before deployment.

Purpose:

  • •Provide controlled test scenarios for optimization
  • •Test edge cases and challenging inputs
  • •Establish baseline performance measurements
  • •Guide prompt optimization strategies

When to use:

  • •Before running any optimization
  • •When discovering new edge cases
  • •To test specific scenarios systematically
  • •For systematic performance validation

👥End User Inputs

Real data from users interacting with your deployed AI application.

Purpose:

  • •Monitor real-world performance patterns
  • •Identify new edge cases from actual usage
  • •Track performance trends over time
  • •Optimize based on actual user behavior

When to use:

  • •After deploying your application
  • •To understand real usage patterns
  • •When performance issues arise in live environments
  • •For continuous improvement strategies

3 Input Optimization Tabs

Access through Actions → Input Optimization. Each tab serves a different purpose in managing your input data.

📋Overview

Introduction to Manual vs End User inputs and how to use each type effectively.

📝Manual Inputs

Create and manage test data for optimization. Add inputs representing different scenarios and edge cases.

👥End User Inputs

View and analyze real user data with search, filter, and pattern identification capabilities.

Creating Manual Inputs

How to Create Manual Inputs

1

Access Manual Inputs

Click Actions → Input Optimization → Manual Inputs tab

2

Enter Test Data

Add your test data in the input fields

3

Include Representative Examples

Add examples that represent real use cases

4

Save for Optimization

Save the input to use in optimization processes

Input Structure

Each manual input contains:

Input Variables: The data your application will process
Context: Additional information or parameters
Expected Scenarios: What type of situation this represents

Best Practices for Manual Inputs

Include Different Input Types

Positive scenarios: Inputs that should work well
Negative scenarios: Challenging or problematic inputs
Edge cases: Unusual or boundary conditions
Typical usage: Common, everyday examples

Focus on Meaningful Examples

Each input should test specific scenarios
Include inputs that have caused problems before
Create inputs based on actual user feedback
Regularly update inputs as requirements evolve

Organize by Scenario Type

Length variations: Short, medium, and long inputs
Content types: Different subject matters or formats
Complexity levels: Simple to complex scenarios
Language patterns: Formal, casual, technical language

Analyzing End User Inputs

Input Log Features

The End User Inputs tab shows real usage data with:

📅Timestamp

When the API call occurred

🔧Input Variables

Exact data sent by the user

📝Response

Generated output from your application

📊Score

Performance rating for this interaction

Search and Filter Capabilities

Search inputs: Find specific examples or patterns
Filter by score: Focus on high or low-performing interactions
Date ranges: Analyze performance over time periods
Pattern identification: Look for common problem scenarios

Using Input Data for Optimization

Before Optimization

Create comprehensive manual inputs covering key scenarios
Run initial optimization to establish baseline performance
Review which inputs perform well vs poorly
Add more inputs for problematic scenarios

During Optimization

Use manual inputs to test prompt variations
Focus optimization on inputs with low scores
Create new inputs when discovering edge cases
Validate improvements across all input types

After Deployment

Review End User Inputs regularly for new patterns
Identify consistently low-scoring real user interactions
Add problematic real inputs as manual test cases
Re-optimize based on actual usage patterns

Input Analysis Strategies

Identifying Problem Patterns

Look for:

  • •Consistent low scores: Inputs that always perform poorly
  • •Score variation: Similar inputs with different performance
  • •New edge cases: User inputs you hadn't considered
  • •Performance degradation: Scores declining over time

Common Issues:

  • •Length-related issues: Long inputs scoring lower than short ones
  • •Format problems: Specific input formats causing confusion
  • •Content challenges: Certain topics performing poorly
  • •User behavior: Unexpected interaction patterns

Using Patterns for Optimization

1
Add representative manual inputs for identified problem patterns
2
Create specific evaluations targeting common issues
3
Run targeted prompt optimization for problematic scenarios
4
Test solutions against both manual and real user inputs

Input Management Workflows

Initial Setup Workflow

Create foundational manual inputs covering expected use cases
Include edge cases and challenging scenarios
Run initial optimization using manual inputs
Deploy application when manual input performance is satisfactory
Monitor end user inputs for real-world validation

Ongoing Maintenance Workflow

Weekly end user input review for new patterns
Monthly analysis of score trends and performance changes
Quarterly input refresh adding new manual inputs based on real usage
Continuous optimization when problematic patterns emerge

Next Steps

→ Prompt Optimization

Use your input data to build effective Prompt Families

→ Edge Case Detection

Use visual tools to identify problematic input patterns

→ Evaluations

Create criteria that work well across your input diversity

Get StartedView Documentation