> ## Documentation Index
> Fetch the complete documentation index at: https://pulze.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Custom Routers

> Build intelligent routing logic trained on your proprietary data

# Custom Routers

Custom routers allow you to build intelligent model routing logic trained on your organization's proprietary data and evaluation results. Beyond our pre-built routers (pulze-v0.1 and pulze-v1.0), you can create routing systems that automatically select the optimal AI model based on your unique requirements.

## What Are Custom Routers?

Custom routers are intelligent systems that:

* **Automatically Select Models**: Choose the best AI model for each request
* **Learn from Your Data**: Train on your proprietary evaluation datasets
* **Adapt to Your Needs**: Optimize for your specific use cases and quality bars
* **Maintain Control**: Allow per-space or per-request overrides when needed

## Pre-Built Routers

### pulze-v0.1

Our open-sourced router with proven performance:

* **Public Benchmarks**: Trained on publicly available evaluation data
* **Proven Track Record**: Battle-tested across thousands of organizations
* **Open Source**: Fully transparent routing logic
* **Default Option**: Enabled by default for all organizations

### pulze-v1.0

Enhanced router with advanced capabilities:

* **Automatic Model Discovery**: New models are automatically routed without manual targeting
* **Improved Evaluation Quality**: Significantly enhanced prompt quality during training
* **Seamless Integration**: Works alongside existing router infrastructure
* **Security-First**: Not enabled by default - requires testing and approval

> [!NOTE]
>
> pulze-v1.0 is not enabled by default in your organization. Your system continues using pulze-v0.1 until you complete testing together with the Pulze team and are comfortable with the results.

## Building Custom Routers

### Why Build Custom Routers?

Build custom routers when you need:

* **Domain-Specific Optimization**: Route based on your industry or use case
* **Proprietary Quality Standards**: Match your organization's unique quality bars
* **Cost-Performance Trade-offs**: Optimize for your specific budget constraints
* **Compliance Requirements**: Route based on data residency or security policies

### Router Types

> [card] Per-Space Routers
>
> Create custom routing for specific workspaces with their own logic

> [card] Organization-Wide Routers
>
> Build routing logic that applies across your entire organization

> [card] Evaluation-Based Routers
>
> Train routers on your own evaluation runs and quality metrics

> [card] Hybrid Routers
>
> Combine multiple routing strategies for maximum flexibility

## How Custom Routers Work

### 1. Create Evaluation Datasets

Build datasets from multiple sources:

* **Benchmark Data**: Public benchmarks relevant to your domain
* **Liked Prompts**: Conversations your team has favorited
* **Manual Additions**: Hand-crafted test cases for your use cases
* **Production Data**: Real queries from your users

**1. Select Data Sources**

Choose which sources to include in your dataset

**2. Curate Examples**

Review and refine your test cases

**3. Tag and Categorize**

Organize by use case, difficulty, or other dimensions

**4. Save Dataset**

Create a reusable evaluation dataset

### 2. Run Evaluations

Test models against your datasets:

* **Multiple Models**: Evaluate several models simultaneously
* **Quality Metrics**: Track accuracy, relevance, tone, compliance
* **Performance Analysis**: Compare speed, cost, and quality
* **Audit Trails**: Maintain complete records of why models pass or fail

### 3. Train Custom Router

Use evaluation results to build routing logic:

* **Automatic Training**: Router learns from evaluation performance
* **Quality Thresholds**: Set minimum quality bars for model selection
* **Cost Optimization**: Balance quality and budget constraints
* **Continuous Improvement**: Retrain as new models become available

### 4. Deploy and Monitor

Roll out your custom router:

* **Gradual Rollout**: Test with select spaces before org-wide deployment
* **Performance Tracking**: Monitor routing decisions and outcomes
* **Override Capability**: Users can still target specific models when needed
* **Iteration**: Refine based on real-world performance

## Dataset Builder

Create comprehensive evaluation datasets tailored to your needs.

### Data Sources

> [card] Benchmark Data
>
> Public benchmarks relevant to your domain (coding, math, reasoning, etc.)

> [card] Liked Prompts
>
> Conversations your team has marked as high-quality examples

> [card] Manual Additions
>
> Hand-crafted test cases for your specific requirements

> [card] Production Queries
>
> Real queries from your users (with appropriate privacy controls)

### Dataset Management

* **Version Control**: Track changes to datasets over time
* **Tagging System**: Organize by category, difficulty, use case
* **Sharing Options**: Share datasets across your organization
* **Import/Export**: Bring in external benchmarks or export for analysis

## Evaluation Engine

Run comprehensive evaluations with full audit trails.

### Evaluation Features

**Quality Metrics**

* Response accuracy and correctness
* Tone and style consistency
* Compliance with guidelines
* Citation and source quality
* Format and structure adherence

**Performance Metrics**

* Response latency
* Token usage and cost
* Throughput and concurrency
* Error rates and reliability

**Audit Trail**

* Complete evaluation history
* Model comparison reports
* Pass/fail reasoning
* Stakeholder explainability

### Use Cases

> [tab] Q\&A Teams
>
> **Complete Transparency**
>
> * See exactly why models pass or fail
> * Understand model strengths and weaknesses
> * Make data-driven model selection decisions
> * Share results with stakeholders

> [tab] Compliance
>
> **Compliance-Ready Reports**
>
> * Download detailed evaluation reports
> * Explain model enablement decisions
> * Demonstrate due diligence
> * Meet audit requirements

> [tab] Product Teams
>
> **Performance Tracking**
>
> * Evaluate models, AI assistants, and spaces
> * Track quality over time
> * Compare against proprietary benchmarks
> * Validate before production deployment

## Router Configuration

### Organization Settings

Control default router behavior at the org level:

**Default Router Selection**

* Choose which router (v0.1, v1.0, or custom) your org uses by default
* Set different defaults for different spaces
* Override at the request level when needed

**Conversation Naming Model**

* Select default model for creating conversation names
* Optimize for speed or quality
* Configure per-space if needed

**Compliance-Safe Fallback**

* If a model is restricted, system automatically selects from available models
* Stays compliant with org policies
* Maintains user experience without disruption

### Space-Level Configuration

Customize routing per workspace:

* **Space-Specific Routers**: Different routing logic per space
* **Model Allowlists**: Restrict which models can be used
* **Quality Thresholds**: Set minimum quality requirements
* **Cost Controls**: Cap spending per space

### Request-Level Overrides

Maintain flexibility when needed:

* **Model Targeting**: Explicitly request a specific model
* **Router Bypass**: Skip routing for specific requests
* **A/B Testing**: Compare router vs. manual selection
* **Debugging**: Understand why specific models were chosen

## Best Practices

### Start Simple

1. **Begin with Pre-Built**: Use pulze-v0.1 or pulze-v1.0 first
2. **Identify Gaps**: Find where pre-built routers don't meet your needs
3. **Build Incrementally**: Start with one space or use case
4. **Validate Thoroughly**: Test extensively before org-wide rollout

### Build Quality Datasets

* **Representative Samples**: Include diverse examples from real usage
* **Edge Cases**: Don't forget unusual or difficult scenarios
* **Regular Updates**: Keep datasets current as your product evolves
* **Balanced Coverage**: Include easy, medium, and hard examples

### Monitor and Iterate

* **Track Routing Decisions**: Understand which models are selected and why
* **Gather Feedback**: Ask users about model performance
* **Compare Results**: Run periodic evaluations to validate router performance
* **Continuous Improvement**: Retrain as new models and data become available

### Maintain Control

* **Keep Override Capability**: Users should be able to target specific models
* **Document Decisions**: Explain routing logic to stakeholders
* **Set Guardrails**: Define clear quality and cost boundaries
* **Plan Fallbacks**: Handle cases where routing fails or models are unavailable

## Advanced Features

### Multi-Objective Optimization

Balance multiple factors:

* Quality vs. cost trade-offs
* Speed vs. accuracy requirements
* Compliance vs. capability needs
* User preference vs. org standards

### Context-Aware Routing

Route based on:

* Query complexity and type
* User role and permissions
* Space configuration and data
* Time of day and load
* Previous conversation context

### Ensemble Routing

Combine multiple models:

* Use multiple models for same request
* Compare and validate responses
* Confidence-based selection
* Voting mechanisms for final answer

## Example Use Cases

### Financial Services

**Requirement**: High accuracy, audit trails, cost optimization

**Solution**:

* Custom router trained on financial benchmarks
* Quality threshold: 95%+ accuracy on math/reasoning
* Compliance fallback to approved model list
* Detailed audit logs for every routing decision

### Healthcare

**Requirement**: HIPAA compliance, high accuracy, data residency

**Solution**:

* Region-specific router respecting data residency
* Only routes to HIPAA-compliant models
* Quality validation on medical terminology
* Automatic fallback if compliant models unavailable

### Software Development

**Requirement**: Code quality, multiple languages, speed

**Solution**:

* Language-specific routing (Python → Model A, JS → Model B)
* Quality benchmarks for code correctness
* Speed optimization for interactive coding
* Cost limits for background batch processing

## Getting Started

**1. Review Current Performance**

Analyze how pre-built routers perform for your use cases

**2. Identify Requirements**

Define your unique quality, cost, and compliance needs

**3. Build Dataset**

Create evaluation dataset from benchmarks and production data

**4. Run Evaluations**

Test models against your dataset to understand performance

**5. Train Router**

Use evaluation results to build custom routing logic

**6. Test Thoroughly**

Validate router performance in non-production space

**7. Deploy Gradually**

Roll out to one space, then expand based on results

**8. Monitor and Iterate**

Track performance and refine based on real-world usage

## Next Steps

> [card] [Evaluations](https://pulze.ai/docs/features/evaluations)
>
> Learn how to create and run evaluations

> [card] [Dataset Management](https://pulze.ai/docs/features/datasets)
>
> Build and manage evaluation datasets

> [card] [Model Catalog](https://pulze.ai/docs/models/overview.md)
>
> Explore available models for routing

> [card] [Organization Settings](https://pulze.ai/docs/features/organization)
>
> Configure org-wide router defaults


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.