Skip to main content
Version: v0.4.2

Experimental APIs

This section contains APIs that are currently in development and may have limited support or stability. These APIs are available for testing and feedback but should not be used in production environments.

Experimental Notice

These APIs are experimental and may change without notice. Use with caution and provide feedback to help improve them.

Current Experimental APIs​

Batch Inference API​

Run inference on a dataset of inputs in batch mode for improved efficiency.

Status: In Development Provider Support: Limited Use Case: Large-scale inference operations

Features:

  • Batch processing of multiple inputs
  • Optimized resource utilization
  • Progress tracking and monitoring

Batch Agents API​

Run agentic workflows on a dataset of inputs in batch mode.

Status: In Development Provider Support: Limited Use Case: Large-scale agent operations

Features:

  • Batch agent execution
  • Parallel processing capabilities
  • Result aggregation and analysis

Synthetic Data Generation API​

Generate synthetic data for model development and testing.

Status: Early Development Provider Support: Very Limited Use Case: Training data augmentation

Features:

  • Automated data generation
  • Quality control mechanisms
  • Customizable generation parameters

Batches API (OpenAI-compatible)​

OpenAI-compatible batch management for inference operations.

Status: In Development Provider Support: Limited Use Case: OpenAI batch processing compatibility

Features:

  • OpenAI batch API compatibility
  • Job scheduling and management
  • Status tracking and monitoring

Getting Started with Experimental APIs​

Prerequisites​

  • Llama Stack server running with experimental features enabled
  • Appropriate provider configurations
  • Understanding of API limitations

Configuration​

Experimental APIs may require special configuration flags or provider settings. Check the specific API documentation for setup requirements.

Usage Guidelines​

  1. Testing Only: Use experimental APIs for testing and development only
  2. Monitor Changes: Watch for updates and breaking changes
  3. Provide Feedback: Report issues and suggest improvements
  4. Backup Data: Always backup important data when using experimental features

Feedback and Contribution​

We encourage feedback on experimental APIs to help improve them:

Reporting Issues​

  • Use GitHub issues with the "experimental" label
  • Include detailed error messages and reproduction steps
  • Specify the API version and provider being used

Feature Requests​

  • Submit feature requests through GitHub discussions
  • Provide use cases and expected behavior
  • Consider contributing implementations

Testing​

  • Test experimental APIs in your environment
  • Report performance issues and optimization opportunities
  • Share success stories and use cases

Migration to Stable APIs​

As experimental APIs mature, they will be moved to the stable API section. When this happens:

  1. Announcement: We'll announce the promotion in release notes
  2. Migration Guide: Detailed migration instructions will be provided
  3. Deprecation Timeline: Experimental versions will be deprecated with notice
  4. Support: Full support will be available for stable versions

Provider Support​

Experimental APIs may have limited provider support. Check the specific API documentation for:

  • Supported providers
  • Configuration requirements
  • Known limitations
  • Performance characteristics

Roadmap​

Experimental APIs are part of our ongoing development roadmap:

  • Q1 2024: Batch Inference API stabilization
  • Q2 2024: Batch Agents API improvements
  • Q3 2024: Synthetic Data Generation API expansion
  • Q4 2024: Batches API full OpenAI compatibility

For the latest updates, follow our GitHub releases and roadmap discussions.