remote::runpod

Description

RunPod inference provider for running models on RunPod's cloud GPU platform.

Configuration

Field	Type	Required	Default	Description
allowed_models	list[str | None	No		List of models that should be registered with the model registry. If None, all models are allowed.
refresh_models	<class 'bool'>	No	False	Whether to refresh models periodically from the provider
api_token	pydantic.types.SecretStr | None	No		The API token
url	str | None	No		The URL for the Runpod model serving endpoint

Sample Configuration

url: ${env.RUNPOD_URL:=}
api_token: ${env.RUNPOD_API_TOKEN}