remote::runpod

Description

RunPod inference provider for running models on RunPod's cloud GPU platform.

Configuration

Field	Type	Required	Default	Description
allowed_models	list[str] | None	No		List of models that should be registered with the model registry. If None, all models are allowed.
refresh_models	bool	No	False	Whether to refresh models periodically from the provider
api_token	SecretStr | None	No		The API token
network	NetworkConfig | None	No		Network configuration including TLS, proxy, and timeout settings.
network.tls	TLSConfig | None	No		TLS/SSL configuration for secure connections.
network.tls.verify	bool | Path	No	True	Whether to verify TLS certificates. Can be a boolean or a path to a CA certificate file.
network.tls.min_version	Literal[TLSv1.2, TLSv1.3] | None	No		Minimum TLS version to use. Defaults to system default if not specified.
network.tls.ciphers	list[str] | None	No		List of allowed cipher suites (e.g., ['ECDHE+AESGCM', 'DHE+AESGCM']).
network.tls.client_cert	Path | None	No		Path to client certificate file for mTLS authentication.
network.tls.client_key	Path | None	No		Path to client private key file for mTLS authentication.
network.proxy	ProxyConfig | None	No		Proxy configuration for HTTP connections.
network.proxy.url	HttpUrl | None	No		Single proxy URL for all connections (e.g., 'http://proxy.example.com:8080').
network.proxy.http	HttpUrl | None	No		Proxy URL for HTTP connections.
network.proxy.https	HttpUrl | None	No		Proxy URL for HTTPS connections.
network.proxy.cacert	Path | None	No		Path to CA certificate file for verifying the proxy's certificate. Required for proxies in interception mode.
network.proxy.no_proxy	list[str] | None	No		List of hosts that should bypass the proxy (e.g., ['localhost', '127.0.0.1', '.internal.corp']).
network.timeout	float | TimeoutConfig | None	No		Timeout configuration. Can be a float (for both connect and read) or a TimeoutConfig object with separate connect and read timeouts.
network.timeout.connect	float | None	No		Connection timeout in seconds.
network.timeout.read	float | None	No		Read timeout in seconds.
network.headers	dict[str, str] | None	No		Additional HTTP headers to include in all requests.
base_url	HttpUrl | None	No		The URL for the Runpod model serving endpoint

Sample Configuration

base_url: ${env.RUNPOD_URL:=}
api_token: ${env.RUNPOD_API_TOKEN:=}