remote::vllm

Description

Remote vLLM inference provider for connecting to vLLM servers.

Configuration

Field	Type	Required	Default	Description
allowed_models	list[str] | None	No		List of models that should be registered with the model registry. If None, all models are allowed.
refresh_models	bool	No	False	Whether to refresh models periodically from the provider
api_token	SecretStr | None	No		The API token
network	NetworkConfig | None	No		Network configuration including TLS, proxy, and timeout settings.
network.tls	TLSConfig | None	No		TLS/SSL configuration for secure connections.
network.tls.verify	bool | Path	No	True	Whether to verify TLS certificates. Can be a boolean or a path to a CA certificate file.
network.tls.min_version	Literal[TLSv1.2, TLSv1.3] | None	No		Minimum TLS version to use. Defaults to system default if not specified.
network.tls.ciphers	list[str] | None	No		List of allowed cipher suites (e.g., ['ECDHE+AESGCM', 'DHE+AESGCM']).
network.tls.client_cert	Path | None	No		Path to client certificate file for mTLS authentication.
network.tls.client_key	Path | None	No		Path to client private key file for mTLS authentication.
network.proxy	ProxyConfig | None	No		Proxy configuration for HTTP connections.
network.proxy.url	HttpUrl | None	No		Single proxy URL for all connections (e.g., 'http://proxy.example.com:8080').
network.proxy.http	HttpUrl | None	No		Proxy URL for HTTP connections.
network.proxy.https	HttpUrl | None	No		Proxy URL for HTTPS connections.
network.proxy.cacert	Path | None	No		Path to CA certificate file for verifying the proxy's certificate. Required for proxies in interception mode.
network.proxy.no_proxy	list[str] | None	No		List of hosts that should bypass the proxy (e.g., ['localhost', '127.0.0.1', '.internal.corp']).
network.timeout	float | TimeoutConfig | None	No		Timeout configuration. Can be a float (for both connect and read) or a TimeoutConfig object with separate connect and read timeouts.
network.timeout.connect	float | None	No		Connection timeout in seconds.
network.timeout.read	float | None	No		Read timeout in seconds.
network.headers	dict[str, str] | None	No		Additional HTTP headers to include in all requests.
base_url	HttpUrl | None	No		The URL for the vLLM model serving endpoint
max_tokens	int	No	4096	Maximum number of tokens to generate.
tls_verify	bool | str | None	No		DEPRECATED: Use 'network.tls.verify' instead. Whether to verify TLS certificates. Can be a boolean or a path to a CA certificate file.

Sample Configuration

base_url: ${env.VLLM_URL:=}
max_tokens: ${env.VLLM_MAX_TOKENS:=4096}
api_token: ${env.VLLM_API_TOKEN:=fake}
network:
  tls:
    verify: ${env.VLLM_TLS_VERIFY:=true}