remote::nvidia

Description

NVIDIA inference provider for accessing NVIDIA NIM models and AI services.

Configuration

Field	Type	Required	Default	Description
allowed_models	list[str] | None	No		List of models that should be registered with the model registry. If None, all models are allowed.
refresh_models	bool	No	False	Whether to refresh models periodically from the provider
api_key	SecretStr | None	No		Authentication credential for the provider
network	NetworkConfig | None	No		Network configuration including TLS, proxy, and timeout settings.
network.tls	TLSConfig | None	No		TLS/SSL configuration for secure connections.
network.tls.verify	bool | Path	No	True	Whether to verify TLS certificates. Can be a boolean or a path to a CA certificate file.
network.tls.min_version	Literal[TLSv1.2, TLSv1.3] | None	No		Minimum TLS version to use. Defaults to system default if not specified.
network.tls.ciphers	list[str] | None	No		List of allowed cipher suites (e.g., ['ECDHE+AESGCM', 'DHE+AESGCM']).
network.tls.client_cert	Path | None	No		Path to client certificate file for mTLS authentication.
network.tls.client_key	Path | None	No		Path to client private key file for mTLS authentication.
network.proxy	ProxyConfig | None	No		Proxy configuration for HTTP connections.
network.proxy.url	HttpUrl | None	No		Single proxy URL for all connections (e.g., 'http://proxy.example.com:8080').
network.proxy.http	HttpUrl | None	No		Proxy URL for HTTP connections.
network.proxy.https	HttpUrl | None	No		Proxy URL for HTTPS connections.
network.proxy.cacert	Path | None	No		Path to CA certificate file for verifying the proxy's certificate. Required for proxies in interception mode.
network.proxy.no_proxy	list[str] | None	No		List of hosts that should bypass the proxy (e.g., ['localhost', '127.0.0.1', '.internal.corp']).
network.timeout	float | TimeoutConfig | None	No		Timeout configuration. Can be a float (for both connect and read) or a TimeoutConfig object with separate connect and read timeouts.
network.timeout.connect	float | None	No		Connection timeout in seconds.
network.timeout.read	float | None	No		Read timeout in seconds.
network.headers	dict[str, str] | None	No		Additional HTTP headers to include in all requests.
base_url	HttpUrl | None	No	https://integrate.api.nvidia.com/v1	A base url for accessing the NVIDIA NIM
timeout	int	No	60	Timeout for the HTTP requests
rerank_model_to_url	dict[str, str]	No	{'nv-rerank-qa-mistral-4b:1': 'https://ai.api.nvidia.com/v1/retrieval/nvidia/reranking', 'nvidia/nv-rerankqa-mistral-4b-v3': 'https://ai.api.nvidia.com/v1/retrieval/nvidia/nv-rerankqa-mistral-4b-v3/reranking', 'nvidia/llama-3.2-nv-rerankqa-1b-v2': 'https://ai.api.nvidia.com/v1/retrieval/nvidia/llama-3_2-nv-rerankqa-1b-v2/reranking'}	Mapping of rerank model identifiers to their API endpoints.

Sample Configuration

base_url: ${env.NVIDIA_BASE_URL:=https://integrate.api.nvidia.com/v1}
api_key: ${env.NVIDIA_API_KEY:=}