remote::hf::endpoint

Description

HuggingFace Inference Endpoints provider for dedicated model serving.

Configuration

Field

Type

Required

Default

Description

endpoint_name

<class 'str'>

No

PydanticUndefined

The name of the Hugging Face Inference Endpoint in the format of ‘{namespace}/{endpoint_name}’ (e.g. ‘my-cool-org/meta-llama-3-1-8b-instruct-rce’). Namespace is optional and will default to the user account if not provided.

api_token

pydantic.types.SecretStr | None

No

Your Hugging Face user access token (will default to locally saved token if not provided)

Sample Configuration

endpoint_name: ${env.INFERENCE_ENDPOINT_NAME}
api_token: ${env.HF_API_TOKEN}