artkit.model.llm.huggingface.base.HuggingfaceConnectorMixin#

class artkit.model.llm.huggingface.base.HuggingfaceConnectorMixin(*, model_id, api_key_env=None, initial_delay=None, exponential_base=None, jitter=None, max_retries=None, **model_params)[source]#

Base class for Huggingface LLMs.

Bases:

ConnectorMixin [~T_Client]

Generic types:

~T_Client

Metaclasses:

ABCMeta

Parameters:

model_id (str) – the ID of the model to use
api_key_env (Optional[str]) – the environment variable that holds the API key; if not specified, use the default API key environment variable for the model as returned by get_default_api_key_env()
initial_delay (Optional[float]) – the initial delay in seconds between client requests (defaults to 1.0)
exponential_base (Optional[float]) – the base for the exponential backoff (defaults to 2.0)
jitter (Optional[bool]) – whether to add jitter to the delay (defaults to True)
max_retries (Optional[int]) – the maximum number of retries for client requests (defaults to 5)
model_params (Any) – additional model parameters, passed with every request; parameters with None values are considered unset and will be ignored

Method summary

`get_api_key`	Get the API key from the environment variable specified by `api_key_env`.
`get_client`	Get a shared client instance for this connector; return a cached instance if for this model's API key if available.
`get_default_api_key_env`	Get the default name of the environment variable that holds the API key.
`get_model_params`	Get the parameters of the model as a mapping.
`get_tokenizer`	The tokenizer used by this connector.
`to_expression`	Render this object as an expression.

Attribute summary

`model_id`	The ID of the model to use.
`api_key_env`	The environment variable that holds the API key.
`initial_delay`	The initial delay in seconds between client requests.
`exponential_base`	The base for the exponential backoff.
`jitter`	Whether to add jitter to the delay.
`max_retries`	The maximum number of retries for client requests.
`model_params`	Additional model parameters, passed with every request.

Definitions

get_api_key()#

Get the API key from the environment variable specified by api_key_env.

Return type:: str
Returns:: the API key
Raises:: ValueError – if the environment variable is not set

get_client()#

Get a shared client instance for this connector; return a cached instance if for this model’s API key if available.

Return type:: HuggingfaceConnectorMixin
Returns:: the shared client instance

classmethod get_default_api_key_env()[source]#

Get the default name of the environment variable that holds the API key.

Return type:: str
Returns:: the default name of the api key environment variable

get_model_params()#

Get the parameters of the model as a mapping.

This includes all parameters that influence the model’s behavior, but not parameters that determine the model itself or are are specific to the client such as the model ID or the API key.

Return type:: Mapping[str, Any]
Returns:: the model parameters

final get_tokenizer()[source]#

The tokenizer used by this connector.

Gets loaded on first access.

Return type:: AutoTokenizer
Returns:: the tokenizer

to_expression()#

Render this object as an expression.

Return type:: Expression
Returns:: the expression representing this object

api_key_env: str#: The environment variable that holds the API key.

exponential_base: float#: The base for the exponential backoff.

initial_delay: float#: The initial delay in seconds between client requests.

jitter: bool#: Whether to add jitter to the delay.

max_retries: int#: The maximum number of retries for client requests.

property model_id: str#: The ID of the model to use.

model_params: dict[str, Any]#: Additional model parameters, passed with every request.