artkit.model.llm.base.HTTPXChatConnector#

class artkit.model.llm.base.HTTPXChatConnector(*, model_id, api_key_env=None, initial_delay=None, exponential_base=None, jitter=None, max_retries=None, system_prompt=None, httpx_client_kwargs=None, **model_params)[source]#

ABC that represents connecting to a custom endpoint.

Bases:

ChatModelConnector [AsyncClient]

Metaclasses:

ABCMeta

Parameters:

model_id (str) – the ID of the model to use
api_key_env (Optional[str]) – the environment variable that holds the API key; if not specified, use the default API key environment variable for the model as returned by get_default_api_key_env()
initial_delay (Optional[float]) – the initial delay in seconds between client requests (defaults to 1.0)
exponential_base (Optional[float]) – the base for the exponential backoff (defaults to 2.0)
jitter (Optional[bool]) – whether to add jitter to the delay (defaults to True)
max_retries (Optional[int]) – the maximum number of retries for client requests (defaults to 5) parameters with None values are considered unset and will be ignored
system_prompt (Optional[str]) – the system prompt to initialise the chat system with (optional)
httpx_client_kwargs (Optional[dict[str, Any]]) – the kwargs to use for making the httpx.AsyncClient (optional):param model_params: additional model parameters, passed with every request;

Method summary

`build_request_arguments`	This method is responsible for formatting the input to the LLM chat system.
`get_api_key`	Get the API key from the environment variable specified by `api_key_env`.
`get_client`	Get the client to use for making requests.
`get_default_api_key_env`	Get the default name of the environment variable that holds the API key.
`get_model_params`	Get the parameters of the model as a mapping.
`get_response`	Get a response, or multiple alternative responses, from the chat system.
`parse_httpx_response`	This method is responsible for formatting the `httpx.Response` after having made the request.
`to_expression`	Render this object as an expression.
`with_system_prompt`	Set the system prompt for the LLM system.

Attribute summary

`model_id`	The ID of the model to use.
`system_prompt`	The system prompt used to set up the LLM system.
`api_key_env`	The environment variable that holds the API key.
`initial_delay`	The initial delay in seconds between client requests.
`exponential_base`	The base for the exponential backoff.
`jitter`	Whether to add jitter to the delay.
`max_retries`	The maximum number of retries for client requests.
`model_params`	Additional model parameters, passed with every request.

Definitions

abstract build_request_arguments(message, *, history=None, **model_params)[source]#

This method is responsible for formatting the input to the LLM chat system. For argument options see httpx.AsyncClient.request.

Parameters:

message (str) – The input message to format.
history (Optional[ChatHistory]) – The chat history preceding the message.
model_params (dict[str, Any]) – Additional parameters for the chat system.

Return type:

dict[str, Any]

Returns:

The necessary httpx request arguments.

get_api_key()#

Get the API key from the environment variable specified by api_key_env.

Return type:: str
Returns:: the API key
Raises:: ValueError – if the environment variable is not set

get_client()[source]#

Get the client to use for making requests. Allows us to create a new async client every time we’re in a new event loop. Internally, we can cache the client using a weak key dictionary that maps event loop objects to async clients

Return type:: AsyncClient
Returns:: The client to use for making requests.

abstract classmethod get_default_api_key_env()#

Get the default name of the environment variable that holds the API key.

Return type:: str
Returns:: the default name of the api key environment variable

get_model_params()#

Get the parameters of the model as a mapping.

This includes all parameters that influence the model’s behavior, but not parameters that determine the model itself or are are specific to the client such as the model ID or the API key.

Return type:: Mapping[str, Any]
Returns:: the model parameters

async get_response(message, *, history=None, **model_params)[source]#

Get a response, or multiple alternative responses, from the chat system.

Parameters:

message (str) – the user prompt to pass to the chat system
history (Optional[ChatHistory]) – the chat history preceding the message
model_params (dict[str, Any]) – additional parameters for the chat system

Return type:

list[str]

Returns:

the response or alternative responses generated by the chat system

Raises:

RequestLimitException – if an error occurs while communicating with the chat system

abstract parse_httpx_response(response)[source]#

This method is responsible for formatting the httpx.Response after having made the request.

Parameters:: response (Response) – The response from the endpoint.
Return type:: list[str]
Returns:: A list of formatted response strings.

to_expression()#

Render this object as an expression.

Return type:: Expression
Returns:: the expression representing this object

with_system_prompt(system_prompt)#

Set the system prompt for the LLM system.

Parameters:: system_prompt (str) – the system prompt to use
Return type:: Self
Returns:: a new LLM system with the system prompt set

api_key_env: str#: The environment variable that holds the API key.

exponential_base: float#: The base for the exponential backoff.

initial_delay: float#: The initial delay in seconds between client requests.

jitter: bool#: Whether to add jitter to the delay.

max_retries: int#: The maximum number of retries for client requests.

property model_id: str#: The ID of the model to use.

model_params: dict[str, Any]#: Additional model parameters, passed with every request.

property system_prompt: str | None#: The system prompt used to set up the LLM system.