artkit.model.llm.vertexai.VertexAIChat#

class artkit.model.llm.vertexai.VertexAIChat(*, model_id, api_key_env=None, initial_delay=1, exponential_base=2, jitter=True, max_retries=10, system_prompt=None, region=None, gcp_project_id, safety=True, **model_params)[source]#

Base class for Vertex AI LLMs.

Bases:

ChatModelConnector [GenerativeModel]

Metaclasses:

ABCMeta

Parameters:

model_id (str) – the ID of the model to use
api_key_env (Optional[str]) – the environment variable that holds the API key; if not specified, use the default API key environment variable for the model as returned by get_default_api_key_env()
initial_delay (float) – the initial delay in seconds between client requests (defaults to 1.0)
exponential_base (float) – the base for the exponential backoff (defaults to 2.0)
jitter (bool) – whether to add jitter to the delay (defaults to True)
max_retries (int) – the maximum number of retries for client requests (defaults to 5) parameters with None values are considered unset and will be ignored
system_prompt (Optional[str]) – the system prompt to initialise the chat system with (optional)
region (Optional[str]) – The specific GCP region to connect to.
gcp_project_id (str) – The GCP project ID.
safety (bool) – Safety settings for the model.:param model_params: additional model parameters, passed with every request;

Method summary

`get_api_key`	Get the API key from the environment variable specified by `api_key_env`.
`get_client`	Get a shared client instance for this connector; return a cached instance if for this model's API key if available.
`get_default_api_key_env`	Get the default name of the environment variable that holds the API key.
`get_model_params`	Get the parameters of the model as a mapping.
`get_response`	Send a message to the Vertex AI model and get a response.
`to_expression`	Render this object as an expression.
`with_system_prompt`	Set the system prompt for the LLM system.

Attribute summary

`model_id`	The ID of the model to use.
`system_prompt`	The system prompt used to set up the LLM system.
`region`
`gcp_project_id`
`safety`	If `False`, disable all accessible safety categories; otherwise, enable them.
`api_key_env`	The environment variable that holds the API key.
`initial_delay`	The initial delay in seconds between client requests.
`exponential_base`	The base for the exponential backoff.
`jitter`	Whether to add jitter to the delay.
`max_retries`	The maximum number of retries for client requests.
`model_params`	Additional model parameters, passed with every request.

Definitions

get_api_key()#

Get the API key from the environment variable specified by api_key_env.

Return type:: str
Returns:: the API key
Raises:: ValueError – if the environment variable is not set

get_client()#

Get a shared client instance for this connector; return a cached instance if for this model’s API key if available.

Return type:: VertexAIChat
Returns:: the shared client instance

classmethod get_default_api_key_env()[source]#

Get the default name of the environment variable that holds the API key.

Return type:: str
Returns:: the default name of the api key environment variable

get_model_params()[source]#

Get the parameters of the model as a mapping.

This includes all parameters that influence the model’s behavior, but not parameters that determine the model itself or are are specific to the client such as the model ID or the API key.

Return type:: Mapping[str, Any]
Returns:: the model parameters

async get_response(message, *, history=None, **model_params)[source]#

Send a message to the Vertex AI model and get a response.

Parameters:

message (str) – The input message to send to the model.
history (Optional[ChatHistory]) – The conversation history.
model_params (dict[str, Any]) – Additional parameters to pass to the model.

Return type:

list[str]

Returns:

The model’s response.

to_expression()#

Render this object as an expression.

Return type:: Expression
Returns:: the expression representing this object

with_system_prompt(system_prompt)#

Set the system prompt for the LLM system.

Parameters:: system_prompt (str) – the system prompt to use
Return type:: Self
Returns:: a new LLM system with the system prompt set

api_key_env: str#: The environment variable that holds the API key.

exponential_base: float#: The base for the exponential backoff.

initial_delay: float#: The initial delay in seconds between client requests.

jitter: bool#: Whether to add jitter to the delay.

max_retries: int#: The maximum number of retries for client requests.

property model_id: str#: The ID of the model to use.

model_params: dict[str, Any]#: Additional model parameters, passed with every request.

safety: bool#: If False, disable all accessible safety categories; otherwise, enable them.

property system_prompt: str | None#: The system prompt used to set up the LLM system.