Connect custom LXM (LLM) with Unique Finance GPT
- Dominik Meyer
- Michelle Heppler
Unique provides the ability to connect a custom LXM (Large X model where X is language, video or audio, etc.) from within Unique Finance GPT to be able to use self- or cloud-hosted language models for the standard use cases of Unique Finance GPT.
This feature is EXPERIMENTAL. Learn what this means...
This feature is available from Release 2024-27 and later.
This feature is recommended to:
research models
benchmark models
develop new modules or features for a model
Unique does not recommend yet to build heavy production uses cases upon it due to the limitations listed below.
Technical Insight
To fulfil this use case Unique created two entities. One is called LanguageModelGroup and one is called ExternalLanguageModel. The LanguageModelGroup is an object containing mainly the groupName and is connected to the ExternalLanguageModel with an n:m relationship. The groupName basically replaces the hardcoded LLM enums of Unique. This fulfils the requirement that clients want to manage the LXMs by themselves.
The ExternalLanguageModel contains all the information which is needed for Unique to be able to call this custom provided LXM. This gives the ability to clients to be able to connect custom LXM with Unique.
How to Create and Connect
Currently Unique does not offer a user interface for it. So the following curl request will create an ExternalLanguageModel and connect it with the LanguageModelGroup. If the LanguageModelGroup is not existing yet, Unique will create a new LanguageModelGroup. If there is already one with this groupName Unique will add it to the existing one.
Create API
curl --location 'https://gateway.<tenantName>.unique.app/chat/graphql' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <token>' \
--data '{"query":"mutation Mutation($groupName: String!, $input: ExternalLanguageModelCreateInput!) {\n externalLanguageModelCreate(groupName: $groupName, input: $input) {\n id\n internalDescription\n languageModelGroups {\n id\n groupName\n }\n }\n}","variables":{"groupName":"<groupName>","input":{"version":"<modelVersion>","model":"<modelName>","maxTokens":<maxToken>,"internalDescription":"<internalDescription>","endpoint":"<endpoint>","deploymentName":"<deploymentName>","authHeaderValue":"<apiSecret>","apiVersion":"<apiVersion>"}}}'Field | Description | Azure OpenAI example |
|---|---|---|
tenantName | Unique Single Tenant Base URL |
|
token |
|
|
groupName | The |
|
modelVersion | The model version deployed at your endpoint |
 Model Version (here turbo-2024-04-09)
|
modelName | The model name deployed at your endpoint |
 Model Name (here gpt-4)
|
maxToken | The maximum amount of tokens that Unique will send in one request to the endpoint |
|
internalDescription | A description of the endpoint for Unique internal reference. You can use it later to reference information of yours to recognise the LXM. |
|
endpoint | The endpoint of your LXM |
|
deploymentName | The deployment name (gets added to the endpoint by the @azure/openai library) that you chose when deploying the model in a specific version |
|
apiSecret | The API secret, see #Security This is getting saved encrypted in the DB and only decrypted when Unique does the API request. There will be no possibility to access or change it afterwards, it is immutable write-only. |
|
apiVersion | API Version that the @azure/openai library will use to talk to the model. |
|
Azure OpenAI GPT-4o full example
curl --location 'https://gateway.qa.unique.app/chat/graphql' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer {TOKEN}' \
--data '{"query":"mutation Mutation($groupName: String!, $input: ExternalLanguageModelCreateInput!) {\n externalLanguageModelCreate(groupName: $groupName, input: $input) {\n id\n internalDescription\n languageModelGroups {\n id\n groupName\n }\n }\n}","variables":{"groupName":"GPT_4O_PTU_CUSTOM","input":{"version":"2024-05-13","model":"gpt-4o","maxTokens":126000,"internalDescription":"Uniques GPT 4o QA deployment in Sweden","endpoint":"https://unique-openai-083.openai.azure.com/","deploymentName":"gpt-4o-2024-05-13","authHeaderValue":"<secret>","apiVersion":"2024-02-01"}}}' \Renaming an Existing GroupName
During this process, after changing to the new groupName, the previous groupName will no longer be valid and all LXM configurations using the old groupName will fail! Until you have manually changed all previous configurations to the new one.
This renaming does not automatically rename all configurations done with the old groupName!
Due to this limitation, Unique recommends to first create a new group (including all the same ExternalLanguageModels - basically do the “Create API” chapter) under the new name, migrate all assistants and then execute a Delete Call for not deprecated objects.
To rename an existing groupName you need to know the current groupName. Then with this curl you can rename the groupName to a different new groupName.
curl --location 'https://gateway.<tenantName>.unique.app/chat/graphql' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <token>' \
--data '{"query":"mutation LanguageModelGroupUpdate($groupName: String!, $input: LanguageModelGroupUpdateInput!) {\n languageModelGroupUpdate(groupName: $groupName, input: $input) {\n groupName\n id\n }\n}","variables":{"groupName":"<currentGroupName>","input":{"groupName":"<newGroupName>"}}}'Field | Description | Azure OpenAI example |
|---|---|---|
tenantName | Unique Single Tenant Base URL |
|
token |
|
|
currentGroupName | The current |
|
newGroupName | The new |
|
Now the LanguageModelGroup.groupName is renamed to the new name and all models attached to it are now available under the new groupName. The old groupName is not valid anymore!
Deleting an Existing LanguageModelGroup
When you want to delete a LanguageModelGroup make first sure that the groupName related to this group is not used anymore in any LXM configuration. Otherwise this configuration will not work anymore.
After you made sure that this groupName is not used anymore you can use the following curl to delete the LanguageModelGroup.
Field | Description | Azure OpenAI example |
|---|---|---|
tenantName | Unique Single Tenant Base URL |
|
token |
|
|
groupNameToDelete | The |
|
Deleting an Existing ExternalLanguageModel
First make sure you do not delete the last ExternalLanguageModel assigned to a LanguageModelGroup which is still in use. You can do this with the following curl which will fetch all ExternalLanguageModels and its LanguageModelGroups.
Field | Description | Azure OpenAI example |
|---|---|---|
tenantName | Unique Single Tenant Base URL |
|
token |
|
|
Also in the response of this list of ExternalLanguageModels you also can find the id of the ExternalLanguageModel you would like to delete. This id then can be used to delete the ExternalLanguageModel with the following curl:
Field | Description | Azure OpenAI example |
|---|---|---|
tenantName | Unique Single Tenant Base URL |
|
token |
|
|
IdOfModelToDelete | The ID of the |
|
Other APIs
There is also the possibility for other kind of mutations via API on the ExternalLanguageModel and the LanguageModelGroup. Consult Unique support to learn more.
Connect
With this groupName you passed you are now able to change the languageModel attribute on the modules. From now on, this module configuration will use one of the ExternalLanguageModel, which is bound to this LanguageModelGroup.
Security
Connectivity
Naturally, as of now, the model must be reachable over the same network. For Unique PaaS and Single Tenants that means; the internet! Customer Managed or On Premise Tenants can leverage this functionality as well where their models must just be reachable from the deployments networks.
API Secret
Unique uses the AES-256-cbc algorithm with a 32 bytes random string to encrypt and decrypt the API Secret symmetrically. Each Unique Deployment has its own unique encryption key. This encryption key is stored in an Azure KeyVault vault of the environment and will be loaded on startup into the chat application. The same key is used for all organisations/companies within the one deployment/tenant.
By enabling clients to directly store the key encrypted, Unique must never receive the key and store it. Unique also does not see or need the key as the key gets decrypted just-in-time to send the request. Technically Unique employees can equally not retrieve the key from the database except in specially triggered incidents by the client where Unique employees get temporary access to inspect the clients data.
Company Scoping
The complete Connect custom LXM (LLM) with Unique Finance GPT feature is company-scoped. Companies cannot reuse LanguageModelGroups or ExternalLanguageModels from other companies. Of course, LanguageModelGroups and ExternalLanguageModels can be added to two or more companies though but that requires Administrators to explicitly send and define the same LXM for two or more companies/organisation.
Limitations
Reserved key words: There are reserved key words for groupNames which cannot be used. Basically are that the ones that Unique provides you as default starting with
AZURE_GPT_API only: Configurations can only be done via API currently.
GPT API standards: For the LXM network request Unique is using the @azure/openai library. Means the APIs of your LXM needs to be compatible with the GPT standards. This is also known as https://learn.microsoft.com/en-us/azure/ai-studio/reference/reference-model-inference-api?tabs=python.
Connecting one model to two groups: Its not possible to connect one ExternalLanguageModel to multiple LangaugeModelGroup. The API call for this mutation is currently missing and is in progress.
Author | @Adrian Gugger |
|---|
© 2024 Unique AG. All rights reserved. Privacy Policy – Terms of Service