.Net: ADR with proposal for the multiple LLM support use cases

[markwallace-microsoft]

Motivation and Context

• ADR describing the design for multiple LLM support. This will be expanded in the future to allow additional use cases such as selecting an AI service by LLM model id.
• The IAIServiceSelector has changed so that it is added to the IKernel instance, see example 62 for an example of a custom IAIServiceSelector implementation.

Contribution Checklist

• The code builds clean without any errors or warnings
• The PR follows the SK Contribution Guidelines and the pre-submission formatting script raises no violations
• All unit tests pass, and I have added new tests where possible
• I didn't break anyone 😄

[matthewbolanos]

Could we also support a scenario where the kernel keys off of "Model ID" instead of "Service ID"? In most cases, the semantic function author doesn't know the names of all the services that the kernel invoker has added, they'll just know the standard model names that are out in the wild (e.g., gpt-4, gpt-3.5-turbo, etc.). Leveraging "Model ID" should likely be the main/predominant scenario.

[markwallace-microsoft]

Could we also support a scenario where the kernel keys off of "Model ID" instead of "Service ID"? In most cases, the semantic function author doesn't know the names of all the services that the kernel invoker has added, they'll just know the standard model names that are out in the wild (e.g., gpt-4, gpt-3.5-turbo, etc.). Leveraging "Model ID" should likely be the main/predominant scenario.

Right now you could make the service if the same as the model id and that would work.

Currently matching specifically on model id is called out of scope because we have no way to store arbitrary attributes for a service. I'll add this as an additional task and will address in a follow up PR.

[SergeyMenshykh]

Could we also support a scenario where the kernel keys off of "Model ID" instead of "Service ID"? In most cases, the semantic function author doesn't know the names of all the services that the kernel invoker has added, they'll just know the standard model names that are out in the wild (e.g., gpt-4, gpt-3.5-turbo, etc.). Leveraging "Model ID" should likely be the main/predominant scenario.

I'm not sure, but it seems we need both the Service ID and Model ID to cover scenarios in which one connector provides multiple models and when the same model is provided by multiple connectors. Otherwise, the connector will have to be registered many times – one time per model it supports for the former scenario, or it won't be possible to register many connectors supporting the same model for the latter scenario.

[markwallace-microsoft]

Could we also support a scenario where the kernel keys off of "Model ID" instead of "Service ID"? In most cases, the semantic function author doesn't know the names of all the services that the kernel invoker has added, they'll just know the standard model names that are out in the wild (e.g., gpt-4, gpt-3.5-turbo, etc.). Leveraging "Model ID" should likely be the main/predominant scenario.

I'm not sure, but it seems we need both the Service ID and Model ID to cover scenarios in which one connector provides multiple models and when the same model is provided by multiple connectors. Otherwise, the connector will have to be registered many times – one time per model it supports for the former scenario, or it won't be possible to register many connectors supporting the same model for the latter scenario.

So right now we can do this:

IKernel kernel = new KernelBuilder()
    .WithAzureChatCompletionService(serviceId: "azure-gpt-4", deploymentName: "gpt-4", endpoint: endpoint, apiKey: apiKey)
    .WithAzureChatCompletionService(serviceId: "azure-gpt-3.5", deploymentName: "gpt-3.5", endpoint: endpoint, apiKey: apiKey)
    .WithOpenAIChatCompletionService(serviceId: "openai-gpt-4", modelId: "gpt-4", apiKey: openAIApiKey)
    .WithOpenAIChatCompletionService(serviceId: "openai-gpt-3.5", modelId: "gpt-3.5", apiKey: openAIApiKey)
    .Build();

I can write an IAIServiceSelector that picks the right service by service id

In the future we would like to be able to support this:

IKernel kernel = new KernelBuilder()
    .WithAzureChatCompletionService(modelId: "gpt-4", deploymentName: "foo", endpoint: endpoint, apiKey: apiKey)
    .WithAzureChatCompletionService(modelId: "gpt-3.5", deploymentName: "bar", endpoint: endpoint, apiKey: apiKey)
    .WithOpenAIChatCompletionService(modelId: "gpt-4", apiKey: openAIApiKey)
    .WithOpenAIChatCompletionService(modelId: "gpt-3.5", apiKey: openAIApiKey)
    .Build();

We need to solve how to write an IAIServiceSelector that picks the right service by model id and service type

[RogerBarreto]

This should be marked as a Breaking Change after making ServiceProvider internal.