.Net: Why do AzureTextCompletion and OpenAITextCompletion exist?

[stephentoub]

Important

Labeled Urgent because it may require a breaking change if we should remove OpenAITextCompletion.

As far as I can tell, its functionality is 100% available via AzureChatCompletion. Why do we have both?

Same question for OpenAITextCompletion and OpenAIChatCompletion

[dmytrostruk]

@stephentoub These were initial classes to interact with Azure OpenAI/OpenAI models. When Chat API became available, we also added new classes to support chat functionality.

Now, since OpenAI text completion API will be deprecated, we will need to do the same for text completion classes (i.e. mark them with Obsolete flags, give some time for users to replace them with Chat classes, then remove them).

[stephentoub]

Now, since OpenAI text completion API will be deprecated, we will need to do the same for text completion classes

Even if they weren't being deprecated, though, the chat completion classes are pure supersets of the text completion ones: as far as I can tell, they share the exact same implementation for the text completion functionality and then the chat types add more. So there's zero need for the text completion ones, functionally.

Regardless, I'm glad to hear they're going away.

[matthewbolanos]

Per conversation, consider deprecating the AzureTextCompletion and OpenAITextCompletion connectors in favor of just having everyone use the Chat Completion versions.

[matthewbolanos]

At the very least, we should add the experimental tag to these.

[stephentoub]

At the very least, we should add the experimental tag to these.

Why not just delete them? Specifically what benefit at all do they add? As far as I can tell, there's literally nothing you can do with them you can't do with the other types.

[dmytrostruk]

At the very least, we should add the experimental tag to these.

Why not just delete them? Specifically what benefit at all do they add? As far as I can tell, there's literally nothing you can do with them you can't do with the other types.

@stephentoub A couple of weeks ago I tried to remove them and change all integration tests to use chat completion models and it appeared that there is some overload on Azure/OpenAI endpoints when all tests start to use just chat completion endpoint. Since our code changed a lot, I will try to do that again and see if our tests will pass successfully. In the future, we will improve our testing infrastructure and add some load balancing to avoid such issues. The final goal is still to remove text completion services, it's in progress now.