Python: Retrieve (or log) Token Counts During Function Execution

[Gal-Gilor]

Hi,

I’m trying to get a precise token counts from kernel, and planners, especially the Stepwise planner, during function execution.

Unfortunately, I did not find the documentation to help with that.

That said , I noticed that I can use PromptTemplateConfig, PromptTemplate, PromptTemplateEngine, SemanticFunctionConfig, and the context object to render the prompt.

If it’s not yet feasible, is it possible to add to the ‘context’ another variable with the rendered prompt or the entire execution token counts (input, outputs both?).

Thoughts?

Thanks!

[moonbox3]

Hi @Gal-Gilor, which version of the SK Python package are you using? By the sound of the classes you're referring to, you're not using the latest beta package. I would highly recommend you upgrade to the latest package so you can take advantage of our latest enhancements. We have introduced a lot of changes to bring us closer to parity with dotnet. For example, functions now return a FunctionResult object that does contain the token count (both input and completion tokens) for OpenAI models.

The usage key is present in the FunctionResult's metadata. As we move towards v1 we do have work items to improve/update our docs. This will be documented soon.

[nehalakhani]

Hi @moonbox3 - Is there alternative way to calculate token based on latest Semantic Kernel version 1.8.3

[Gal-Gilor]

Thanks for the quick response! Unfortunately, I am forced to used the older version. That said, I will bring that up.

[moonbox3]

Could you help us understand why you're forced to use the older version? Is it an internal requirement? We won't be able to provide any support for SK Python previous to our beta package. Please keep this in mind. Thanks.

[Gal-Gilor]

It is an internal requirement. You can consider this as resolved.