[BUG] Promptflow Error unsupported operand type(s) for +: 'dict' and 'dict

[DrGriff]

Describe the bug
I am using Prompt Flow (in VS Code and Azure) and am getting an issue where my call to the AzureOpenAI client results in an error when calling my client.chat.completions.create(....). The error is:

How To Reproduce the bug
The error occurs third step

1. Call the client.chat.completions.create(...) method
2. This works. I inspect the response object and find that it didn't call the right Tools' Function.
3. I then call the client.chat.completions.create(...) method again and it returns the following:

line 79, in execute
completion = client.chat.completions.create(**completion_params) # type: ignore
File "/azureml-envs/prompt-flow/runtime/lib/python3.9/site-packages/promptflow/tracing/_integrations/_openai_injector.py", line 95, in wrapper
return f(*args, **kwargs)\n File "/azureml-envs/prompt-flow/runtime/lib/python3.9/site-packages/promptflow/tracing/_trace.py", line 562, in wrapped
token_collector.collect_openai_tokens_for_parent_span(span)
File "/azureml-envs/prompt-flow/runtime/lib/python3.9/site-packages/promptflow/tracing/_trace.py", line 143, in collect_openai_tokens_for_parent_span
merged_tokens = {\n File "/azureml-envs/prompt-flow/runtime/lib/python3.9/site-packages/promptflow/tracing/_trace.py", line 145, in
key: (self._span_id_to_tokens[parent_span_id].get(key, 0) or 0) + (tokens.get(key, 0) or 0)
TypeError: unsupported operand type(s) for +: 'dict' and 'dict'

I followed the advice given in:

https://learn.microsoft.com/en-us/answers/questions/2136875/promptflow-error-unsupported-operand-type(s)-for-d

• I tried disabling the tracing, but the exception was still thrown. Even if this had stopped the exception it would still have caused issues since we require the details of the tokens consumed to be returned.
• I tried changing the version of my PromptFlow (went all the way back to 1.15.0 from 1.16.0, but still have the issue)

Expected behavior
This was working perfectly on 2024-12-22 and has since stopped working with no changes to my code. The expected behaviour is that this should still be working. Based on the link given above, it's affecting a lot of people!

Screenshots
I think the above explanation gives everything you need.

Running Information(please complete the following information):
pf -v gives:

{
  "promptflow": "1.16.2",       
  "promptflow-core": "1.16.2",  
  "promptflow-devkit": "1.16.2",
  "promptflow-tracing": "1.16.2"
}

But I did down-grade this iteratively to 1.15.0 and still the problem exists.

Operating System when running in VS Code is Windows 10.

Python 3.12.2

Additional context
Like others, I'm dead in the water until this is fixed.

[SophieBoule99]

I am facing a similar issue but with unsupported operand type for dict and int. I have also not made any changes to my code and have tried to roll back to previous promptflow versions.

I have also tried rolling back the openai but it seems to not be compatible with newer versions of promptflow that I would need.

[ChrnyaevEK]

@DrGriff hi! Do you use images in prompt? I have the same issue with same setup as you, but it seems like promptflow is failing due to error you mentioned only when image is in the prompt. I've tried default LLM with vision tool (in preview), it also cause flow to fail (I have custom python tool that calls Open AI). However when promptflow is composed with only text LLMs tools (no images), error is not observed (even when such tools are chained).

[DrGriff]

@ChrnyaevEK

@DrGriff hi! Do you use images in prompt?

No, I don't use images in prompts.

[DSmmartin]

I have the same bug, but in my case this error happens randomly, or I don't found any pattern, I cover in a try except block where makes a retry.
In your case @DrGriff the TypeError is happening always?

[DrGriff]

@DSmmartin

I have the same bug, but in my case this error happens randomly, or I don't found any pattern, I cover in a try except block where makes a retry. In your case @DrGriff the TypeError is happening always?

It's happening consistently, but only in the manner I describe. So if I make one call via the Client then it succeeds. But a second call, even with a brand new client, results in failure.

Tip: I'm making the call to my LLM within a Python block, so have been able to solve this by not using the SDK client and instead call the LLM via an HTTP call and handle the JSON response directly.

[nihadtz]

It is happening while using Azure OpenAI API in my case. When I use local models via LLM studio this issue is not present.

Code has not been changed, but we built new images.

After investigation this is related to number of tokens calculation:

promptflow/src/promptflow-tracing/promptflow/tracing/_trace.py

Line 154 in 407970e

merged_tokens[key] = int(parent_value or 0) + int(token_value or 0)

parent_value and token_value sometimes are not integers.

[jomalsan]

#3751 has discussion and recommendations for the same thing

[jackdgolding]

Upgrading all promptflow packages to the Jan 6th release, promptflow 1.17.0, fixed the issue for me. The fix is in the release notes.
https://pypi.org/project/promptflow/1.17.0/

[nihadtz]

Upgrading all promptflow packages to the Jan 6th release, promptflow 1.17.0, fixed the issue for me. The fix is in the release notes. https://pypi.org/project/promptflow/1.17.0/

The issue with 1.17 is marshmallow which needs to be fixed to the 3.22.0
#3904

[orasik]

Upgrading to 1.17.0 fixed the issue, but if you are sending traces to Azure App Insights, then it is still broken. I've raised a PR #3911