Overwriting request attributes on content tests

[jpjpjp]

Hi.

I've managed to build some contract and variation tests and am now looking into content tests. In reading through the doc and learning the Portman syntax for testing attributes in the response body, I found myself wondering how I could write meaningful tests without knowing what the request parameter or request body is.

Upon inspection, it appears that the default behavior for Portman when it generates a request is to use the first example it finds in the OpenAPI spec for the request parameters and request body. Could you please confirm if my understanding is correct? If this behavior is documented, I would greatly appreciate a reference to that part of the documentation.

I did find the docs describe the use of overwrites, which seem to work in the following sections:

• Variation tests—The variationTests configuration supports a list of variations, which allow for an overwrites section. I've successfully used this to update request parameters/bodies and subsequent response tests on several unique requests.

• Overwrites—It also seems possible to have a top-level overwrites section, but the documentation doesn't make it 100% clear to me what the scope of these overwrites is. From observation, it appears to overwrite all the test requests?

  My goal is to create content tests that allow for a multitude of unique requests and tests per endpoint. I envision being able to manipulate the request parameters/bodies and subsequent response tests, like I was able to do in my variation tests Is this a feasible approach with Portman?

[thim81]

hi @jpjpjp

Portman uses the "assignVariables" in combination, with overwrite. It does not automatically can detect the primary identifier to be used in CRUD operations, since there is no indication for that in OpenAPI. But using the flow, like described below, you can easily have Portman generate ready-to-use variables and use these when making RUD requests.

Have a look at this example (which I used in a meetup):
https://github.com/thim81/spec-driven-openapi-contract-performance-testing/blob/main/testing-contract/config/portman-config-flow.yaml

assignVariables:
  - openApiOperation: POST::*
    collectionVariables:
      - responseBodyProp: id
        name: <tag>Id
  - openApiOperation: GET::*
    excludeForOperations:
      - "GET::/*/{id}"
    collectionVariables:
      - responseBodyProp: characters[0].id
        name: <tag>Id
overwrites:
  - openApiOperation: "*::/*/{id}"
    overwriteRequestPathVariables:
      - key: id
        value: '{{<tag>Id}}'
        overwrite: true

The part openApiOperation: POST::* will try to assign a variable from the "id" of the response body after a POST request.
The name name: <tag>Id of the variable is using the OpenAPI tag name in combination with "Id".

The part openApiOperation: GET::* will try to assign a variable from the nested "id" of the response body after a GET request.
The variable name name: <tag>Id is the same variable, and used the template option, which is using the OpenAPI tag name in combination with "Id".

For the usage of the assigned variables, we use the "overwriteRequestPathVariables", where we use the same template option as variable name to be insert the request path variable.

More examples and docs:

• https://github.com/apideck-libraries/portman/tree/main/examples/testsuite-assign-overwrite#template-expressions
• https://github.com/apideck-libraries/portman/tree/main/examples/testsuite-assign-variables

[jpjpjp]

Thanks @thim81, this is super helpful and definitely improves my understanding of how the assignVariables directive works!

I've got a follow-up question, but I'll mark this as answered since that definitely answered my question.