From 96e6e0761643993144688f9dd95bf60271402e5f Mon Sep 17 00:00:00 2001 From: SachinAkash01 Date: Tue, 17 Dec 2024 09:11:28 +0530 Subject: [PATCH 1/6] Add relaxed data binding proposal to the docs --- .../introducing_relaxed_data_binding.md | 98 +++++++++++++++++++ 1 file changed, 98 insertions(+) create mode 100644 docs/proposals/introducing_relaxed_data_binding.md diff --git a/docs/proposals/introducing_relaxed_data_binding.md b/docs/proposals/introducing_relaxed_data_binding.md new file mode 100644 index 0000000000..ed791b803a --- /dev/null +++ b/docs/proposals/introducing_relaxed_data_binding.md @@ -0,0 +1,98 @@ +# Proposal: Introduce Relaxed Data Binding to Ballerina HTTP Module + +_Owners_: @SachinAkash01 +_Reviewers_: @lnash94 @TharmiganK @shafreenAnfar +_Created_: 2024/11/13 +_Updated_: 2024/11/18 +_Issue_: [#7366](https://github.com/ballerina-platform/ballerina-library/issues/7366) + +## Summary + +Using the projection support from conversion APIs from the data module, the HTTP module will add a fix by providing a configurable to handle undefined nillable property scenarios where the user can choose strict data binding or relaxed data binding. + +## Goals +- Enhance the Ballerina HTTP module by adding support to handle undefined nillable properties where the users can choose strict data binding or relaxed data binding. + +## Motivation +According to OpenAPI specification if a given property is required or/and nullable we need to specifically mention it as explained below. + +If a given property is `required`, +```yaml +type: object +properties: + id: + type: integer + username: + type: string +required: + - id + - username +``` + +If a given property is `nullable`, +```yaml +type: integer +nullable: true +``` + +Therefore if a given property is not specifically mentioned as required, then that property is optional by default. Similarly if a given property is not specifically mentioned as `nullable: true`, then that property is not accepting null values by default. + +There are several scenarios where the received response has null values even though that property is not configured as `nullable: true`. In such cases http target type binding fails, returning a type conversion error. + +As a workaround, there is a `--nullable` flag to make all the fields nillable when it is not defined in the OpenAPI specification. However this approach also makes all the fields nullable which is not an ideal solution. + +A more user-friendly solution is to implement relaxed data binding, which would allow the tool to accept null values for non-nullable fields without raising runtime errors. This approach would handle unexpected null values gracefully, reducing type conversion issues and enhancing the developer experience by aligning more closely with API responses. + +## Description +Following is the breakdown of the cases we need to reconsider within the context of payload data binding: +
+Scenario | Ballerina Representation | Current Behavior | Expected Behavior +-- | -- | -- | -- +Required & non-nullable | T foo; | Both null values and absent fields cause runtime failures | Same +Required & nullable | T? foo; | Absent fields will cause runtime failures | Need relaxed data binding to map absent fields as nil values +Optional & non-nullable | T foo?; | Null values will cause runtime failures | Need relaxed data binding to map null values as absent fields +Optional & nullable | T? foo?; | Both nil values and absent fields are allowed | Same + +
 + +In the `data.jsondata` package, the `Options` configuration already provides support for handling various field types in JSON data binding, accommodating scenarios with both absent fields and nil values. The `Options` record allows us to adjust how nullability and optionality are treated, making it a versatile solution for managing the differences between required and optional fields, as well as nullable and non-nullable fields. + +The `Options` record in the data.jsondata module is defined as follows:, + +```ballerina +public type Options record { + record { + # If `true`, nil values will be considered as optional fields in the projection. + boolean nilAsOptionalField = false; + # If `true`, absent fields will be considered as nilable types in the projection. + boolean absentAsNilableType = false; + }|false allowDataProjection = {}; + boolean enableConstraintValidation = true; +}; +``` + +This configuration offers flexibility through two primary settings, +- `nilAsOptionalField`: when set to `true`, nil values are treated as optional fields, allowing relaxed data binding for Optional and Non-Nullable cases. This way, null values in the input can be mapped into an absent field, avoiding runtime failures. +- `absentAsOptionalField`: when enabled, absent fields are interpreted as nullable, providing support for Required & Nullable cases by mapping absent fields into nil values instead of causing runtime errors. + +### Configure Relaxed Data Binding in Client Level +To enable relaxed data binding for the HTTP client at the client level, we can integrate the desired behavior into the `ClientConfiguration` settings. Using `ClientConfiguration`, we can adjust the handling of null values and absent fields, enhancing compatibility with APIs that return inconsistent field specifications. + +The approach involves adding a new field, `laxDataBinding`, to the `CommonClientConfiguration` to enable relaxed data binding specifically for null and absent fields in the JSON payload. When `laxDataBinding` is set to `true`, it will internally map to enabling both `nilAsOptionalField` and `absentAsNilableType` in the `data.jsondata` module. This option will be enabled by default for clients generated using the Ballerina OpenAPI tool. + +```ballerina +public type CommonClientConfiguration record {| + boolean laxDataBinding = false; + // Other fields +|}; +``` + +### Configure Relaxed Data Binding in Service Level +We can configure the relaxed data binding in the service level by improving the `HttpServiceConfig` configuration by introducing a new boolean field called `laxDataBinding`, similar to the approach used in the client configuration. + +```ballerina +public type HttpServiceConfig record {| + boolean laxDataBinding = false; + // Other fields +|}; +``` From a9eb59bc414ee8546cd4e2d8d454f2de437ffd5b Mon Sep 17 00:00:00 2001 From: Sachin Akash Date: Tue, 17 Dec 2024 10:06:54 +0530 Subject: [PATCH 2/6] Apply suggestions from code review Co-authored-by: Krishnananthalingam Tharmigan <63336800+TharmiganK@users.noreply.github.com> --- docs/proposals/introducing_relaxed_data_binding.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/proposals/introducing_relaxed_data_binding.md b/docs/proposals/introducing_relaxed_data_binding.md index ed791b803a..a0f154c9ba 100644 --- a/docs/proposals/introducing_relaxed_data_binding.md +++ b/docs/proposals/introducing_relaxed_data_binding.md @@ -1,6 +1,6 @@ # Proposal: Introduce Relaxed Data Binding to Ballerina HTTP Module -_Owners_: @SachinAkash01 +_Owners_: @shafreenAnfar @daneshk @NipunaRanasinghe @lnash94 @TharmiganK @SachinAkash01 _Reviewers_: @lnash94 @TharmiganK @shafreenAnfar _Created_: 2024/11/13 _Updated_: 2024/11/18 @@ -55,9 +55,9 @@ Optional & nullable | T? foo?; | Both nil values and absent fields are allowed | -In the `data.jsondata` package, the `Options` configuration already provides support for handling various field types in JSON data binding, accommodating scenarios with both absent fields and nil values. The `Options` record allows us to adjust how nullability and optionality are treated, making it a versatile solution for managing the differences between required and optional fields, as well as nullable and non-nullable fields. +In the `data.jsondata` package, the `Options` configuration already provides support for handling various field types in JSON data binding, accommodating scenarios with both absent fields and null values. The `Options` record allows us to adjust how nullability and optionality are treated, making it a versatile solution for managing the differences between required and optional fields, as well as nullable and non-nullable fields. -The `Options` record in the data.jsondata module is defined as follows:, +The `Options` record in the `data.jsondata` module is defined as follows:, ```ballerina public type Options record { From aaf71c5b399585499b69b51b06e0ea3c2a542d46 Mon Sep 17 00:00:00 2001 From: Sachin Akash Date: Tue, 17 Dec 2024 10:12:36 +0530 Subject: [PATCH 3/6] Fix formatting issues --- docs/proposals/introducing_relaxed_data_binding.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/proposals/introducing_relaxed_data_binding.md b/docs/proposals/introducing_relaxed_data_binding.md index a0f154c9ba..da5bafc4e8 100644 --- a/docs/proposals/introducing_relaxed_data_binding.md +++ b/docs/proposals/introducing_relaxed_data_binding.md @@ -82,8 +82,8 @@ The approach involves adding a new field, `laxDataBinding`, to the `CommonClient ```ballerina public type CommonClientConfiguration record {| - boolean laxDataBinding = false; - // Other fields + boolean laxDataBinding = false; + // Other fields |}; ``` @@ -92,7 +92,7 @@ We can configure the relaxed data binding in the service level by improving the ```ballerina public type HttpServiceConfig record {| - boolean laxDataBinding = false; - // Other fields + boolean laxDataBinding = false; + // Other fields |}; ``` From 694329daf7cc4d1d26f9c9bc74b0362717c55520 Mon Sep 17 00:00:00 2001 From: Sachin Akash Date: Tue, 17 Dec 2024 10:14:23 +0530 Subject: [PATCH 4/6] Format table --- .../proposals/introducing_relaxed_data_binding.md | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-) diff --git a/docs/proposals/introducing_relaxed_data_binding.md b/docs/proposals/introducing_relaxed_data_binding.md index da5bafc4e8..e37bb78500 100644 --- a/docs/proposals/introducing_relaxed_data_binding.md +++ b/docs/proposals/introducing_relaxed_data_binding.md @@ -45,15 +45,12 @@ A more user-friendly solution is to implement relaxed data binding, which would ## Description Following is the breakdown of the cases we need to reconsider within the context of payload data binding: -
-Scenario | Ballerina Representation | Current Behavior | Expected Behavior --- | -- | -- | -- -Required & non-nullable | T foo; | Both null values and absent fields cause runtime failures | Same -Required & nullable | T? foo; | Absent fields will cause runtime failures | Need relaxed data binding to map absent fields as nil values -Optional & non-nullable | T foo?; | Null values will cause runtime failures | Need relaxed data binding to map null values as absent fields -Optional & nullable | T? foo?; | Both nil values and absent fields are allowed | Same - -
 +| Scenario | Ballerina Record Field Representation | Default Behavior | Relaxed Behavior | +|-------------------------|---------------------------------------|-----------------------------------------------------------|---------------------------------------------------------------| +| Required & non-nullable | `T foo;` | Both null values and absent fields cause runtime failures | Same | +| Required & nullable | `T? foo;` | Absent fields will cause runtime failures | Need relaxed data binding to map absent fields as nil values | +| Optional & non-nullable | `T foo?;` | Null values will cause runtime failures | Need relaxed data binding to map null values as absent fields | +| Optional & nullable | `T? foo?;` | Both nil values and absent fields are allowed | Same | In the `data.jsondata` package, the `Options` configuration already provides support for handling various field types in JSON data binding, accommodating scenarios with both absent fields and null values. The `Options` record allows us to adjust how nullability and optionality are treated, making it a versatile solution for managing the differences between required and optional fields, as well as nullable and non-nullable fields. From fcc579186cf663d351dd34326250f7bb3f1c3c74 Mon Sep 17 00:00:00 2001 From: SachinAkash01 Date: Tue, 17 Dec 2024 10:30:35 +0530 Subject: [PATCH 5/6] Refactor redundant information --- .../introducing_relaxed_data_binding.md | 39 ++++++++----------- 1 file changed, 17 insertions(+), 22 deletions(-) diff --git a/docs/proposals/introducing_relaxed_data_binding.md b/docs/proposals/introducing_relaxed_data_binding.md index ed791b803a..8a604fee2a 100644 --- a/docs/proposals/introducing_relaxed_data_binding.md +++ b/docs/proposals/introducing_relaxed_data_binding.md @@ -8,10 +8,10 @@ _Issue_: [#7366](https://github.com/ballerina-platform/ballerina-library/issues/ ## Summary -Using the projection support from conversion APIs from the data module, the HTTP module will add a fix by providing a configurable to handle undefined nillable property scenarios where the user can choose strict data binding or relaxed data binding. +Using the projection support from conversion APIs from the data module, the HTTP module will add a fix by providing a configurable to handle undefined nullable property scenarios where the user can choose strict data binding or relaxed data binding. ## Goals -- Enhance the Ballerina HTTP module by adding support to handle undefined nillable properties where the users can choose strict data binding or relaxed data binding. +- Enhance the Ballerina HTTP module by adding support to handle undefined nullable properties where the users can choose strict data binding or relaxed data binding. ## Motivation According to OpenAPI specification if a given property is required or/and nullable we need to specifically mention it as explained below. @@ -39,23 +39,20 @@ Therefore if a given property is not specifically mentioned as required, then th There are several scenarios where the received response has null values even though that property is not configured as `nullable: true`. In such cases http target type binding fails, returning a type conversion error. -As a workaround, there is a `--nullable` flag to make all the fields nillable when it is not defined in the OpenAPI specification. However this approach also makes all the fields nullable which is not an ideal solution. +As a workaround, there is a `--nullable` flag to make all the fields nullable when it is not defined in the OpenAPI specification. However this approach also makes all the fields nullable which is not an ideal solution. A more user-friendly solution is to implement relaxed data binding, which would allow the tool to accept null values for non-nullable fields without raising runtime errors. This approach would handle unexpected null values gracefully, reducing type conversion issues and enhancing the developer experience by aligning more closely with API responses. ## Description Following is the breakdown of the cases we need to reconsider within the context of payload data binding: -
-Scenario | Ballerina Representation | Current Behavior | Expected Behavior --- | -- | -- | -- -Required & non-nullable | T foo; | Both null values and absent fields cause runtime failures | Same -Required & nullable | T? foo; | Absent fields will cause runtime failures | Need relaxed data binding to map absent fields as nil values -Optional & non-nullable | T foo?; | Null values will cause runtime failures | Need relaxed data binding to map null values as absent fields -Optional & nullable | T? foo?; | Both nil values and absent fields are allowed | Same +| Scenario | Ballerina Record Field Representation | Default Behavior | Relaxed Behavior | +|-------------------------|---------------------------------------|-----------------------------------------------------------|---------------------------------------------------------------| +| Required & non-nullable | `T foo;` | Both null values and absent fields cause runtime failures | Same | +| Required & nullable | `T? foo;` | Absent fields will cause runtime failures | Need relaxed data binding to map absent fields as nil values | +| Optional & non-nullable | `T foo?;` | Null values will cause runtime failures | Need relaxed data binding to map null values as absent fields | +| Optional & nullable | `T? foo?;` | Both nil values and absent fields are allowed | Same | -
 - -In the `data.jsondata` package, the `Options` configuration already provides support for handling various field types in JSON data binding, accommodating scenarios with both absent fields and nil values. The `Options` record allows us to adjust how nullability and optionality are treated, making it a versatile solution for managing the differences between required and optional fields, as well as nullable and non-nullable fields. +In the `data.jsondata` package, the `Options` configuration already provides support for handling various field types in JSON data binding, accommodating scenarios with both absent fields and null values. The `Options` record allows us to adjust how nullability and optionality are treated, making it a versatile solution for managing the differences between required and optional fields, as well as nullable and non-nullable fields. The `Options` record in the data.jsondata module is defined as follows:, @@ -72,18 +69,16 @@ public type Options record { ``` This configuration offers flexibility through two primary settings, -- `nilAsOptionalField`: when set to `true`, nil values are treated as optional fields, allowing relaxed data binding for Optional and Non-Nullable cases. This way, null values in the input can be mapped into an absent field, avoiding runtime failures. -- `absentAsOptionalField`: when enabled, absent fields are interpreted as nullable, providing support for Required & Nullable cases by mapping absent fields into nil values instead of causing runtime errors. +- `nilAsOptionalField`: when set to `true`, null values are treated as optional fields, allowing relaxed data binding for Optional and Non-Nullable cases. This way, null values in the input can be mapped into an absent field, avoiding runtime failures. +- `absentAsOptionalField`: when enabled, absent fields are interpreted as nullable, providing support for Required & Nullable cases by mapping absent fields into null values instead of causing runtime errors. ### Configure Relaxed Data Binding in Client Level -To enable relaxed data binding for the HTTP client at the client level, we can integrate the desired behavior into the `ClientConfiguration` settings. Using `ClientConfiguration`, we can adjust the handling of null values and absent fields, enhancing compatibility with APIs that return inconsistent field specifications. - -The approach involves adding a new field, `laxDataBinding`, to the `CommonClientConfiguration` to enable relaxed data binding specifically for null and absent fields in the JSON payload. When `laxDataBinding` is set to `true`, it will internally map to enabling both `nilAsOptionalField` and `absentAsNilableType` in the `data.jsondata` module. This option will be enabled by default for clients generated using the Ballerina OpenAPI tool. +The approach involves adding a new field, `laxDataBinding`, to the `CommonClientConfiguration`. When set to true, it enables relaxed data binding for null and absent fields in the JSON payload by internally mapping to both `nilAsOptionalField` and `absentAsNilableType` in the `data.jsondata` module. This option will be enabled by default for clients generated using the Ballerina OpenAPI tool. ```ballerina public type CommonClientConfiguration record {| - boolean laxDataBinding = false; - // Other fields + boolean laxDataBinding = false; + // Other fields |}; ``` @@ -92,7 +87,7 @@ We can configure the relaxed data binding in the service level by improving the ```ballerina public type HttpServiceConfig record {| - boolean laxDataBinding = false; - // Other fields + boolean laxDataBinding = false; + // Other fields |}; ``` From a74e4eb3be5ffe2bb066609278b03673469f5536 Mon Sep 17 00:00:00 2001 From: SachinAkash01 Date: Tue, 17 Dec 2024 10:38:42 +0530 Subject: [PATCH 6/6] Refactor indentations --- docs/proposals/introducing_relaxed_data_binding.md | 4 ---- 1 file changed, 4 deletions(-) diff --git a/docs/proposals/introducing_relaxed_data_binding.md b/docs/proposals/introducing_relaxed_data_binding.md index f5a5f4da19..855af1f4bf 100644 --- a/docs/proposals/introducing_relaxed_data_binding.md +++ b/docs/proposals/introducing_relaxed_data_binding.md @@ -79,8 +79,6 @@ The approach involves adding a new field, `laxDataBinding`, to the `CommonClient public type CommonClientConfiguration record {| boolean laxDataBinding = false; // Other fields - boolean laxDataBinding = false; - // Other fields |}; ``` @@ -91,7 +89,5 @@ We can configure the relaxed data binding in the service level by improving the public type HttpServiceConfig record {| boolean laxDataBinding = false; // Other fields - boolean laxDataBinding = false; - // Other fields |}; ```