[spike] [Snowflake] Support complex types unit testing

[jtcohen6]

To get unit tests working on Snowflake at all, we need case-insensitive comparisons in a few places.

To get "complex data types" working on Snowflake in particular, I needed to:

• escape single quotes in strings
• hack extend snowflake__safe_cast to also do JSON-parsing for me, as a treat:

{% macro snowflake__safe_cast(field, type) %}
  {#-- Hack for now --#}
  {% if type|lower in ['array', 'variant'] %}
    cast(parse_json({{field}}) as {{type}})
  {% else %}
    try_cast({{field}} as {{type}})
  {% endif %}
{% endmacro %}

Then, this "just works":

-- models/my_upstream_model.sql
select
  ['a','b','c'] as tested_column

-- models/my_model.sql
select
    tested_column
from {{ ref('my_upstream_model')}}

unit_tests:
  - name: test_array
    model: my_model
    given:
      - input: ref('my_upstream_model')
        rows:
          - tested_column: "['a','b','c']"
    expect:
      rows:
        - tested_column: "['a','b','c']"
  - name: test_object
    model: my_model
    given:
      - input: ref('my_upstream_model')
        rows:
          - tested_column: "{'a': 'b', 'c': 123}"
    expect:
      rows:
        - tested_column: "{'a': 'b', 'c': 123}"
  - name: test_combined
    model: my_model
    given:
      - input: ref('my_upstream_model')
        rows:
          - tested_column: "[{'a': 'b', 'c': [1,2,3]}]"
    expect:
      rows:
        - tested_column: "[{'a': 'b', 'c': [1,2,3]}]"

[github-actions]

Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the contributing guide.

[jtcohen6]

If a column name is quoted, Snowflake will preserve its case. The alternative is to lowercase column names in the agate result before we access it here:

dbt-core/core/dbt/task/unit_test.py

Line 162 in e001991

lambda row: row["actual_or_expected"] == actual_or_expected

Otherwise, we get one of these:

22:45:30  Unhandled error while executing target/run/my_dbt_project/models/my_model.yml/models/my_model__test_my_model.sql
'actual_or_expected'
22:45:30  1 of 1 ERROR my_model__test_my_model ........................................... [ERROR in 24.16s]
22:45:30
22:45:30  Finished running 1 unit_test in 0 hours 0 minutes and 24.17 seconds (24.17s).
22:45:30
22:45:30  Completed with 1 error and 0 warnings:
22:45:30
22:45:30    'actual_or_expected'
22:45:30

[jtcohen6]

This is naively assuming that the user has provided their columns in all lowercase. We'd obviously want to lower on both sides of the comparison.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (8197fa7) 86.92% compared to head (e2be7ba) 86.87%.

Additional details and impacted files

@@                          Coverage Diff                           @@
##           support-complex-types-unit-testing    #9131      +/-   ##
======================================================================
- Coverage                               86.92%   86.87%   -0.05%     
======================================================================
  Files                                     181      181              
  Lines                                   27156    27156              
======================================================================
- Hits                                    23604    23592      -12     
- Misses                                   3552     3564      +12     

Flag	Coverage Δ
integration	83.81% <ø> (-0.10%)	⬇️
unit	64.72% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[jtcohen6]

So that it's:

select cast(parse_json('[\'a\',\'b\',\'c\']') as ARRAY) AS tested_column

Rather than:

select cast(parse_json('['a','b','c']') as ARRAY) AS tested_column

[jtcohen6]

Closing since @MichelleArk has pulled these commits into #9102