Skip to content

Commit

Permalink
[SPARK-50690][SQL] Fix discrepancy in DESCRIBE TABLE view query outpu…
Browse files Browse the repository at this point in the history 
…t columns quoting

### What changes were proposed in this pull request?

Previously, the `DESCRIBE TABLE` SQL command displayed column lists such as bucket and sort columns as so:

```
Bucket Columns          [`a`]
Sort Columns            [`b`]
```

However view query output columns were not quoted:

```
View Query Output Columns      [c1]
```

This PR fixes the discrepancy by quoting the view query output columns with ``:

```
View Query Output Columns      [`c1`]
```

### Why are the changes needed?

The change ensures cohesive `DESCRIBE TABLE` output, which can improve parsing and readability.

### Does this PR introduce _any_ user-facing change?

Yes, this modifies the view query output columns portion of the `DESCRIBE TABLE` output.

### How was this patch tested?

Existing SQL tests, including golden file output

### Was this patch authored or co-authored using generative AI tooling?

No

Closes #49321 from asl3/asl3/describe-viewqueryoutputcolumns.

Authored-by: Amanda Liu <amanda.liu@databricks.com>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
  • Loading branch information
@asl3 @HyukjinKwon
asl3 authored and HyukjinKwon committed Dec 28, 2024
1 parent 51b011f commit c1e51f2
Show file tree
Hide file tree
Showing 12 changed files with 94 additions and 93 deletions.
3 changes: 2 additions & 1 deletion sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala
View file
Original file line number Diff line number Diff line change
Expand Up @@ -559,7 +559,8 @@ case class CatalogTable(
map.put("View Catalog and Namespace", viewCatalogAndNamespace.quoted)
}
if (viewQueryColumnNames.nonEmpty) {
map.put("View Query Output Columns", viewQueryColumnNames.mkString("[", ", ", "]"))
map.put("View Query Output Columns",
viewQueryColumnNames.map(quoteIdentifier).mkString("[", ", ", "]"))
}
}

Expand Down
8 changes: 4 additions & 4 deletions sql/core/src/test/resources/sql-tests/results/charvarchar.sql.out
View file
Original file line number Diff line number Diff line change
Expand Up @@ -195,7 +195,7 @@ View Text select * from char_tbl
View Original Text select * from char_tbl
View Schema Mode COMPENSATION
View Catalog and Namespace spark_catalog.default
View Query Output Columns [c, v]
View Query Output Columns [`c`, `v`]


-- !query
Expand Down Expand Up @@ -366,7 +366,7 @@ View Text select * from char_tbl2
View Original Text select * from char_tbl2
View Schema Mode COMPENSATION
View Catalog and Namespace spark_catalog.default
View Query Output Columns [c, v]
View Query Output Columns [`c`, `v`]


-- !query
Expand Down Expand Up @@ -427,7 +427,7 @@ View Text select * from char_tbl2
View Original Text select * from char_tbl2
View Schema Mode COMPENSATION
View Catalog and Namespace spark_catalog.default
View Query Output Columns [c, v]
View Query Output Columns [`c`, `v`]
Table Properties [yes=no]


Expand Down Expand Up @@ -488,7 +488,7 @@ View Text select * from char_tbl2
View Original Text select * from char_tbl2
View Schema Mode COMPENSATION
View Catalog and Namespace spark_catalog.default
View Query Output Columns [c, v]
View Query Output Columns [`c`, `v`]


-- !query
Expand Down
4 changes: 2 additions & 2 deletions sql/core/src/test/resources/sql-tests/results/describe.sql.out
View file
Original file line number Diff line number Diff line change
Expand Up @@ -538,7 +538,7 @@ View Text SELECT * FROM t
View Original Text SELECT * FROM t
View Schema Mode COMPENSATION
View Catalog and Namespace spark_catalog.default
View Query Output Columns [a, b, c, d]
View Query Output Columns [`a`, `b`, `c`, `d`]


-- !query
Expand All @@ -563,7 +563,7 @@ View Text SELECT * FROM t
View Original Text SELECT * FROM t
View Schema Mode COMPENSATION
View Catalog and Namespace spark_catalog.default
View Query Output Columns [a, b, c, d]
View Query Output Columns [`a`, `b`, `c`, `d`]


-- !query
Expand Down
28 changes: 14 additions & 14 deletions sql/core/src/test/resources/sql-tests/results/postgreSQL/create_view.sql.out
View file
Original file line number Diff line number Diff line change
Expand Up @@ -269,7 +269,7 @@ View Text SELECT * FROM base_table
View Original Text SELECT * FROM base_table
View Schema Mode COMPENSATION
View Catalog and Namespace spark_catalog.temp_view_test
View Query Output Columns [a, id]
View Query Output Columns [`a`, `id`]


-- !query
Expand Down Expand Up @@ -335,7 +335,7 @@ View Text SELECT * FROM base_table
View Original Text SELECT * FROM base_table
View Schema Mode COMPENSATION
View Catalog and Namespace spark_catalog.temp_view_test
View Query Output Columns [a, id]
View Query Output Columns [`a`, `id`]


-- !query
Expand Down Expand Up @@ -391,7 +391,7 @@ View Original Text SELECT t1.a AS t1_a, t2.a AS t2_a
WHERE t1.id = t2.id
View Schema Mode COMPENSATION
View Catalog and Namespace spark_catalog.temp_view_test
View Query Output Columns [t1_a, t2_a]
View Query Output Columns [`t1_a`, `t2_a`]


-- !query
Expand Down Expand Up @@ -464,7 +464,7 @@ View Text SELECT * FROM base_table WHERE id IN (SELECT id FROM base_t
View Original Text SELECT * FROM base_table WHERE id IN (SELECT id FROM base_table2)
View Schema Mode COMPENSATION
View Catalog and Namespace spark_catalog.temp_view_test
View Query Output Columns [a, id]
View Query Output Columns [`a`, `id`]


-- !query
Expand Down Expand Up @@ -495,7 +495,7 @@ View Text SELECT t1.id, t2.a FROM base_table t1, (SELECT * FROM base_
View Original Text SELECT t1.id, t2.a FROM base_table t1, (SELECT * FROM base_table2) t2
View Schema Mode COMPENSATION
View Catalog and Namespace spark_catalog.temp_view_test
View Query Output Columns [id, a]
View Query Output Columns [`id`, `a`]


-- !query
Expand Down Expand Up @@ -526,7 +526,7 @@ View Text SELECT * FROM base_table WHERE EXISTS (SELECT 1 FROM base_t
View Original Text SELECT * FROM base_table WHERE EXISTS (SELECT 1 FROM base_table2)
View Schema Mode COMPENSATION
View Catalog and Namespace spark_catalog.temp_view_test
View Query Output Columns [a, id]
View Query Output Columns [`a`, `id`]


-- !query
Expand Down Expand Up @@ -557,7 +557,7 @@ View Text SELECT * FROM base_table WHERE NOT EXISTS (SELECT 1 FROM ba
View Original Text SELECT * FROM base_table WHERE NOT EXISTS (SELECT 1 FROM base_table2)
View Schema Mode COMPENSATION
View Catalog and Namespace spark_catalog.temp_view_test
View Query Output Columns [a, id]
View Query Output Columns [`a`, `id`]


-- !query
Expand Down Expand Up @@ -588,7 +588,7 @@ View Text SELECT * FROM base_table WHERE EXISTS (SELECT 1)
View Original Text SELECT * FROM base_table WHERE EXISTS (SELECT 1)
View Schema Mode COMPENSATION
View Catalog and Namespace spark_catalog.temp_view_test
View Query Output Columns [a, id]
View Query Output Columns [`a`, `id`]


-- !query
Expand Down Expand Up @@ -800,7 +800,7 @@ View Text SELECT * FROM t1 CROSS JOIN t2
View Original Text SELECT * FROM t1 CROSS JOIN t2
View Schema Mode COMPENSATION
View Catalog and Namespace spark_catalog.testviewschm2
View Query Output Columns [num, name, num2, value]
View Query Output Columns [`num`, `name`, `num2`, `value`]


-- !query
Expand Down Expand Up @@ -851,7 +851,7 @@ View Text SELECT * FROM t1 INNER JOIN t2 ON t1.num = t2.num2
View Original Text SELECT * FROM t1 INNER JOIN t2 ON t1.num = t2.num2
View Schema Mode COMPENSATION
View Catalog and Namespace spark_catalog.testviewschm2
View Query Output Columns [num, name, num2, value]
View Query Output Columns [`num`, `name`, `num2`, `value`]


-- !query
Expand Down Expand Up @@ -902,7 +902,7 @@ View Text SELECT * FROM t1 LEFT JOIN t2 ON t1.num = t2.num2
View Original Text SELECT * FROM t1 LEFT JOIN t2 ON t1.num = t2.num2
View Schema Mode COMPENSATION
View Catalog and Namespace spark_catalog.testviewschm2
View Query Output Columns [num, name, num2, value]
View Query Output Columns [`num`, `name`, `num2`, `value`]


-- !query
Expand Down Expand Up @@ -953,7 +953,7 @@ View Text SELECT * FROM t1 LEFT JOIN t2 ON t1.num = t2.num2 AND t2.va
View Original Text SELECT * FROM t1 LEFT JOIN t2 ON t1.num = t2.num2 AND t2.value = 'xxx'
View Schema Mode COMPENSATION
View Catalog and Namespace spark_catalog.testviewschm2
View Query Output Columns [num, name, num2, value]
View Query Output Columns [`num`, `name`, `num2`, `value`]


-- !query
Expand Down Expand Up @@ -1074,7 +1074,7 @@ BETWEEN (SELECT d FROM tbl2 WHERE c = 1) AND (SELECT e FROM tbl3 WHERE f = 2)
AND EXISTS (SELECT g FROM tbl4 LEFT JOIN tbl3 ON tbl4.h = tbl3.f)
View Schema Mode COMPENSATION
View Catalog and Namespace spark_catalog.testviewschm2
View Query Output Columns [a, b]
View Query Output Columns [`a`, `b`]


-- !query
Expand Down Expand Up @@ -1114,7 +1114,7 @@ AND EXISTS (SELECT g FROM tbl4 LEFT JOIN tbl3 ON tbl4.h = tbl3.f)
AND NOT EXISTS (SELECT g FROM tbl4 LEFT JOIN tmptbl ON tbl4.h = tmptbl.j)
View Schema Mode COMPENSATION
View Catalog and Namespace spark_catalog.testviewschm2
View Query Output Columns [a, b]
View Query Output Columns [`a`, `b`]


-- !query
Expand Down
40 changes: 20 additions & 20 deletions sql/core/src/test/resources/sql-tests/results/view-schema-binding-config.sql.out
View file
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ Type VIEW
View Text SELECT 1
View Original Text SELECT 1
View Catalog and Namespace spark_catalog.default
View Query Output Columns [1]
View Query Output Columns [`1`]


-- !query
Expand All @@ -155,7 +155,7 @@ Type: VIEW
View Text: SELECT 1
View Original Text: SELECT 1
View Catalog and Namespace: spark_catalog.default
View Query Output Columns: [1]
View Query Output Columns: [`1`]
Schema: root
|-- 1: integer (nullable = false)

Expand Down Expand Up @@ -206,7 +206,7 @@ Created By [not included in comparison]
Type: VIEW
View Text: SELECT 1
View Catalog and Namespace: spark_catalog.default
View Query Output Columns: [1]
View Query Output Columns: [`1`]
Schema: root
|-- 1: integer (nullable = false)

Expand Down Expand Up @@ -269,7 +269,7 @@ Type VIEW
View Text SELECT * FROM t
View Original Text SELECT * FROM t
View Catalog and Namespace spark_catalog.default
View Query Output Columns [c1]
View Query Output Columns [`c1`]


-- !query
Expand Down Expand Up @@ -324,7 +324,7 @@ Type VIEW
View Text SELECT * FROM t
View Original Text SELECT * FROM t
View Catalog and Namespace spark_catalog.default
View Query Output Columns [c1]
View Query Output Columns [`c1`]


-- !query
Expand Down Expand Up @@ -402,7 +402,7 @@ View Text SELECT * FROM t
View Original Text SELECT * FROM t
View Schema Mode BINDING
View Catalog and Namespace spark_catalog.default
View Query Output Columns [c1]
View Query Output Columns [`c1`]


-- !query
Expand Down Expand Up @@ -477,7 +477,7 @@ View Text SELECT * FROM t
View Original Text SELECT * FROM t
View Schema Mode BINDING
View Catalog and Namespace spark_catalog.default
View Query Output Columns [c1]
View Query Output Columns [`c1`]


-- !query
Expand Down Expand Up @@ -550,7 +550,7 @@ View Text SELECT * FROM t
View Original Text SELECT * FROM t
View Schema Mode COMPENSATION
View Catalog and Namespace spark_catalog.default
View Query Output Columns [c1]
View Query Output Columns [`c1`]


-- !query
Expand Down Expand Up @@ -615,7 +615,7 @@ View Text SELECT * FROM t
View Original Text SELECT * FROM t
View Schema Mode COMPENSATION
View Catalog and Namespace spark_catalog.default
View Query Output Columns [c1]
View Query Output Columns [`c1`]


-- !query
Expand Down Expand Up @@ -680,7 +680,7 @@ View Text SELECT * FROM t
View Original Text SELECT * FROM t
View Schema Mode COMPENSATION
View Catalog and Namespace spark_catalog.default
View Query Output Columns [c1]
View Query Output Columns [`c1`]


-- !query
Expand Down Expand Up @@ -774,7 +774,7 @@ View Text SELECT * FROM t
View Original Text SELECT * FROM t
View Schema Mode COMPENSATION
View Catalog and Namespace spark_catalog.default
View Query Output Columns [c1]
View Query Output Columns [`c1`]


-- !query
Expand Down Expand Up @@ -837,7 +837,7 @@ View Text SELECT * FROM t
View Original Text SELECT * FROM t
View Schema Mode COMPENSATION
View Catalog and Namespace spark_catalog.default
View Query Output Columns [c1, c2]
View Query Output Columns [`c1`, `c2`]


-- !query
Expand Down Expand Up @@ -895,7 +895,7 @@ View Text SELECT * FROM t
View Original Text SELECT * FROM t
View Schema Mode COMPENSATION
View Catalog and Namespace spark_catalog.default
View Query Output Columns [c1, c2]
View Query Output Columns [`c1`, `c2`]


-- !query
Expand Down Expand Up @@ -953,7 +953,7 @@ View Text SELECT * FROM t
View Original Text SELECT * FROM t
View Schema Mode COMPENSATION
View Catalog and Namespace spark_catalog.default
View Query Output Columns [c1, c2]
View Query Output Columns [`c1`, `c2`]


-- !query
Expand Down Expand Up @@ -1007,7 +1007,7 @@ View Text SELECT 1
View Original Text SELECT 1
View Schema Mode BINDING
View Catalog and Namespace spark_catalog.default
View Query Output Columns [1]
View Query Output Columns [`1`]


-- !query
Expand All @@ -1026,7 +1026,7 @@ View Text: SELECT 1
View Original Text: SELECT 1
View Schema Mode: BINDING
View Catalog and Namespace: spark_catalog.default
View Query Output Columns: [1]
View Query Output Columns: [`1`]
Schema: root
|-- 1: integer (nullable = false)

Expand Down Expand Up @@ -1069,7 +1069,7 @@ View Text SELECT 1
View Original Text SELECT 1
View Schema Mode BINDING
View Catalog and Namespace spark_catalog.default
View Query Output Columns [1]
View Query Output Columns [`1`]


-- !query
Expand All @@ -1088,7 +1088,7 @@ View Text: SELECT 1
View Original Text: SELECT 1
View Schema Mode: BINDING
View Catalog and Namespace: spark_catalog.default
View Query Output Columns: [1]
View Query Output Columns: [`1`]
Schema: root
|-- 1: integer (nullable = false)

Expand Down Expand Up @@ -1165,7 +1165,7 @@ Type: VIEW
View Text: SELECT 1
View Schema Mode: BINDING
View Catalog and Namespace: spark_catalog.default
View Query Output Columns: [1]
View Query Output Columns: [`1`]
Schema: root
|-- 1: integer (nullable = false)

Expand Down Expand Up @@ -1199,7 +1199,7 @@ Type: VIEW
View Text: SELECT 1
View Schema Mode: BINDING
View Catalog and Namespace: spark_catalog.default
View Query Output Columns: [1]
View Query Output Columns: [`1`]
Schema: root
|-- 1: integer (nullable = false)

Expand Down
Loading

0 comments on commit c1e51f2

Please sign in to comment.