Skip to content

Version 0.1.6

Compare
Choose a tag to compare
@menishmueli menishmueli released this 26 Mar 11:06
· 63 commits to main since this release

Apache Iceberg support:

  1. Better node naming
  2. Read metrics and reading small files alerts
  3. Write metrics and overwriting most of table alerts
    Write metrics require enabling an iceberg metric reporter, can be done for you by enabling spark.dataflint.iceberg.autoCatalogDiscovery to true, or setting the iceberg metric reporter manually for each catalog, for example:
    spark.sql.catalog.[catalog name].metrics-reporter-impl org.apache.spark.dataflint.iceberg.DataflintIcebergMetricsReporter