Skip to content

Minor feature release
Compare
Choose a tag to compare
@yruslan yruslan released this 04 Feb 10:05
· 374 commits to master since this release
v2.4.8
fae87b9
This commit was signed with the committer’s verified signature.
yruslan Ruslan Yushchenko
GPG key ID: 683FABE27EE16A62
Learn about vigilant mode.
  • Improved schema flattening method SparkUtils.flattenSchema() for dataframes that have arrays. Array size metadata is used to determine maximum array elements, making it much faster for dataframes produced from mainframe files.
  • #324 Allow removing of FILLERs from AST when parsing using 'parseSimple()'. The signature of the method has
    changed. The boolean arguments now reflect more clearly what they do.
  • #466 Added maxElements and minElements to Spark schema metadata for
    array fields created from fields with OCCURS. This allows knowing the maximum number of elements in arrays when flattening the schema.
Assets 2
Loading