Skip to content

Commit

Permalink
#134: Added validation for duplicate destinationTable entries (#138)
Browse files Browse the repository at this point in the history
  • Loading branch information
kaklakariada authored Oct 25, 2023
1 parent 830195a commit fa8d574
Show file tree
Hide file tree
Showing 11 changed files with 156 additions and 138 deletions.
56 changes: 24 additions & 32 deletions dependencies.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions doc/changes/changelog.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

40 changes: 40 additions & 0 deletions doc/changes/changes_2.8.2.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# S3 Document Files Virtual Schema 2.8.2, released 2023-10-25

Code name: Validate EDML for duplicate table names

## Summary

This release upgrades to [virtual-schema-common-document-files 7.3.6](https://github.com/exasol/virtual-schema-common-document-files/releases/tag/7.3.6). This new version validates that the given EDML mapping uses unique values for each `destinationTable` entry as duplicate values led to unexpected behavior.

## Features

* #134: Added validation for duplicate `destinationTable` entries

## Dependency Updates

### Virtual Schema for Document Data in Files on AWS S3

#### Compile Dependency Updates

* Updated `com.exasol:virtual-schema-common-document-files:7.3.4` to `7.3.6`
* Updated `software.amazon.awssdk:s3:2.20.156` to `2.21.6`

#### Test Dependency Updates

* Updated `com.amazonaws:aws-java-sdk-s3:1.12.559` to `1.12.572`
* Updated `com.exasol:extension-manager-integration-test-java:0.5.1` to `0.5.3`
* Updated `com.exasol:virtual-schema-common-document-files:7.3.4` to `7.3.6`
* Updated `com.fasterxml.jackson.core:jackson-databind:2.15.2` to `2.15.3`
* Updated `org.jacoco:org.jacoco.agent:0.8.10` to `0.8.11`
* Updated `org.mockito:mockito-core:5.5.0` to `5.6.0`
* Updated `org.testcontainers:junit-jupiter:1.19.0` to `1.19.1`
* Updated `org.testcontainers:localstack:1.19.0` to `1.19.1`

#### Plugin Dependency Updates

* Updated `com.exasol:error-code-crawler-maven-plugin:1.3.0` to `1.3.1`
* Updated `com.exasol:project-keeper-maven-plugin:2.9.12` to `2.9.14`
* Updated `org.apache.maven.plugins:maven-enforcer-plugin:3.4.0` to `3.4.1`
* Updated `org.codehaus.mojo:versions-maven-plugin:2.16.0` to `2.16.1`
* Updated `org.jacoco:jacoco-maven-plugin:0.8.10` to `0.8.11`
* Updated `org.sonarsource.scanner.maven:sonar-maven-plugin:3.9.1.2184` to `3.10.0.2594`
6 changes: 3 additions & 3 deletions doc/hands_on/hands_on.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ For the document Virtual Schemas, and by that also the S3 Virtual Schema, this i
To install the Virtual Schema adapter, download its latest jar from the [releases](https://github.com/exasol/s3-document-files-virtual-schema/releases) and upload to BucketFS:

``` shell script
curl -I -X PUT -T document-files-virtual-schema-dist-7.3.4-s3-2.8.1.jar http://w:writepw@<YOUR_DB_IP>:2580/default/
curl -I -X PUT -T document-files-virtual-schema-dist-7.3.6-s3-2.8.2.jar http://w:writepw@<YOUR_DB_IP>:2580/default/
```

(If you have never used BucketFS, you can check out [its documentation](https://docs.exasol.com/database_concepts/bucketfs/bucketfs.htm))
Expand All @@ -75,7 +75,7 @@ CREATE SCHEMA ADAPTER;
--/
CREATE OR REPLACE JAVA ADAPTER SCRIPT ADAPTER.S3_FILES_ADAPTER AS
%scriptclass com.exasol.adapter.RequestDispatcher;
%jar /buckets/bfsdefault/default/document-files-virtual-schema-dist-7.3.4-s3-2.8.1.jar;
%jar /buckets/bfsdefault/default/document-files-virtual-schema-dist-7.3.6-s3-2.8.2.jar;
/

--/
Expand All @@ -85,7 +85,7 @@ CREATE OR REPLACE JAVA SET SCRIPT ADAPTER.IMPORT_FROM_S3_DOCUMENT_FILES(
CONNECTION_NAME VARCHAR(500))
EMITS(...) AS
%scriptclass com.exasol.adapter.document.UdfEntryPoint;
%jar /buckets/bfsdefault/default/document-files-virtual-schema-dist-7.3.4-s3-2.8.1.jar;
%jar /buckets/bfsdefault/default/document-files-virtual-schema-dist-7.3.6-s3-2.8.2.jar;
/
```

Expand Down
6 changes: 3 additions & 3 deletions doc/hands_on/hands_on_parquet.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ For the document Virtual Schemas, and by that also the S3 Virtual Schema, this i
To install the Virtual Schema adapter, download its latest jar from the [releases](https://github.com/exasol/s3-document-files-virtual-schema/releases) and upload to BucketFS:

``` shell script
curl -I -X PUT -T document-files-virtual-schema-dist-7.3.4-s3-2.8.1.jar http://w:writepw@<YOUR_DB_IP>:2580/default/
curl -I -X PUT -T document-files-virtual-schema-dist-7.3.6-s3-2.8.2.jar http://w:writepw@<YOUR_DB_IP>:2580/default/
```

(If you have never used BucketFS, you can check out [its documentation](https://docs.exasol.com/database_concepts/bucketfs/bucketfs.htm))
Expand All @@ -59,7 +59,7 @@ CREATE SCHEMA ADAPTER;

CREATE OR REPLACE JAVA ADAPTER SCRIPT ADAPTER.S3_FILES_ADAPTER AS
%scriptclass com.exasol.adapter.RequestDispatcher;
%jar /buckets/bfsdefault/default/document-files-virtual-schema-dist-7.3.4-s3-2.8.1.jar;
%jar /buckets/bfsdefault/default/document-files-virtual-schema-dist-7.3.6-s3-2.8.2.jar;
/

CREATE OR REPLACE JAVA SET SCRIPT ADAPTER.IMPORT_FROM_S3_DOCUMENT_FILES(
Expand All @@ -68,7 +68,7 @@ CREATE OR REPLACE JAVA SET SCRIPT ADAPTER.IMPORT_FROM_S3_DOCUMENT_FILES(
CONNECTION_NAME VARCHAR(500))
EMITS(...) AS
%scriptclass com.exasol.adapter.document.UdfEntryPoint;
%jar /buckets/bfsdefault/default/document-files-virtual-schema-dist-7.3.4-s3-2.8.1.jar;
%jar /buckets/bfsdefault/default/document-files-virtual-schema-dist-7.3.6-s3-2.8.2.jar;
/
```

Expand Down
6 changes: 3 additions & 3 deletions doc/user_guide/user_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Next create the Adapter Script:
```sql
CREATE OR REPLACE JAVA ADAPTER SCRIPT ADAPTER.S3_FILES_ADAPTER AS
%scriptclass com.exasol.adapter.RequestDispatcher;
%jar /buckets/bfsdefault/default/document-files-virtual-schema-dist-7.3.4-s3-2.8.1.jar;
%jar /buckets/bfsdefault/default/document-files-virtual-schema-dist-7.3.6-s3-2.8.2.jar;
/
```

Expand All @@ -30,7 +30,7 @@ CREATE OR REPLACE JAVA SET SCRIPT ADAPTER.IMPORT_FROM_S3_DOCUMENT_FILES(
CONNECTION_NAME VARCHAR(500))
EMITS(...) AS
%scriptclass com.exasol.adapter.document.UdfEntryPoint;
%jar /buckets/bfsdefault/default/document-files-virtual-schema-dist-7.3.4-s3-2.8.1.jar;
%jar /buckets/bfsdefault/default/document-files-virtual-schema-dist-7.3.6-s3-2.8.2.jar;
/
```

Expand Down Expand Up @@ -126,7 +126,7 @@ In DbVisualizer use exactly this command:
--/
CREATE OR REPLACE JAVA ADAPTER SCRIPT ADAPTER.S3_FILES_ADAPTER AS
%scriptclass com.exasol.adapter.RequestDispatcher;
%jar /buckets/bfsdefault/default/vs/document-files-virtual-schema-dist-7.3.4-s3-2.8.1.jar;
%jar /buckets/bfsdefault/default/vs/document-files-virtual-schema-dist-7.3.6-s3-2.8.2.jar;
/
```

Expand Down
16 changes: 8 additions & 8 deletions pk_generated_parent.pom

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 10 additions & 10 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,14 @@
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<artifactId>s3-document-files-virtual-schema</artifactId>
<version>2.8.1</version>
<version>2.8.2</version>
<name>Virtual Schema for document data in files on AWS S3</name>
<description>Adapter for document data access from files from AWS S3.</description>
<url>https://github.com/exasol/s3-document-files-virtual-schema/</url>
<properties>
<junit.version>5.10.0</junit.version>
<org.testcontainers.version>1.19.0</org.testcontainers.version>
<vs-common-document-files.version>7.3.4</vs-common-document-files.version>
<org.testcontainers.version>1.19.1</org.testcontainers.version>
<vs-common-document-files.version>7.3.6</vs-common-document-files.version>
<!-- Additional sonar configuration for analyzing TypeScript extension -->
<sonar.sources>src/main/,extension/src/</sonar.sources>
<sonar.exclusions>extension/src/*.test.ts</sonar.exclusions>
Expand Down Expand Up @@ -39,7 +39,7 @@
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>bom</artifactId>
<version>2.20.156</version>
<version>2.21.6</version>
<type>pom</type>
<scope>import</scope>
</dependency>
Expand Down Expand Up @@ -110,7 +110,7 @@
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<version>5.5.0</version>
<version>5.6.0</version>
<scope>test</scope>
</dependency>
<!--Integration test dependencies-->
Expand All @@ -136,7 +136,7 @@
<!-- Required by org.testcontainer:localstack -->
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-s3</artifactId>
<version>1.12.559</version>
<version>1.12.572</version>
<scope>test</scope>
</dependency>
<dependency>
Expand Down Expand Up @@ -190,14 +190,14 @@
<dependency>
<groupId>com.exasol</groupId>
<artifactId>extension-manager-integration-test-java</artifactId>
<version>0.5.1</version>
<version>0.5.3</version>
<scope>test</scope>
</dependency>
<dependency>
<!-- Required for integration testing the extension. This is required by extension-manager-integration-test-java but seems to be overwritten by another dependency. -->
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.15.2</version>
<version>2.15.3</version>
<scope>test</scope>
</dependency>
</dependencies>
Expand Down Expand Up @@ -232,7 +232,7 @@
<plugin>
<groupId>com.exasol</groupId>
<artifactId>project-keeper-maven-plugin</artifactId>
<version>2.9.12</version>
<version>2.9.14</version>
<executions>
<execution>
<goals>
Expand Down Expand Up @@ -351,7 +351,7 @@
<parent>
<artifactId>s3-document-files-virtual-schema-generated-parent</artifactId>
<groupId>com.exasol</groupId>
<version>2.8.1</version>
<version>2.8.2</version>
<relativePath>pk_generated_parent.pom</relativePath>
</parent>
</project>
Loading

0 comments on commit fa8d574

Please sign in to comment.