+ +For each object in Hackolade, we've defined a set of standard properties that appear in the properties pane.  But it is possible that your company wants to define and track additional properties for models, containers, entities, and attributes.  Hackolade let's you do just that, by leveraging our plugin architecture (used also to integrate our modeling engine with all kinds of NoSQL document databases.) + + + + +## 1) Download and enable plugin + +To enable the custom properties capability, you first need to download a plugin for each database target for which you wish to add properties.  To do so, go to Help > DB Target Plugin Manager + + + + + + +You may choose which plugin to install on your computer. + + + + + + +This will result in the plugin appearing in the Installed tab of the plugin manager. + + + + + + +## 2) Access plugin configuration files + +You are now ready to add custom properties via editing of configuration files.  The plugin configurations files can be found by going to Help > Show plugin directory: + + + + + + +For each custom properties plugin, you will find the a directory structure similar to this one: + + + +Notes: + +i) do NOT make any changes to the package.json file!  Only the LevelConfig.json files should be edited according to the specifications below. + +ii) it is advised to keep a backup of the files before making changes, so you can revert back in case of errors. + +iii) it is always necessary to restart the application after having saved changes before you can see these changes relected in the properties pane. + +iv) for field-level definitions, since field types have different property lists, it may be necessary to define custom properties for multiple field types. + +## 3) Levels + +As a reminder, terminology differs between NoSQL database: + +- container means: dbs in MongoDB, region in DynamoDB, bucket in Couchbase, collection in Cosmos DB + +- entity means: collection in MongoDB, table in DynamoDB, document kind in Couchbase, and document type in Cosmos DB + +- field means: field in MongoDB, Couchbase, and Cosmos DB. And attribute in DynamoDB + + + + +You need to edit the corresponding LevelConfig.json file to add custom properties. + + + + +## 4) Lower tabs + +For each level, the Hackolade properties pane may have one or more lower tab: + +- MongoDB model lower tab: + + + +- MongoDB dbs lower tab: + + + +- MongoDB collection lower tab: + + + +- MongoDB field lower tab: + + + + + + +If the level allows multiple tabs, you need to choose to which lower tab you want to add properties. + + + + +## 5) Property types + +The following controls are possible for user-defined properties: + + + +* simple text: one line of text +* text area: popup for multi-line text entry +* dropdown selection from a deined list of options +* numeric-only field +* checkbox: for true/false entry + + + + + + +## 6) Property definition + +Examples are provided in the comments section of each config file.  Here's an overview of the schema: + + + +Here's another view, consolidated: + + + +- propertyName: mandatory, this is the property label that will appear in the Property Pane + +- propertyKeyword: mandatory, this is the unique key for the property + +- shouldValidate: optional, boolean true/false to define whether to validate the regular expression of the text input [default: false] + +- propertyTooltip: optional, in the case of input types textarea and select, it is possible to display a tooltip  defined here + +- propertyType: mandatory, this is the control definition, with possible values: text, details, select (i.e. dropdown), checkbox + +- options: optional, this is the array of possible checkbox options + +- template: optional, this is needed in the case of propertyType = details, to define a popup multi-line text.  Possible value: textarea + +- valueType: optional, this is needed in to specify that a property is numberic only.  Possible values: numeric + + + + +## 7) Share customization with team members + +After making, saving and testing your changes, you should share them with everyone on your team to insure consistency. This is a 3-step process: + +- return to the plugin directory via Help > Show plugin directory, and zip up the whole plugin directory where you made your changes; + +- transfer this zip file to each team member using Hackolade; + +- on each team member's computer, start Hackolade, go to Help > DB target plugin manager, then click the button 'Install from zip file', and choose the zip file file. + + + + +For the changes to take effect on each computer, it is required to exit Hackolade and restart it. + + + + + + + + + + + + + diff --git a/central_pane/dtdAbbreviation.json b/central_pane/dtdAbbreviation.json new file mode 100644 index 0000000..19894de --- /dev/null +++ b/central_pane/dtdAbbreviation.json @@ -0,0 +1,9 @@ +{ + "object": "{...}", + "array": "[...]", + "string": "{ABC}", + "number": "{123}", + "bool": "{0/1}", + "bytes": "{BYTES}", + "null": "{null}" +} \ No newline at end of file diff --git a/forward_engineering/api.js b/forward_engineering/api.js new file mode 100644 index 0000000..8ba3293 --- /dev/null +++ b/forward_engineering/api.js @@ -0,0 +1,180 @@ +'use strict' + +const ADDITIONAL_PROPS = ['name', 'doc', 'order', 'aliases', 'symbols', 'namespace']; + +module.exports = { + generateScript(data, logger, cb) { + try { + const name = getRecordName(data); + let avroSchema = { name }; + let jsonSchema = JSON.parse(data.jsonSchema); + + handleRecursiveSchema(jsonSchema, avroSchema); + + if (data.containerData) { + avroSchema.namespace = data.containerData.name; + } + avroSchema.type = 'record'; + avroSchema = reorderAvroSchema(avroSchema); + avroSchema = JSON.stringify(avroSchema, null, 4); + return cb(null, avroSchema); + } catch(err) { + logger.log('error', { message: err.message, stack: err.stack }, 'Avro Forward-Engineering Error'); + setTimeout(() => { + callback({ message: err.message, stack: err.stack }); + }, 150); + } + } +}; + +const getRecordName = (data) => { + return data.entityData.name || data.entityData.collectionName; +}; + +const reorderAvroSchema = (avroSchema) => { + const schemaFields = avroSchema.fields; + delete avroSchema.fields; + return Object.assign({}, avroSchema, { + fields: schemaFields + }); +}; + +const handleRecursiveSchema = (schema, avroSchema, parentSchema = {}) => { + if (schema.oneOf) { + handleOneOf(schema, avroSchema); + } + + for (let prop in schema) { + switch(prop) { + case 'type': + handleType(schema, prop, avroSchema, parentSchema); + break; + case 'properties': + handleFields(schema, prop, avroSchema); + break; + case 'items': + handleItems(schema, prop, avroSchema); + break; + default: + handleOtherProps(schema, prop, avroSchema); + } + } + + return; +}; + +const handleType = (schema, prop, avroSchema, parentSchema) => { + if (Array.isArray(schema[prop])) { + avroSchema = handleMultiple(avroSchema, schema, prop); + } else { + avroSchema = getFieldWithConvertedType(avroSchema, schema, schema[prop]); + } +}; + +const handleMultiple = (avroSchema, schema, prop) => { + avroSchema[prop] = schema[prop].map(type => { + const field = getFieldWithConvertedType({}, schema, type); + return field.type; + }); + return avroSchema; +}; + +const getFieldWithConvertedType = (schema, field, type) => { + switch(type) { + case 'string': + case 'bytes': + case 'boolean': + case 'null': + case 'record': + case 'array': + case 'enum': + case 'fixed': + return Object.assign(schema, { type }); + case 'number': + return Object.assign(schema, { type: field.mode || 'int' }); + case 'map': + return Object.assign(schema, { + type, + values: getValues(type, field.subtype) + }); + default: + return Object.assign(schema, { type: 'string' }); + } +}; + +const getValues = (type, subtype) => { + const regex = new RegExp('\\' + type + '<(.*?)\>'); + return subtype.match(regex)[1] || 'string'; +}; + +const handleFields = (schema, prop, avroSchema) => { + avroSchema.fields = Object.keys(schema[prop]).map(key => { + let field = schema[prop][key]; + let avroField = Object.assign({}, { name: key }); + handleRecursiveSchema(field, avroField, schema); + return avroField; + }); +}; + +const handleItems = (schema, prop, avroSchema) => { + if (!Array.isArray(schema[prop])) { + schema[prop] = [schema[prop]]; + } + + avroSchema[prop] = {}; + handleRecursiveSchema(schema[prop][0], avroSchema[prop], schema); +}; + +const handleOneOf = (schema, avroSchema) => { + let allSubSchemaFields = []; + schema.oneOf.forEach(subSchema => { + allSubSchemaFields = allSubSchemaFields.concat(Object.keys(subSchema.properties).map(item => { + return Object.assign({ + name: item + }, subSchema.properties[item]); + })); + }); + const sharedFieldNames = uniqBy(allSubSchemaFields, 'name'); + const commonFields = allSubSchemaFields.filter(item => sharedFieldNames.includes(item.name)); + + let multipleFieldsHash = {}; + commonFields.forEach(field => { + if (!multipleFieldsHash[field.name]) { + multipleFieldsHash[field.name] = { + name: field.name, + type: [] + }; + } + let multipleField = multipleFieldsHash[field.name]; + let fieldTypes = (Array.isArray(field.type) ? field.type : [field.type]); + multipleField.type = multipleField.type.concat(fieldTypes); + + if (field.properties) { + multipleField.properties = Object.assign((multipleField.properties || {}), field.properties); + } + + if (field.items) { + multipleField.items = Object.assign((multipleField.items || {}), field.items); + } + + [...ADDITIONAL_PROPS, 'mode', 'subtype'].forEach(prop => { + if (field[prop]) { + multipleField[prop] = field[prop]; + } + }); + }); + + schema.properties = Object.assign((schema.properties || {}), multipleFieldsHash); +}; + +const uniqBy = (arr, prop) => { + return arr.map(function(e) { return e[prop]; }).filter(function(e,i,a){ + return i === a.indexOf(e); + }); +}; + +const handleOtherProps = (schema, prop, avroSchema) => { + if (ADDITIONAL_PROPS.includes(prop)) { + avroSchema[prop] = schema[prop]; + } +}; \ No newline at end of file diff --git a/forward_engineering/config.json b/forward_engineering/config.json new file mode 100644 index 0000000..f18b82e --- /dev/null +++ b/forward_engineering/config.json @@ -0,0 +1,5 @@ +{ + "extension": "avsc", + "filterName": "Avro schema", + "namePrefix": "Avro Schema" +} diff --git a/forward_engineering/package.json b/forward_engineering/package.json new file mode 100644 index 0000000..23b5f08 --- /dev/null +++ b/forward_engineering/package.json @@ -0,0 +1,7 @@ +{ + "name": "Avro", + "version": "1.0.0", + "description": "", + "author": "Hackolade", + "installed": false +} \ No newline at end of file diff --git a/jsonSchemaProperties.json b/jsonSchemaProperties.json new file mode 100644 index 0000000..ea6a87b --- /dev/null +++ b/jsonSchemaProperties.json @@ -0,0 +1,4 @@ +{ + "unneededFieldProps": ["collectionName", "name", "users", "indexes", "collectionUsers", "additionalPropertieserties"], + "removeIfPropsNegative": ["partitionKey", "sortKey"] +} \ No newline at end of file diff --git a/lib/MongoDBcollectionlowertab.png b/lib/MongoDBcollectionlowertab.png new file mode 100644 index 0000000..ffaea78 Binary files /dev/null and b/lib/MongoDBcollectionlowertab.png differ diff --git a/lib/MongoDBdbslowertab.png b/lib/MongoDBdbslowertab.png new file mode 100644 index 0000000..315a9ad Binary files /dev/null and b/lib/MongoDBdbslowertab.png differ diff --git a/lib/MongoDBfieldlowertab.png b/lib/MongoDBfieldlowertab.png new file mode 100644 index 0000000..142c860 Binary files /dev/null and b/lib/MongoDBfieldlowertab.png differ diff --git a/lib/MongoDBmodellowertab.png b/lib/MongoDBmodellowertab.png new file mode 100644 index 0000000..1c9eac1 Binary files /dev/null and b/lib/MongoDBmodellowertab.png differ diff --git a/lib/Plugin-CustomPropdirectorystructure.png b/lib/Plugin-CustomPropdirectorystructure.png new file mode 100644 index 0000000..ce7e09a Binary files /dev/null and b/lib/Plugin-CustomPropdirectorystructure.png differ diff --git a/lib/Plugin-Managerinstalledcustomprops.png b/lib/Plugin-Managerinstalledcustomprops.png new file mode 100644 index 0000000..10a2264 Binary files /dev/null and b/lib/Plugin-Managerinstalledcustomprops.png differ diff --git a/lib/Plugin-Showplugindirectory.png b/lib/Plugin-Showplugindirectory.png new file mode 100644 index 0000000..6677e97 Binary files /dev/null and b/lib/Plugin-Showplugindirectory.png differ diff --git a/lib/Plugin-custompropsconsolidatedschema.png b/lib/Plugin-custompropsconsolidatedschema.png new file mode 100644 index 0000000..f9b11cc Binary files /dev/null and b/lib/Plugin-custompropsconsolidatedschema.png differ diff --git a/lib/Plugin-manageravailablecustomprops.png b/lib/Plugin-manageravailablecustomprops.png new file mode 100644 index 0000000..701fd00 Binary files /dev/null and b/lib/Plugin-manageravailablecustomprops.png differ diff --git a/lib/Plugin-managermenu.png b/lib/Plugin-managermenu.png new file mode 100644 index 0000000..d6530c5 Binary files /dev/null and b/lib/Plugin-managermenu.png differ diff --git a/lib/Plugin-possiblepropertytypes.png b/lib/Plugin-possiblepropertytypes.png new file mode 100644 index 0000000..b5e2321 Binary files /dev/null and b/lib/Plugin-possiblepropertytypes.png differ diff --git a/lib/Plugin-propertyschema.png b/lib/Plugin-propertyschema.png new file mode 100644 index 0000000..acd412d Binary files /dev/null and b/lib/Plugin-propertyschema.png differ diff --git a/localization/en.json b/localization/en.json new file mode 100644 index 0000000..7ef0d85 --- /dev/null +++ b/localization/en.json @@ -0,0 +1,148 @@ +{ + "WELCOME_PAGE___REVERSE_ENGINEER_DESCRIPTION": "Create a Hackolade model from an existing DB instance", + "MAIN_MENU___ADD_BUCKET": "Add Namespace", + "MAIN_MENU___ADD_COLLECTION": "Add Record", + "MAIN_MENU___ADD_CHILD_COLLECTION": "Add Child Collection", + "MAIN_MENU___ADD_VIEW": "Add View", + "MAIN_MENU___ADD_RELATIONSHIP": "Add Relationship", + "MAIN_MENU___ADD_ATTRIBUTE": "Add Field", + "MAIN_MENU___INSERT_FIELD": "Insert Field", + "MAIN_MENU___APPEND_FIELD": "Append Field", + "MAIN_MENU___FORWARD_DB_COLLECTIONS": "Avro Schema", + "TOOLBAR___ADD_BUCKET": "Add namespace", + "TOOLBAR___ADD_COLLECTION": "Add record", + "TOOLBAR___ADD_CHILD_COLLECTION": "Add Child Collection", + "TOOLBAR___ADD_VIEW": "Add view", + "TOOLBAR___ADD_RELATIONSHIP": "Add Relationship", + "TOOLBAR___ADD_ATTRIBUTE": "Add Field", + "TOOLBAR___INSERT_FIELD": "Insert Field", + "TOOLBAR___APPEND_FIELD": "Append Field", + "TOOLBAR___TOGGLE_FIELD_DETAILS": "Toogle field details", + "TOOLBAR___SHOW_FOREIGN_MASTER": "Toggle foreign master", + "TOOLBAR___SHOW_MODEL_VIEW": "Toggle model views", + "TOOLBAR___DISTRIBUTE_ORTHOGONALLY": "Distribute records orthogonally", + "OBJECT___BROWSER_BUCKET": "Records", + "OBJECT___BROWSER_NOT_IN_BUCKET": "Undefined Record", + "OBJECT___BROWSER_COLLECTION": "Records", + "OBJECT___BROWSER_VIEWS": "Views", + "OBJECT___BROWSER_DEFINITIONS": "Definitions", + "OBJECT___BROWSER_FIELDS": "Fields", + "PROPERTIES_PANE___BUCKET_NAME": "Namespace", + "PROPERTIES_PANE___VIEW_NAME": "View name", + "PROPERTIES_PANE___COLLECTION_NAME": "Record", + "PROPERTIES_PANE___FOREIGN_COLLECTION": "Foreign record", + "PROPERTIES_PANE___FOREIGN_FIELD": "Foreign field", + "PROPERTIES_PANE___PARENT_COLLECTION": "Parent record", + "PROPERTIES_PANE___PARENT_FIELD": "Parent field", + "PROPERTIES_PANE___PARENT_CARDINALITY": "Parent cardinality", + "PROPERTIES_PANE___CHILD_COLLECTION": "Child record", + "PROPERTIES_PANE___CHILD_FIELD": "Child field", + "PROPERTIES_PANE___CHILD_CARDINALITY": "Child cardinality", + "PROPERTIES_PANE___PRIMARY_KEY": "Primary key", + "PROPERTIES_PANE___COLLECTION_BUCKET": "Namespace", + "PROPERTIES_PANE___VIEW_ON": "View on", + "PROPERTIES_PANE___PIPELINE": "Pipeline", + "PROPERTIES_PANE___COLLATION": "Collation", + "PROPERTIES_PANE___LOCALE": "Locale", + "PROPERTIES_PANE___VARIANT": "Variant", + "PROPERTIES_PANE___STRENGTH": "Strength", + "PROPERTIES_PANE___CASE_LEVEL": "Case level", + "PROPERTIES_PANE___CASE_FIRST": "Case first", + "PROPERTIES_PANE___NUMERIC_ORDERING": "Numeric ordering", + "PROPERTIES_PANE___ALTERNATE": "Alternate", + "PROPERTIES_PANE___MAX_VARIABLE": "Max variable", + "PROPERTIES_PANE___BACKWARDS": "Backwards", + "PROPERTIES_PANE___NORMALIZATION": "Normalization", + "CENTRAL_PANE___TAB_MONGODB_VIEW_SCRIPT": "Create View Script", + "CONTEXT_MENU___ADD_BUCKET": "Add namespace", + "CONTEXT_MENU___ADD_COLLECTION": "Add record", + "CONTEXT_MENU___ADD_CHILD_COLLECTION": "Add child record", + "CONTEXT_MENU___ADD_VIEW": "Add view", + "CONTEXT_MENU___ALIGN_COLLECTIONS": "Align records", + "CONTEXT_MENU___ADD_ATTRIBUTE": "Add field", + "CONTEXT_MENU___INSERT_ATTRIBUTE": "Insert field", + "CONTEXT_MENU___APPEND_ATTRIBUTE": "Append field", + "CONTEXT_MENU___OPEN_COLLECTION_IN_NEW_TAB": "Open record in new tab", + "CONTEXT_MENU___FIELD": "Field", + "CONTEXT_MENU___PATTERN_FIELD": "Pattern Field", + "CONTEXT_MENU___ARRAY_ITEM": "Array Item", + "MODAL_WINDOW___COLLECTION": "record:", + "MODAL_WINDOW___EMPTY_MODEL_MESSAGE": "The Model does not contain any namespaces.", + "MODAL_WINDOW___FIELD_INFERENCE": "Field Inference", + "MODAL_WINDOW___KEEP_FIELD_ORDER": "Keep field order", + "MODAL_WINDOW___CANNOT_CONNECT_TO_DB": "Cannot connect to Avro", + "MODAL_WINDOW___SUCCESSFULLY_CONNECT_TO_DB": "Successfully connected to Avro", + "MODAL_WINDOW___UNABLE_CONNECT_TO_DB": "Unable to connect to Avro", + "MODAL_WINDOW___DB_ENTITIES_SELECTION_TITLE": "Type selection", + "MODAL_WINDOW___RECORDS_MAX": "Documents max", + "MODAL_WINDOW___SUBDOCUMENT_IN_CHILD": "Sub-document in child", + "MODAL_WINDOW___ARRAY_IN_PARENT": "Array in parent", + "MODAL_WINDOW___INCLUDE_EMPTY_COLLECTION": "Include empty namespaces", + "MODAL_WINDOW___CREATE_COLLECTION": "Create record", + "MODAL_WINDOW___CREATE_BUCKET": "Create namespace", + "MODAL_WINDOW___ALL_COLLECTIONS": "and all nested records", + "MODAL_WINDOW___CONNENTION_ERROR": "The Avro instance you are connected to does not contain any records.", + "MODAL_WINDOW___CONTAIN_BUCKETS": "namespaces", + "MODAL_WINDOW___CONTAIN_COLLECTIONS": "records", + "MODAL_WINDOW___CONTAIN_BUCKET": "namespaces", + "MODAL_WINDOW___CONTAIN_COLLECTION": "record", + "MODAL_WINDOW___DB_CONNECTION_PROCESS": "Avro Reverse-Engineering Process", + "PROGRESS_BAR___DATABASE": "Namespace", + "PROGRESS_BAR___COLLECTION": "Record", + "PROGRESS_BAR___PROCESS": "Process", + "DOCUMENTATION___PARENT_COLLECTION": "Parent Record", + "DOCUMENTATION___PARENT_FIELD": "Parent field", + "DOCUMENTATION___PARENT_CARDINALITY": "Parent Cardinality", + "DOCUMENTATION___CHILD_COLLECTION": "Child Record", + "DOCUMENTATION___CHILD_FIELD": "Child field", + "DOCUMENTATION___CHILD_CARDINALITY": "Child Cardinality", + "DOCUMENTATION___FIELD": "Field", + "DOCUMENTATION___COLLECTIONS": "Records", + "DOCUMENTATION___COLLECTION": "Record", + "DOCUMENTATION___BUCKETS": "Namespaces", + "DOCUMENTATION___BUCKET": "Namespace", + "DOCUMENTATION___FIELDS": "Fields", + "DOCUMENTATION___CHILD_FIELDS": "Child field(s)", + "DOCUMENTATION___PHYSICAL_MODEL": "Avro Physical Model", + "DOCUMENTATION___VIEWS": "Views", + "DOCUMENTATION___VIEW": "Views", + "TOOLTIPS___FOREIGN_COLLECTION": "foreign records", + "TOOLTIPS___COLLECTION_BUCKET": "Namespaces", + "TOOLTIPS___FOREIGN_FIELD": "foreign field", + "TOOLTIPS___PARENT_COLLECTION": "parent record", + "TOOLTIPS___PARENT_FIELD": "parent field", + "TOOLTIPS___CHILD_COLLECTION": "child record", + "TOOLTIPS___CHILD_FIELD": "child field", + "TOOLTIPS___VIEW_ON": "view on", + "TOOLTIPS___PIPELINE": "pipeline", + "TOOLTIPS___COLLATION": "collation", + "TOOLTIPS___LOCALE": "locale", + "TOOLTIPS___VARIANT": "variant", + "TOOLTIPS___STRENGTH": "strength", + "TOOLTIPS___CASE_LEVEL": "case level", + "TOOLTIPS___CASE_FIRST": "case first", + "TOOLTIPS___NUMERIC_ORDERING": "numeric ordering", + "TOOLTIPS___ALTERNATE": "alternate", + "TOOLTIPS___MAX_VARIABLE": "max variable", + "TOOLTIPS___BACKWARDS": "Backwards", + "TOOLTIPS___NORMALIZATION": "Normalization", + "NEW___MODEL_NAME": "New model", + "NEW___BUCKET_NAME": "New namespace", + "NEW___COLLECTION_NAME": "New record", + "NEW___FIELD_NAME": "New field", + "NEW___PATTERN_FIELD_NAME": "^New Pattern Field$", + "COLLECTION_SCHEMA_DEFINITION_NAME": "Type definitions", + "COLLECTION_SCHEMA_DEFINITION_TYPE": "document", + "MONGODB_SCRIPT_WARNING_MESSAGE": "This view is not associated to a type (viewOn property).", + "TYPE": {}, + "CENTRAL_PANE___TAB_MODEL_DEFINITIONS": "Model Definitions", + "CENTRAL_PANE___FE_SCRIPT": "Avro Schema", + "CONTEXT_MENU___ADD_MODEL_REFERENCE": "Model Definition", + "CONTEXT_MENU___GO_TO_DEFINITION": "Go to Model Definition", + "DOCUMENTATION___DB_DEFINITIONS": "Model Definitions", + "CONTEXT_MENU___CONVERT_TO_PATTERN_FIELD": "Convert to Pattern Field", + "CONTEXT_MENU___CONVERT_PATTERN_TO_REGULAR_FIELD": "Convert to Regular Field", + "MODAL_WINDOW___DB_CONNECTION_PROCESS": "Avro Reverse-Engineering Process", + "MODAL_WINDOW___DB_CONNECTIONS_LIST_TITLE": "Avro Connections", + "COLLECTION_SCHEMA_ROOT_TYPE": "record" +} \ No newline at end of file diff --git a/logo.jpg b/logo.jpg new file mode 100644 index 0000000..86e8e7f Binary files /dev/null and b/logo.jpg differ diff --git a/package.json b/package.json new file mode 100644 index 0000000..839aa10 --- /dev/null +++ b/package.json @@ -0,0 +1,23 @@ +{ + "name": "Avro", + "version": "0.1.0", + "versionDate": "2018-10-30", + "author": "hackolade", + "engines": { + "hackolade": "2.1.x", + "hackoladePlugin": "1.0.0" + }, + "contributes": { + "target": { + "applicationTarget": "Avro", + "title": "Apache Avro Schema", + "versions": [] + }, + "features": { + "enableForwardEngineering": true, + "disableReverseEngineering": false + } + }, + "description": "Hackolade plugin for Apache Avro Schema", + "disabled": false +} \ No newline at end of file diff --git a/properties_pane/container_level/containerLevelConfig.json b/properties_pane/container_level/containerLevelConfig.json new file mode 100644 index 0000000..d0c74d0 --- /dev/null +++ b/properties_pane/container_level/containerLevelConfig.json @@ -0,0 +1,106 @@ +/* +* Copyright © 2016-2017 by IntegrIT S.A. dba Hackolade. All rights reserved. +* +* The copyright to the computer software herein is the property of IntegrIT S.A. +* The software may be used and/or copied only with the written permission of +* IntegrIT S.A. or in accordance with the terms and conditions stipulated in +* the agreement/contract under which the software has been supplied. + + +In order to define custom properties for any object's properties pane, you may copy/paste from the following, +making sure that you maintain a proper JSON format. + + { + "propertyName": "Simple text", + "propertyKeyword": "simpletextProp", + "shouldValidate": false, + "propertyType": "text", + "sampleGen": "&containerName|&entityName|&random|" + }, + { + "propertyName": "Text area", + "propertyKeyword": "textareaProp", + "propertyValidate": false, + "propertyTooltip": "Popup for multi-line text entry", + "propertyType": "details", + "template": "textarea" + }, + { + "propertyName": "Dropdown selection", + "propertyKeyword": "dropdownProp", + "shouldValidate": false, + "propertyTooltip": "Select from list of options", + "propertyType": "select", + "options": [ + "Option 1", + "Option 2", + "Option 3", + "Option 4" + ] + }, + { + "propertyName": "Numeric", + "propertyKeyword": "numericProp", + "propertyValidate": true, + "propertyType": "numeric", + "valueType": "number", + "allowNegative": false, + "sampleGen": "&containerName|&entityName|&random|" + }, + { + "propertyName": "Checkbox", + "propertyKeyword": "checkboxProp", + "shouldValidate": false, + "propertyType": "checkbox" + }, + { + "propertyName": "Group", + "propertyType": "group", + "propertyKeyword": "grpProp", + "shouldValidate": true, + "propertyTooltip": "", + "structure": [ + { + "propertyName": "Simple Grp Text", + "propertyKeyword": "simpleGrpText", + "shouldValidate": false, + "propertyTooltip": "", + "propertyType": "text" + }, + { + "propertyName": "Group Number", + "propertyKeyword": "grpNumber", + "propertyValidate": true, + "propertyType": "numeric", + "valueType": "number", + "allowNegative": false + } + ] + } + +*/ + +[ + { + "lowerTab": "Details", + "containerLevelKeys": [], + "structure": [ + { + "propertyName": "Description", + "propertyKeyword": "description", + "shouldValidate": false, + "propertyTooltip": "description", + "propertyType": "details", + "template": "textarea" + }, + { + "propertyName": "Comments", + "propertyKeyword": "comments", + "shouldValidate": false, + "propertyTooltip": "comments", + "propertyType": "details", + "template": "textarea" + } + ] + } +] diff --git a/properties_pane/defaultData.json b/properties_pane/defaultData.json new file mode 100644 index 0000000..941a07d --- /dev/null +++ b/properties_pane/defaultData.json @@ -0,0 +1,35 @@ +{ + "model": { + "modelName": "New model", + "dbVersion": "", + "dbVendor": "Avro" + }, + "container": { + "name": "New namespace", + "indexes": [] + }, + "collection": { + "collectionName": "New record", + "collectionUsers": [], + "collation": {} + }, + "field": { + "name": "New field", + "required": true + }, + "patternField": { + "name": "^[a-zA-Z0-9_.-]{20}$" + }, + "multipleField": { + "primaryKey": false + }, + "subschema": {}, + "arrayItem": {}, + "choice": {}, + "relationship": {}, + "user": {}, + "view": { + "viewOn": "", + "pipeline": "" + } +} \ No newline at end of file diff --git a/properties_pane/entity_level/entityLevelConfig.json b/properties_pane/entity_level/entityLevelConfig.json new file mode 100644 index 0000000..8fbca57 --- /dev/null +++ b/properties_pane/entity_level/entityLevelConfig.json @@ -0,0 +1,127 @@ +/* +* Copyright © 2016-2017 by IntegrIT S.A. dba Hackolade. All rights reserved. +* +* The copyright to the computer software herein is the property of IntegrIT S.A. +* The software may be used and/or copied only with the written permission of +* IntegrIT S.A. or in accordance with the terms and conditions stipulated in +* the agreement/contract under which the software has been supplied. + + +In order to define custom properties for any object's properties pane, you may copy/paste from the following, +making sure that you maintain a proper JSON format. + + { + "propertyName": "Simple text", + "propertyKeyword": "simpletextProp", + "shouldValidate": false, + "propertyType": "text", + "sampleGen": "&containerName|&entityName|&random|" + }, + { + "propertyName": "Text area", + "propertyKeyword": "textareaProp", + "propertyValidate": false, + "propertyTooltip": "Popup for multi-line text entry", + "propertyType": "details", + "template": "textarea" + }, + { + "propertyName": "Dropdown selection", + "propertyKeyword": "dropdownProp", + "shouldValidate": false, + "propertyTooltip": "Select from list of options", + "propertyType": "select", + "options": [ + "Option 1", + "Option 2", + "Option 3", + "Option 4" + ] + }, + { + "propertyName": "Numeric", + "propertyKeyword": "numericProp", + "propertyValidate": true, + "propertyType": "numeric", + "valueType": "number", + "allowNegative": false, + "sampleGen": "&containerName|&entityName|&random|" + }, + { + "propertyName": "Checkbox", + "propertyKeyword": "checkboxProp", + "shouldValidate": false, + "propertyType": "checkbox" + }, + { + "propertyName": "Group", + "propertyType": "group", + "propertyKeyword": "grpProp", + "shouldValidate": true, + "propertyTooltip": "", + "structure": [ + { + "propertyName": "Simple Grp Text", + "propertyKeyword": "simpleGrpText", + "shouldValidate": false, + "propertyTooltip": "", + "propertyType": "text" + }, + { + "propertyName": "Group Number", + "propertyKeyword": "grpNumber", + "propertyValidate": true, + "propertyType": "numeric", + "valueType": "number", + "allowNegative": false + } + ] + } + +*/ + +[ + { + "lowerTab": "Details", + "structure": [ + { + "propertyName": "Doc", + "propertyKeyword": "doc", + "propertyValidate": false, + "propertyTooltip": "Popup for multi-line text entry", + "propertyType": "details", + "template": "textarea" + }, + { + "propertyName": "Namespace", + "propertyKeyword": "bucketId", + "propertyTooltip": "Namespace", + "propertyType": "selecthashed" + }, + { + "propertyName": "Additional properties", + "propertyKeyword": "additionalProperties", + "propertyTooltip": "Description", + "propertyType": "checkbox", + "template": "boolean" + }, + { + "propertyName": "Description", + "propertyKeyword": "description", + "propertyValidate": false, + "propertyTooltip": "Description", + "propertyType": "details", + "template": "textarea" + }, + { + "propertyName": "Comments", + "propertyKeyword": "comments", + "shouldValidate": false, + "propertyTooltip": "comments", + "propertyType": "details", + "template": "textarea", + "valueType": "string" + } + ] + } +] \ No newline at end of file diff --git a/properties_pane/field_level/fieldLevelConfig.json b/properties_pane/field_level/fieldLevelConfig.json new file mode 100644 index 0000000..69605ac --- /dev/null +++ b/properties_pane/field_level/fieldLevelConfig.json @@ -0,0 +1,558 @@ +/* +* Copyright © 2016-2017 by IntegrIT S.A. dba Hackolade. All rights reserved. +* +* The copyright to the computer software herein is the property of IntegrIT S.A. +* The software may be used and/or copied only with the written permission of +* IntegrIT S.A. or in accordance with the terms and conditions stipulated in +* the agreement/contract under which the software has been supplied. + + +In order to define custom properties for any object's properties pane, you may copy/paste from the following, +making sure that you maintain a proper JSON format. + + { + "propertyName": "Simple text", + "propertyKeyword": "simpletextProp", + "shouldValidate": false, + "propertyType": "text", + "sampleGen": "&containerName|&entityName|&random|" + }, + { + "propertyName": "Text area", + "propertyKeyword": "textareaProp", + "propertyValidate": false, + "propertyTooltip": "Popup for multi-line text entry", + "propertyType": "details", + "template": "textarea" + }, + { + "propertyName": "Dropdown selection", + "propertyKeyword": "dropdownProp", + "shouldValidate": false, + "propertyTooltip": "Select from list of options", + "propertyType": "select", + "options": [ + "Option 1", + "Option 2", + "Option 3", + "Option 4" + ] + }, + { + "propertyName": "Numeric", + "propertyKeyword": "numericProp", + "propertyValidate": true, + "propertyType": "numeric", + "valueType": "number", + "allowNegative": false, + "sampleGen": "&containerName|&entityName|&random|" + }, + { + "propertyName": "Checkbox", + "propertyKeyword": "checkboxProp", + "shouldValidate": false, + "propertyType": "checkbox" + }, + { + "propertyName": "Group", + "propertyType": "group", + "propertyKeyword": "grpProp", + "shouldValidate": true, + "propertyTooltip": "", + "structure": [ + { + "propertyName": "Simple Grp Text", + "propertyKeyword": "simpleGrpText", + "shouldValidate": false, + "propertyTooltip": "", + "propertyType": "text" + }, + { + "propertyName": "Group Number", + "propertyKeyword": "grpNumber", + "propertyValidate": true, + "propertyType": "numeric", + "valueType": "number", + "allowNegative": false + } + ] + }, + { + "propertyName": "Field List", + "propertyKeyword": "keyList", + "shouldValidate": false, + "propertyType": "tagInput", + "template": "collectiontree" + }, + { + "propertyName": "Field List w/ dropdown", + "propertyKeyword": "keyListOrder", + "shouldValidate": false, + "propertyType": "tagInput", + "template": "collectiontree", + "attributeList": [ + "ascending", + "descending" + ] + } +*/ + +{ + "lowerTab": "JsonDetails", + "structure": { + "string": [ + "name", + "schemaId", + "type", + { + "propertyName": "Doc", + "propertyKeyword": "doc", + "propertyValidate": false, + "propertyTooltip": "Popup for multi-line text entry", + "propertyType": "details", + "template": "textarea" + }, + "default", + { + "propertyName": "Order", + "propertyKeyword": "order", + "shouldValidate": false, + "propertyTooltip": "Select from list of options", + "propertyType": "select", + "options": [ + "ascending", + "descending", + "ignore" + ] + }, + { + "propertyName": "Aliases", + "propertyType": "text", + "propertyKeyword": "aliases", + "multi": true, + "valueType": "array" + }, + "description", + "dependencies", + { + "propertyName": "Required", + "propertyKeyword": "required", + "propertyType": "checkbox", + "template": "boolean", + "disabledOnCondition": [{ + "key": "required", + "value": true + }] + }, + "primaryKey", + "foreignCollection", + "foreignField", + "relationshipType", + "sample", + "comments" + ], + "bytes": [ + "name", + "schemaId", + "type", + { + "propertyName": "Doc", + "propertyKeyword": "doc", + "propertyValidate": false, + "propertyTooltip": "Popup for multi-line text entry", + "propertyType": "details", + "template": "textarea" + }, + { + "propertyName": "Aliases", + "propertyType": "text", + "propertyKeyword": "aliases", + "multi": true, + "valueType": "array" + }, + "description", + "dependencies", + { + "propertyName": "Required", + "propertyKeyword": "required", + "propertyType": "checkbox", + "template": "boolean", + "disabledOnCondition": [{ + "key": "required", + "value": true + }] + }, + "sample", + "comments" + ], + "number": [ + "name", + "schemaId", + "type", + { + "propertyName": "Subtype", + "propertyKeyword": "mode", + "shouldValidate": false, + "propertyType": "select", + "options": [ + "int", + "long", + "float", + "double" + ], + "data": "options", + "valueType": "string" + }, + { + "propertyName": "Doc", + "propertyKeyword": "doc", + "propertyValidate": false, + "propertyTooltip": "Popup for multi-line text entry", + "propertyType": "details", + "template": "textarea" + }, + "default", + { + "propertyName": "Order", + "propertyKeyword": "order", + "shouldValidate": false, + "propertyTooltip": "Select from list of options", + "propertyType": "select", + "options": [ + "ascending", + "descending", + "ignore" + ] + }, + { + "propertyName": "Aliases", + "propertyType": "text", + "propertyKeyword": "aliases", + "multi": true, + "valueType": "array" + }, + "description", + "dependencies", + { + "propertyName": "Required", + "propertyKeyword": "required", + "propertyType": "checkbox", + "template": "boolean", + "disabledOnCondition": [{ + "key": "required", + "value": true + }] + }, + "primaryKey", + "foreignCollection", + "foreignField", + "relationshipType", + "default", + "unit", + "minimum", + "exclusiveMinimum", + "maximum", + "exclusiveMaximum", + "multipleOf", + "divisibleBy", + "sample", + "comments" + ], + "boolean": [ + "name", + "schemaId", + "type", + { + "propertyName": "Doc", + "propertyKeyword": "doc", + "propertyValidate": false, + "propertyTooltip": "Popup for multi-line text entry", + "propertyType": "details", + "template": "textarea" + }, + "default", + { + "propertyName": "Aliases", + "propertyType": "text", + "propertyKeyword": "aliases", + "multi": true, + "valueType": "array" + }, + "description", + "dependencies", + { + "propertyName": "Required", + "propertyKeyword": "required", + "propertyType": "checkbox", + "template": "boolean", + "disabledOnCondition": [{ + "key": "required", + "value": true + }] + }, + "sample", + "comments" + ], + "null": [ + "name", + "schemaId", + { + "propertyName": "Doc", + "propertyKeyword": "doc", + "propertyValidate": false, + "propertyTooltip": "Popup for multi-line text entry", + "propertyType": "details", + "template": "textarea" + }, + { + "propertyName": "Aliases", + "propertyType": "text", + "propertyKeyword": "aliases", + "multi": true, + "valueType": "array" + }, + "type", + "description", + "dependencies", + { + "propertyName": "Required", + "propertyKeyword": "required", + "propertyType": "checkbox", + "template": "boolean", + "disabledOnCondition": [{ + "key": "required", + "value": true + }] + }, + "comments" + ], + "___1": [], + "record": [ + "name", + "schemaId", + "type", + { + "propertyName": "Namespace", + "propertyKeyword": "namespace", + "shouldValidate": false, + "propertyType": "text" + }, + { + "propertyName": "Doc", + "propertyKeyword": "doc", + "propertyValidate": false, + "propertyTooltip": "Popup for multi-line text entry", + "propertyType": "details", + "template": "textarea" + }, + { + "propertyName": "Aliases", + "propertyType": "text", + "propertyKeyword": "aliases", + "multi": true, + "valueType": "array" + }, + "description", + "dependencies", + { + "propertyName": "Required", + "propertyKeyword": "required", + "propertyType": "checkbox", + "template": "boolean", + "disabledOnCondition": [{ + "key": "required", + "value": true + }] + }, + "minProperties", + "maxProperties", + "additionalProperties", + "comments" + ], + "array": [ + "name", + "schemaId", + "type", + { + "propertyName": "Doc", + "propertyKeyword": "doc", + "propertyValidate": false, + "propertyTooltip": "Popup for multi-line text entry", + "propertyType": "details", + "template": "textarea" + }, + { + "propertyName": "Aliases", + "propertyType": "text", + "propertyKeyword": "aliases", + "multi": true, + "valueType": "array" + }, + "description", + "dependencies", + { + "propertyName": "Required", + "propertyKeyword": "required", + "propertyType": "checkbox", + "template": "boolean", + "disabledOnCondition": [{ + "key": "required", + "value": true + }] + }, + "minItems", + "maxItems", + "uniqueItems", + "additionalItems", + "comments" + ], + "map": [ + "name", + "schemaId", + "type", + { + "propertyName": "Doc", + "propertyKeyword": "doc", + "propertyValidate": false, + "propertyTooltip": "Popup for multi-line text entry", + "propertyType": "details", + "template": "textarea" + }, + { + "propertyName": "Values", + "propertyKeyword": "subtype", + "shouldValidate": false, + "propertyType": "select", + "options": [ + { "value": "map", "name": "string" }, + { "value": "map", "name": "int" }, + { "value": "map", "name": "long" }, + { "value": "map", "name": "float" }, + { "value": "map", "name": "double" } + ] + }, + { + "propertyName": "Aliases", + "propertyType": "text", + "propertyKeyword": "aliases", + "multi": true, + "valueType": "array" + }, + "description", + "dependencies", + { + "propertyName": "Required", + "propertyKeyword": "required", + "propertyType": "checkbox", + "template": "boolean", + "disabledOnCondition": [{ + "key": "required", + "value": true + }] + }, + "minProperties", + "maxProperties", + "additionalProperties", + "comments" + ], + "enum": [ + "name", + "schemaId", + "type", + { + "propertyName": "Namespace", + "propertyKeyword": "namespace", + "shouldValidate": false, + "propertyType": "text" + }, + { + "propertyName": "Doc", + "propertyKeyword": "doc", + "propertyValidate": false, + "propertyTooltip": "Popup for multi-line text entry", + "propertyType": "details", + "template": "textarea" + }, + { + "propertyName": "Aliases", + "propertyType": "text", + "propertyKeyword": "aliases", + "multi": true, + "valueType": "array" + }, + { + "propertyName": "Symbols", + "propertyType": "text", + "propertyKeyword": "symbols", + "multi": true, + "valueType": "array" + }, + "dependencies", + "description", + { + "propertyName": "Required", + "propertyKeyword": "required", + "propertyType": "checkbox", + "template": "boolean", + "disabledOnCondition": [{ + "key": "required", + "value": true + }] + }, + "default", + "pattern", + "sample", + "comments" + ], + "fixed": [ + "name", + "schemaId", + "type", + { + "propertyName": "Namespace", + "propertyKeyword": "namespace", + "shouldValidate": false, + "propertyType": "text" + }, + { + "propertyName": "Doc", + "propertyKeyword": "doc", + "propertyValidate": false, + "propertyTooltip": "Popup for multi-line text entry", + "propertyType": "details", + "template": "textarea" + }, + { + "propertyName": "Aliases", + "propertyType": "text", + "propertyKeyword": "aliases", + "multi": true, + "valueType": "array" + }, + "schemaId", + "type", + { + "propertyName": "Size", + "propertyKeyword": "fixedSize", + "propertyValidate": true, + "propertyType": "numeric", + "valueType": "number", + "allowNegative": false + }, + "dependencies", + "description", + { + "propertyName": "Required", + "propertyKeyword": "required", + "propertyType": "checkbox", + "template": "boolean", + "disabledOnCondition": [{ + "key": "required", + "value": true + }] + }, + "sample", + "comments" + ] + } +} \ No newline at end of file diff --git a/properties_pane/model_level/modelLevelConfig.json b/properties_pane/model_level/modelLevelConfig.json new file mode 100644 index 0000000..4c28c0e --- /dev/null +++ b/properties_pane/model_level/modelLevelConfig.json @@ -0,0 +1,118 @@ +/* +* Copyright © 2016-2017 by IntegrIT S.A. dba Hackolade. All rights reserved. +* +* The copyright to the computer software herein is the property of IntegrIT S.A. +* The software may be used and/or copied only with the written permission of +* IntegrIT S.A. or in accordance with the terms and conditions stipulated in +* the agreement/contract under which the software has been supplied. + + +In order to define custom properties for any object's properties pane, you may copy/paste from the following, +making sure that you maintain a proper JSON format. + + { + "propertyName": "Simple text", + "propertyKeyword": "simpletextProp", + "shouldValidate": false, + "propertyType": "text", + "sampleGen": "&containerName|&entityName|&random|" + }, + { + "propertyName": "Text area", + "propertyKeyword": "textareaProp", + "propertyValidate": false, + "propertyTooltip": "Popup for multi-line text entry", + "propertyType": "details", + "template": "textarea" + }, + { + "propertyName": "Dropdown selection", + "propertyKeyword": "dropdownProp", + "shouldValidate": false, + "propertyTooltip": "Select from list of options", + "propertyType": "select", + "options": [ + "Option 1", + "Option 2", + "Option 3", + "Option 4" + ] + }, + { + "propertyName": "Numeric", + "propertyKeyword": "numericProp", + "propertyValidate": true, + "propertyType": "numeric", + "valueType": "number", + "allowNegative": false, + "sampleGen": "&containerName|&entityName|&random|" + }, + { + "propertyName": "Checkbox", + "propertyKeyword": "checkboxProp", + "shouldValidate": false, + "propertyType": "checkbox" + }, + { + "propertyName": "Group", + "propertyType": "group", + "propertyKeyword": "grpProp", + "shouldValidate": true, + "propertyTooltip": "", + "structure": [ + { + "propertyName": "Simple Grp Text", + "propertyKeyword": "simpleGrpText", + "shouldValidate": false, + "propertyTooltip": "", + "propertyType": "text" + }, + { + "propertyName": "Group Number", + "propertyKeyword": "grpNumber", + "propertyValidate": true, + "propertyType": "numeric", + "valueType": "number", + "allowNegative": false + } + ] + } + +*/ + +[ + { + "lowerTab": "Details", + "structure": [ + { + "propertyName": "DB vendor", + "propertyKeyword": "dbVendor", + "shouldValidate": false, + "propertyTooltip": "DB vendor", + "propertyType": "select", + "options": [ + "Avro" + ], + "disabledOption": true + }, + { + "propertyName": "DB version", + "propertyKeyword": "dbVersion", + "shouldValidate": false, + "propertyTooltip": "DB version", + "propertyType": "select", + "options": [ + ], + "disabledOption": true + }, + { + "propertyName": "Comments", + "propertyKeyword": "comments", + "shouldValidate": false, + "propertyTooltip": "comments", + "propertyType": "details", + "template": "textarea" + } + ] + } +] \ No newline at end of file diff --git a/properties_pane/view_level/viewLevelConfig.json b/properties_pane/view_level/viewLevelConfig.json new file mode 100644 index 0000000..37ad202 --- /dev/null +++ b/properties_pane/view_level/viewLevelConfig.json @@ -0,0 +1,97 @@ +/* +* Copyright © 2016-2017 by IntegrIT S.A. dba Hackolade. All rights reserved. +* +* The copyright to the computer software herein is the property of IntegrIT S.A. +* The software may be used and/or copied only with the written permission of +* IntegrIT S.A. or in accordance with the terms and conditions stipulated in +* the agreement/contract under which the software has been supplied. + + +In order to define custom properties for any object's properties pane, you may copy/paste from the following, +making sure that you maintain a proper JSON format. + + { + "propertyName": "Simple text", + "propertyKeyword": "simpletextProp", + "shouldValidate": false, + "propertyType": "text", + "sampleGen": "&containerName|&entityName|&random|" + }, + { + "propertyName": "Text area", + "propertyKeyword": "textareaProp", + "propertyValidate": false, + "propertyTooltip": "Popup for multi-line text entry", + "propertyType": "details", + "template": "textarea" + }, + { + "propertyName": "Dropdown selection", + "propertyKeyword": "dropdownProp", + "shouldValidate": false, + "propertyTooltip": "Select from list of options", + "propertyType": "select", + "options": [ + "Option 1", + "Option 2", + "Option 3", + "Option 4" + ] + }, + { + "propertyName": "Numeric", + "propertyKeyword": "numericProp", + "propertyValidate": true, + "propertyType": "numeric", + "valueType": "number", + "allowNegative": false, + "sampleGen": "&containerName|&entityName|&random|" + }, + { + "propertyName": "Checkbox", + "propertyKeyword": "checkboxProp", + "shouldValidate": false, + "propertyType": "checkbox" + }, + { + "propertyName": "Group", + "propertyType": "group", + "propertyKeyword": "grpProp", + "shouldValidate": true, + "propertyTooltip": "", + "structure": [ + { + "propertyName": "Simple Grp Text", + "propertyKeyword": "simpleGrpText", + "shouldValidate": false, + "propertyTooltip": "", + "propertyType": "text" + }, + { + "propertyName": "Group Number", + "propertyKeyword": "grpNumber", + "propertyValidate": true, + "propertyType": "numeric", + "valueType": "number", + "allowNegative": false + } + ] + } + +*/ + +[ + { + "lowerTab": "Details", + "structure": [ + { + "propertyName": "Comments", + "propertyKeyword": "comments", + "shouldValidate": false, + "propertyTooltip": "comments", + "propertyType": "details", + "template": "textarea" + } + ] + } +] \ No newline at end of file diff --git a/reverse_engineering/api.js b/reverse_engineering/api.js new file mode 100644 index 0000000..0429dd4 --- /dev/null +++ b/reverse_engineering/api.js @@ -0,0 +1,247 @@ +'use strict' + +const fs = require('fs'); +const path = require('path'); +const _ = require('lodash'); +const avro = require('avsc'); +const snappy = require('snappyjs'); +const DEFAULT_FIELD_NAME = 'New Field'; +let stateExtension = null; + +const ADDITIONAL_PROPS = ['name', 'doc', 'order', 'aliases', 'symbols', 'namespace']; + +module.exports = { + reFromFile(data, logger, callback) { + handleFileData(data.filePath) + .then(fileData => { + return parseData(fileData); + }) + .then(schema => { + const jsonSchema = convertToJsonSchema(schema); + try { + const namespace = jsonSchema.namespace; + jsonSchema.title = jsonSchema.name; + delete jsonSchema.namespace; + delete jsonSchema.name; + const strJsonSchema = JSON.stringify(jsonSchema, null, 4); + return callback(null, { jsonSchema: strJsonSchema, extension: stateExtension, containerName: namespace }); + } catch (err) { + logger.log('error', { message: err.message, stack: err.stack }, 'Parsing Avro Schema Error'); + return callback(handleErrorObject(err)) + } + }) + .catch(err => { + logger.log('error', { message: err.message, stack: err.stack }, 'Avro Reverse-Engineering Error'); + callback(err) + }); + } +}; + +const getFileExt = (filePath) => { + return path.extname(filePath); +}; + +const handleFileData = (filePath) => { + return new Promise((resolve, reject) => { + const extension = getFileExt(filePath); + stateExtension = extension; + const respond = (err, content) => { + if(err){ + reject(handleErrorObject(err)); + } else { + resolve(content); + } + }; + + if (extension === '.avro') { + readAvroData(filePath, respond); + } else if (extension === '.avsc') { + fs.readFile(filePath, 'utf-8', respond); + } else { + const error = new Error(`The file ${filePath} is not recognized as Avro Schema or Data.`) + respond(error); + } + }); +}; + +const readAvroData = (filePath, cb) => { + const codecs = { + snappy: function (buf, cb) { + const uncompressed = snappy.uncompress(buf.slice(0, buf.length - 4)); + return cb(uncompressed); + }, + null: function (buf, cb) { cb(null, buf); } + }; + + + avro.createFileDecoder(filePath, { codecs }) + .on('metadata', (type, codecs, header) => { + try { + const schema = JSON.stringify(type); + return cb(null, schema); + } catch (error) { + return cb(handleErrorObject(error)); + } + }) + .on('error', cb); +}; + + +const parseData = (fileData) => { + return new Promise((resolve, reject) => { + try { + resolve(JSON.parse(fileData)); + } catch(err) { + reject(handleErrorObject(err)); + } + }); +}; + +const convertToJsonSchema = (data) => { + let jsonSchema = {}; + handleRecursiveSchema(data, jsonSchema); + jsonSchema.type = 'object'; + jsonSchema.$schema = 'http://json-schema.org/draft-04/schema#'; + return jsonSchema; +}; + +const handleRecursiveSchema = (data, schema, parentSchema = {}) => { + for (let prop in data) { + switch(prop) { + case 'type': + handleType(data, prop, schema, parentSchema); + break; + case 'fields': + handleFields(data, prop, schema); + break; + case 'items': + handleItems(data, prop, schema); + break; + default: + handleOtherProps(data, prop, schema); + } + } + return; +}; + + +const handleType = (data, prop, schema, parentSchema) => { + if (Array.isArray(data[prop])) { + schema = handleMultipleTypes(data, prop, schema, parentSchema); + } else if (typeof data[prop] === 'object') { + handleRecursiveSchema(data[prop], schema); + } else { + schema = getType(schema, data, data[prop]); + } +}; + + +const handleMultipleTypes = (data, prop, schema, parentSchema) => { + const hasComplexType = data[prop].find(item => typeof item !== 'string'); + + if (hasComplexType) { + parentSchema = getChoice(data, prop, parentSchema); + parentSchema = removeChangedField(parentSchema, data.name); + } else { + schema[prop] = data[prop]; + } +}; + +const removeChangedField = (parentSchema, name) => { + if (parentSchema.properties) { + delete parentSchema.properties[name]; + } else if (parentSchema.items) { + // delete multiple array item + } + return parentSchema; +}; + +const getType = (schema, field, type) => { + switch(type) { + case 'string': + case 'bytes': + case 'boolean': + case 'null': + case 'record': + case 'array': + case 'enum': + case 'fixed': + return Object.assign(schema, { type }); + case 'int': + case 'long': + case 'float': + case 'double': + return Object.assign(schema, { + type: 'number', + mode: type + }); + case 'map': + return Object.assign(schema, { + type, + subtype: `map<${field.values}>` + }); + default: + return Object.assign(schema, { type: 'string' }); + } +}; + +const getChoice = (data, prop, parentSchema) => { + parentSchema.oneOf = []; + data[prop].forEach(item => { + const name = data.name || DEFAULT_FIELD_NAME; + const subField = getSubField(item); + const subFieldSchema = {}; + handleRecursiveSchema(subField, subFieldSchema); + + const subSchema = { + type: 'object', + properties: { + [name]: subFieldSchema + } + }; + parentSchema.oneOf.push(subSchema); + }); + return parentSchema; +}; + +const getSubField = (item) => { + const field = (typeof item === 'object') ? item : { type: item }; + return field; +}; + +const handleFields = (data, prop, schema) => { + schema.properties = {}; + data[prop].forEach(element => { + const name = element.name || DEFAULT_FIELD_NAME; + schema.properties[name] = {}; + handleRecursiveSchema(element, schema.properties[name], schema); + }); +}; + +const handleItems = (data, prop, schema) => { + const items = data[prop]; + + if (typeof items === 'object') { + schema.items = {}; + handleRecursiveSchema(items, schema.items, schema); + } else { + schema.items = { + type: items + }; + } +}; + +const handleOtherProps = (data, prop, schema) => { + if (ADDITIONAL_PROPS.includes(prop)) { + schema[prop] = data[prop]; + } + return; +}; + +const handleErrorObject = (error) => { + let plainObject = {}; + Object.getOwnPropertyNames(error).forEach(function(key) { + plainObject[key] = error[key]; + }); + return plainObject; +}; \ No newline at end of file diff --git a/reverse_engineering/config.json b/reverse_engineering/config.json new file mode 100644 index 0000000..836c770 --- /dev/null +++ b/reverse_engineering/config.json @@ -0,0 +1,5 @@ +{ + "menu_label": "Avro schema or data file", + "name": "avro", + "extensions": ["avsc", "avro"] +} \ No newline at end of file diff --git a/reverse_engineering/package.json b/reverse_engineering/package.json new file mode 100644 index 0000000..f143c6e --- /dev/null +++ b/reverse_engineering/package.json @@ -0,0 +1,13 @@ +{ + "name": "Avro", + "version": "1.0.0", + "description": "", + "author": "Hackolade", + "dependencies": { + "avsc": "^5.4.6", + "lodash": "^4.17.11", + "snappyjs": "^0.6.0" + }, + "type": "file", + "installed": false +} \ No newline at end of file diff --git a/snippets/geopoint-object.json b/snippets/geopoint-object.json new file mode 100644 index 0000000..44531d4 --- /dev/null +++ b/snippets/geopoint-object.json @@ -0,0 +1,11 @@ +{ + "name": "geopoint-object", + "parentType": "document", + "properties": [{ + "name": "lat", + "type": "number" + }, { + "name": "lon", + "type": "number" + }] +} diff --git a/types/array.json b/types/array.json new file mode 100644 index 0000000..c1bdd6a --- /dev/null +++ b/types/array.json @@ -0,0 +1,22 @@ +{ + "name": "array", + "erdAbbreviation": "", + "dtdAbbreviation": "[...]", + "parentType": "array", + "sample": ["sample"], + "useSample": true, + "defaultValues": { + "required": true, + "properties": [], + "primaryKey": false, + "relationshipType": "", + "parentRelationship": "", + "childRelationships": [], + "foreignCollection": "", + "foreignField": [], + "additionalItems": true, + "minItems": "", + "maxItems": "", + "uniqueItems": false + } +} diff --git a/types/boolean.json b/types/boolean.json new file mode 100644 index 0000000..fa56767 --- /dev/null +++ b/types/boolean.json @@ -0,0 +1,20 @@ +{ + "name": "boolean", + "erdAbbreviation": "", + "dtdAbbreviation": "{0/1}", + "parentType": "boolean", + "sample": true, + "useSample": true, + "defaultValues": { + "required": true, + "default": "", + "enum": [], + "primaryKey": false, + "relationshipType": "", + "parentRelationship": "", + "childRelationships": [], + "foreignEntity": "", + "foreignField": [], + "sample": "" + } +} diff --git a/types/bytes.json b/types/bytes.json new file mode 100644 index 0000000..8624476 --- /dev/null +++ b/types/bytes.json @@ -0,0 +1,22 @@ +{ + "name": "bytes", + "erdAbbreviation": "", + "dtdAbbreviation": "{bytes}", + "parentType": "string", + "useSample": true, + "defaultValues": { + "required": true, + "minLength": "", + "maxLength": "", + "pattern": "", + "default": "", + "primaryKey": false, + "relationshipType": "", + "parentRelationship": "", + "childRelationships": [], + "foreignCollection": "", + "foreignField": [], + "enum": [], + "sample": "" + } +} \ No newline at end of file diff --git a/types/enum.json b/types/enum.json new file mode 100644 index 0000000..1afffb7 --- /dev/null +++ b/types/enum.json @@ -0,0 +1,22 @@ +{ + "name": "enum", + "erdAbbreviation": "", + "dtdAbbreviation": "{enum}", + "parentType": "string", + "useSample": true, + "defaultValues": { + "required": true, + "minLength": "", + "maxLength": "", + "pattern": "[A-Za-z0-9_]", + "default": "", + "primaryKey": false, + "relationshipType": "", + "parentRelationship": "", + "childRelationships": [], + "foreignCollection": "", + "foreignField": [], + "enum": [], + "sample": "" + } +} \ No newline at end of file diff --git a/types/fixed.json b/types/fixed.json new file mode 100644 index 0000000..0000bc7 --- /dev/null +++ b/types/fixed.json @@ -0,0 +1,22 @@ +{ + "name": "fixed", + "erdAbbreviation": "", + "dtdAbbreviation": "{fxd}", + "parentType": "string", + "useSample": true, + "defaultValues": { + "required": true, + "minLength": "", + "maxLength": "", + "pattern": "", + "default": "", + "primaryKey": false, + "relationshipType": "", + "parentRelationship": "", + "childRelationships": [], + "foreignCollection": "", + "foreignField": [], + "enum": [], + "sample": "" + } +} \ No newline at end of file diff --git a/types/map.json b/types/map.json new file mode 100644 index 0000000..0eb34f0 --- /dev/null +++ b/types/map.json @@ -0,0 +1,33 @@ +{ + "name": "map", + "erdAbbreviation": "", + "dtdAbbreviation": "{...}", + "parentType": "document", + "useSample": true, + "defaultValues": { + "subtype": "map", + "dependencies": [], + "primaryKey": false, + "properties": [], + "minProperties": "", + "maxProperties": "", + "additionalProperties": false + }, + "subtypes": { + "map": { + "childValueType": "string" + }, + "map": { + "childValueType": "number" + }, + "map": { + "childValueType": "number" + }, + "map": { + "childValueType": "number" + }, + "map": { + "childValueType": "number" + } + } +} \ No newline at end of file diff --git a/types/null.json b/types/null.json new file mode 100644 index 0000000..f8b0afb --- /dev/null +++ b/types/null.json @@ -0,0 +1,17 @@ +{ + "name": "null", + "erdAbbreviation": "", + "dtdAbbreviation": "{null}", + "useSample": false, + "defaultValues": { + "required": true, + "type": "null", + "primaryKey": false, + "relationshipType": "", + "parentRelationship": "", + "childRelationships": [], + "foreignEntity": "", + "foreignField": [], + "format": "" + } +} diff --git a/types/number.json b/types/number.json new file mode 100644 index 0000000..222de1e --- /dev/null +++ b/types/number.json @@ -0,0 +1,28 @@ +{ + "name": "number", + "erdAbbreviation": "", + "dtdAbbreviation": "{123}", + "parentType": "numeric", + "sample": 15, + "useSample": true, + "defaultValues": { + "required": true, + "unit": "", + "minimum": "", + "exclusiveMinimum": false, + "maximum": "", + "exclusiveMaximum": false, + "multipleOf": "", + "divisibleBy": "", + "default": "", + "primaryKey": false, + "relationshipType": "", + "parentRelationship": "", + "childRelationships": [], + "foreignCollection": "", + "foreignField": [], + "enum": [], + "mode": "", + "sample": "" + } +} diff --git a/types/record.json b/types/record.json new file mode 100644 index 0000000..2569604 --- /dev/null +++ b/types/record.json @@ -0,0 +1,21 @@ +{ + "name": "record", + "erdAbbreviation": "", + "dtdAbbreviation": "{...}", + "parentType": "document", + "defaultValues": { + "required": true, + "primaryKey": false, + "relationshipType": "", + "parentRelationship": "", + "childRelationships": [], + "foreignCollection": "", + "foreignField": [], + "properties": [], + "dependencies": [], + "minProperties": "", + "maxProperties": "", + "additionalProperties": false, + "enum": [] + } +} \ No newline at end of file diff --git a/types/string.json b/types/string.json new file mode 100644 index 0000000..4ec5036 --- /dev/null +++ b/types/string.json @@ -0,0 +1,21 @@ +{ + "name": "string", + "erdAbbreviation": "", + "dtdAbbreviation": "{ABC}", + "useSample": true, + "defaultValues": { + "required": true, + "minLength": "", + "maxLength": "", + "pattern": "", + "default": "", + "primaryKey": false, + "relationshipType": "", + "parentRelationship": "", + "childRelationships": [], + "foreignCollection": "", + "foreignField": [], + "enum": [], + "sample": "" + } +} \ No newline at end of file diff --git a/validation/validationRegularExpressions.json b/validation/validationRegularExpressions.json new file mode 100644 index 0000000..4eb7c3f --- /dev/null +++ b/validation/validationRegularExpressions.json @@ -0,0 +1,3 @@ +{ + +} \ No newline at end of file