Passing key into update/remove process xslt for iso 19139 to fix issue with updating/deleting resources with same url #7431
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -33,6 +33,8 @@ | |
| xmlns:xlink="http://www.w3.org/1999/xlink" | ||
| xmlns:gn-fn-index="http://geonetwork-opensource.org/xsl/functions/index" | ||
| xmlns:index="java:org.fao.geonet.kernel.search.EsSearchManager" | ||
| xmlns:digestUtils="java:org.apache.commons.codec.digest.DigestUtils" | ||
| xmlns:exslt="http://exslt.org/common" | ||
| xmlns:util="java:org.fao.geonet.util.XslUtil" | ||
| xmlns:date-util="java:org.fao.geonet.utils.DateUtil" | ||
| xmlns:daobs="http://daobs.org" | ||
|
|
@@ -1105,6 +1107,8 @@ | |
| <atomfeed><xsl:value-of select="gmd:linkage/gmd:URL"/></atomfeed> | ||
| </xsl:if> | ||
| <link type="object">{ | ||
| "hash": "<xsl:value-of select="digestUtils:md5Hex(string(exslt:node-set(normalize-space(.))))"/>", | ||
|
There was a problem hiding this comment. Working on a new indexing module, I think we could simplify this to There was a problem hiding this comment. @fxprunayre you are correct that this seems cleaner. |
||
| "idx": <xsl:value-of select="position()"/>, | ||
| "protocol":"<xsl:value-of select="util:escapeForJson((gmd:protocol/*/text())[1])"/>", | ||
| "mimeType":"<xsl:value-of select="if (*/gmx:MimeFileType) | ||
| then util:escapeForJson(*/gmx:MimeFileType/@type) | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -25,11 +25,20 @@ | |
| <!-- | ||
| Processing to insert or update an online resource element. | ||
| Insert is made in first transferOptions found. | ||
|
|
||
| Note: It assumes that it will be adding new items in | ||
| the first /gmd:distributionInfo | ||
| and first /gmd:MD_Distribution | ||
| and first /gmd:transferOptions | ||
| --> | ||
| <xsl:stylesheet xmlns:gmd="http://www.isotc211.org/2005/gmd" | ||
| xmlns:gco="http://www.isotc211.org/2005/gco" | ||
| xmlns:gmx="http://www.isotc211.org/2005/gmx" | ||
| xmlns:xsl="http://www.w3.org/1999/XSL/Transform" | ||
| xmlns:xs="http://www.w3.org/2001/XMLSchema" | ||
| xmlns:digestUtils="java:org.apache.commons.codec.digest.DigestUtils" | ||
| xmlns:exslt="http://exslt.org/common" | ||
wangf1122 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| exclude-result-prefixes="#all" | ||
| version="2.0"> | ||
|
|
||
| <!-- Main properties for the link. | ||
|
|
@@ -52,17 +61,29 @@ Insert is made in first transferOptions found. | |
| in this one. --> | ||
| <xsl:param name="extra_metadata_uuid"/> | ||
|
|
||
| <!-- Target element to update. | ||
| updateKey is used to identify the resource name to be updated - it is for backwards compatibility. Will not be used if resourceHash is set. | ||
| The key is based on the concatenation of URL+Protocol+Name | ||
| resourceHash is hash value of the object to be removed which will ensure the correct value is removed. It will override the usage of updateKey | ||
| resourceIdx is the index location of the object to be removed - can be used when duplicate entries exists to ensure the correct one is removed. | ||
| --> | ||
|
|
||
| <xsl:param name="updateKey" select="''"/> | ||
| <xsl:param name="resourceHash" select="''"/> | ||
| <xsl:param name="resourceIdx" select="''"/> | ||
|
|
||
| <xsl:variable name="update_flag"> | ||
| <xsl:value-of select="boolean($updateKey != '' or $resourceHash != '' or $resourceIdx != '')"/> | ||
| </xsl:variable> | ||
|
|
||
| <xsl:variable name="mainLang"> | ||
| <xsl:value-of | ||
| select="(gmd:MD_Metadata|*[@gco:isoType='gmd:MD_Metadata'])/gmd:language/gmd:LanguageCode/@codeListValue"/> | ||
| </xsl:variable> | ||
|
|
||
| <!-- Add new gmd:onLine and consider cases where parent elements don't exist --> | ||
| <!-- <gmd:distributionInfo> does not exist--> | ||
| <xsl:template match="gmd:MD_Metadata[not(gmd:distributionInfo) and $update_flag = false()]|*[@gco:isoType='gmd:MD_Metadata' and not(gmd:distributionInfo) and $update_flag = false()]"> | ||
| <xsl:copy> | ||
| <xsl:apply-templates select="@*"/> | ||
| <xsl:apply-templates | ||
|
|
@@ -86,41 +107,15 @@ Insert is made in first transferOptions found. | |
|
|
||
| <gmd:distributionInfo> | ||
| <gmd:MD_Distribution> | ||
| <gmd:transferOptions> | ||
| <gmd:MD_DigitalTransferOptions> | ||
| <xsl:call-template name="createOnlineSrc"/> | ||
| </gmd:MD_DigitalTransferOptions> | ||
| </gmd:transferOptions> | ||
| </gmd:MD_Distribution> | ||
| </gmd:distributionInfo> | ||
|
|
||
|
|
||
| <xsl:apply-templates | ||
| select="gmd:dataQualityInfo| | ||
| gmd:portrayalCatalogueInfo| | ||
|
|
@@ -135,14 +130,72 @@ Insert is made in first transferOptions found. | |
| </xsl:copy> | ||
| </xsl:template> | ||
|
|
||
| <!-- <gmd:MD_Distribution> does not exist--> | ||
| <xsl:template match="*/gmd:distributionInfo[not(gmd:MD_Distribution) and $update_flag = false() and position() = 1]"> | ||
| <xsl:copy> | ||
| <xsl:apply-templates select="node()|@*"/> | ||
| <gmd:MD_Distribution> | ||
| <gmd:transferOptions> | ||
| <gmd:MD_DigitalTransferOptions> | ||
| <xsl:call-template name="createOnlineSrc"/> | ||
| </gmd:MD_DigitalTransferOptions> | ||
| </gmd:transferOptions> | ||
| </gmd:MD_Distribution> | ||
| </xsl:copy> | ||
| </xsl:template> | ||
|
|
||
| <!-- <gmd:transferOptions> does not exist--> | ||
| <xsl:template match="*/gmd:distributionInfo[1]/gmd:MD_Distribution[not(gmd:transferOptions) and $update_flag = false() and position() = 1]"> | ||
| <xsl:copy> | ||
| <xsl:apply-templates select="node()|@*"/> | ||
| <gmd:transferOptions> | ||
| <gmd:MD_DigitalTransferOptions> | ||
| <xsl:call-template name="createOnlineSrc"/> | ||
| </gmd:MD_DigitalTransferOptions> | ||
| </gmd:transferOptions> | ||
| </xsl:copy> | ||
| </xsl:template> | ||
|
|
||
| <!-- <gmd:MD_DigitalTransferOptions> does not exist--> | ||
| <xsl:template match="*/gmd:distributionInfo[1]/gmd:MD_Distribution[1]/gmd:transferOptions[not(gmd:MD_DigitalTransferOptions) and $update_flag = false() and position() = 1]"> | ||
| <xsl:copy> | ||
| <xsl:apply-templates select="node()|@*"/> | ||
| <gmd:MD_DigitalTransferOptions> | ||
| <xsl:call-template name="createOnlineSrc"/> | ||
| </gmd:MD_DigitalTransferOptions> | ||
| </xsl:copy> | ||
| </xsl:template> | ||
|
|
||
| <!-- Add new gmd:gmd:onLine--> | ||
| <xsl:template match="*/gmd:distributionInfo[1]/gmd:MD_Distribution[1]/gmd:transferOptions[1]/gmd:MD_DigitalTransferOptions[$update_flag = false() and position() = 1]"> | ||
| <xsl:copy> | ||
| <xsl:apply-templates select="@*"/> | ||
| <xsl:apply-templates | ||
| select="gmd:unitsOfDistribution| | ||
| gmd:transferSize| | ||
| gmd:onLine"/> | ||
|
|
||
| <xsl:call-template name="createOnlineSrc"/> | ||
|
|
||
| <xsl:apply-templates select="gmd:offLine"/> | ||
| </xsl:copy> | ||
| </xsl:template> | ||
|
|
||
| <!-- End of inserting gmd:onLine --> | ||
|
|
||
|
|
||
| <!-- Updating the gmd:onLine based on update parameters --> | ||
| <!-- Note: first part of the match needs to match the xsl:for-each select from extract-relations.xsl in order to get the position() to match --> | ||
|
There was a problem hiding this comment. I would write a comment explaining how it works, something like this:
That would facilitate the maintenance in the long term, as the condition is not that simple. Please if you can also update the other xslt files with similar conditions. There was a problem hiding this comment. I have added some extra comment explain what the resourceIdx and resourceHash for both add and remove xsl |
||
| <!-- The unique identifier is marked with resourceIdx which is the position index and resourceHash which is hash code of the current node (combination of url, resource name, and description) --> | ||
| <xsl:template | ||
| match="*//gmd:MD_DigitalTransferOptions/gmd:onLine | ||
| [gmd:CI_OnlineResource[gmd:linkage/gmd:URL!=''] and ($resourceIdx = '' or position() = xs:integer($resourceIdx))] | ||
|
There was a problem hiding this comment. The position here is computed in the context of Maybe we should simplify the process with the hash only support? <xsl:template
match="*//gmd:MD_DigitalTransferOptions/gmd:onLine[digestUtils:md5Hex(normalize-space(.)) = $resourceHash]"
priority="2">
</xsl:template>This was also not applied to 115-3 https://github.com/geonetwork/core-geonetwork/blob/main/schemas/iso19115-3.2018/src/main/plugin/iso19115-3.2018/process/onlinesrc-remove.xsl There was a problem hiding this comment. @fxprunayre
There was a problem hiding this comment.
Our intent was to add it but did not get around to it. Now that there are a couple bugs need to be fix. Probably once these are fixed then we can add support to 115-3 There was a problem hiding this comment. Yes indeed it depends a bit on how people organize things in ISO distribution blocks or not. From use cases I have here, I would say hash of (URL+name+protocol) should be unique but user can always create duplicates in different block. We can keep it backward compatible like it is now but if relying on the position, we need to compute a position using the same context (currently the context in indexing is different). Will think about it, if any better options. |
||
| [($resourceHash != '' or ($updateKey != '' and normalize-space($updateKey) = concat( | ||
| gmd:CI_OnlineResource/gmd:linkage/gmd:URL, | ||
| gmd:CI_OnlineResource/gmd:protocol/*, | ||
| gmd:CI_OnlineResource/gmd:name/gco:CharacterString))) | ||
| and ($resourceHash = '' or digestUtils:md5Hex(string(exslt:node-set(normalize-space(.)))) = $resourceHash)]" | ||
| priority="2"> | ||
| <xsl:call-template name="createOnlineSrc"/> | ||
| </xsl:template> | ||
|
|
||
|
|
@@ -170,7 +223,7 @@ Insert is made in first transferOptions found. | |
|
|
||
| <xsl:if test="$url"> | ||
| <!-- In case the protocol is an OGC protocol | ||
| the name parameter may contain a list of layers | ||
| separated by comma. | ||
| In that case on one online element is added per | ||
| layer/featureType. | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@wangf1122 this is required for https://github.com/geonetwork/core-geonetwork/pull/7431/files#diff-8ba6fa3b50cb72ae6c6e65d46bf59e1b4f7a4c2102df0c7f1e1c779ae08db967R134
I tested uploading a thumbnail and it's failing due to this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for pointing it out. I just realized this namespace is used. I have placed it back.