Skip to content

Commit

Permalink
feat: Attempt to simply migrations
Browse files Browse the repository at this point in the history
  • Loading branch information
f-necas committed Dec 1, 2023
1 parent 9d809d2 commit 2dad0f8
Show file tree
Hide file tree
Showing 13 changed files with 189 additions and 0 deletions.
8 changes: 8 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
**Geochestra/geonetwork checklist**
<!--- In order to ease future geonetwork migration : -->

- [ ] PR only involves cherry picked commits from upstream.
- [ ] PR contains custom code which will be soon available in a release on upstream and can be overrided.
- [ ] PR contains custom georchestra code, which need to be verified in future migration.
- [ ] I have properly filled [migration-helper-changelog.md](..%2Fgeorchestra-migration%2Fmigration-helper-changelog.md) file.

12 changes: 12 additions & 0 deletions georchestra-migration/migration-helper-changelog.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Migration changelog.

<!--
Example of migration helper.
### Description
Describe what's the purpose of those modification.
**Files modified**
List of absolute files to check.
-->

## Already updated in latest diff-tree
120 changes: 120 additions & 0 deletions georchestra-migration/migration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
# Geonetwork migration in Georchestra

This document aims to help simplify future migration for geonetwork in Georchestra.

## Upgrade Process

### Simplified guide

This is an ultra-simplified guide to be able to build an upgraded version of georchestra/geonetwork
- Merge gn-core tag into georchestra/gn wanted branch. Resolve coflicts and get georchestra custom implementions (see below).
- Copy folders `config` and `data` from `web/src/main/webapp/WEB-INF/data/` into the repo [geonetwork_minimal_datadir](https://github.com/georchestra/geonetwork_minimal_datadir) and create associated branch with new version name.
- Build geonetwork (war, docker ...)
- Delete wro4j cache if necessary (use ?debug can be sometimes useful)
- Reindex records if necessary

A more detailed guide is available in [upgrade_geonetwork.md](upgrade_geonetwork.md).

## Georchestra custom implementations

### Georchestra 4.2.7 and Gn 4.2.7

All versions in `pom.xml` files must be updated. E.g. for this version `4.2.7-georchestra`.
All italic folder just have the `pom.xml` change.

- .github
- workflows `linux.yml` `mvn-dep-tree.yml` `sonarcloud.yml` `dependabot.yml` are deleted. Only `georchestra-gn4.yml` is used
- *cachingxslt*
- common
- `ZipUtilTest.java` : In `assertExampleZip` method, assertions which are supposed check folders must check with trailing slash too.
- core
- `JeevesContextLoaderListener.java` : remove java 8 runtime exception as we use java 11.
- `XslUtil.java`: Implement georchestra header specific code
- `config-spring-geonetwork.xml` : Implement `context:property-placeholder` for georchestra's datadir
- `cleanoutdatabase.sql`: Delete from settings-ui at the end of the file
- *csw-server*
- **docker**
- Mandatory, get everything from geOrchestra
- *docs*
- *doi*
- domain
- ExternalGroupLink.java : get file from geOrchestra
- ExternalUserLink.java : get file from geOrchestra
- *es*
- *estest*
- *events*
- **georchestra-integration**
- Mandatory, get everything from geOrchestra
- harvesters
- `Harvester.java`: Use parseJDK11 method
- `HarvesterTest.java` : Ignore and assume true Java 8
- *healthmonitor*
- *index*
- *inspire-atom*
- *jmeter*
- *listeners*
- *messaging*
- *oaipmh*
- *release*
- schemas
- `src/main/plugin/iso19115-3.2018`
- OGC API features and 3Dtiles added. Differences in files:
- `config/associated-panel/default.json`
- `loc/eng/labels.xml`
- `loc/fre/labels.xml`
- `test/resources/metadata-for-editing.xml`
- `test/resources/metadata-for-editing-light.xml`
- `test/resources/metadata-iso19139-for-editing.xml`
- `src/main/plugin/iso19139/loc` : 3Dtiles added in labels.xml files.
- *schemas-test*
- *sde*
- services
- `config-spring-geonetwork.xml`: Implement `context:property-placeholder` for georchestra's datadir
- `MetadataExtentApiTest.java` : Update image signatures if necessary (tests may fail see [MetadataExtentApiTest-reference](resources%2FMetadataExtentApiTest-reference) for image reference)
- `BatchOpsMetadatReindexerTest.java` : Add PowerMockIgnore
- `pom.xml`: version to update **and to add to gn-services**
- *slave*
- web
- `src/docker`: Mandatory, get everything from geOrchestra
- `main/filters/prod.properties`: Session timeout variable updated
- `main/filters/java/org/fao/geonet/proxy`: Mandatory, get everything from geOrchestra
- `data-db-default.sql` : Some data to retrieve: setting-ui, inspire activated, georchestra name and (backported value) sitemapLinkUrl
- `UpdateMetadataStatus.java`: Some fixes
- `config-security.xml`: Remove config security add start of file, get end of the file from georchestra.
- `config-geonetwork-georchestra.properties`, `config-georchestra-geonetwork-datadirs.xml`, `DKAN-to-ISO19115-3-2018.xsl`, `GeoIDE-services-OGC.xsl`, `udata-to-ISO19115-3-2018.xsl`
- Get files from georchestra
- `base-variables.xsl` and `base-layout.xsl`
- Get header from georchestra
- `defaultJdbcDataSource.xml`: Implement `context:property-placeholder` for georchestra's db
- `config-spring-geonetwork.xml`
- Implement `context:property-placeholder` for georchestra's datadir
- Don't forget to import config resource and logging bean
- `spring-servlet.xml`: Implement `context:property-placeholder` for georchestra's datadir
- `pom.xml`: Update `dockerGnDatadirScmVersion` variable accordingly
- web-ui
- `pom.xml`: version to update **and to add to gn-web-ui**
- `RelatedResourcesService.js`: Add 3DTiles
- `CatController.js`: Menu bar accessible and 3DTiles
- `src/main/resources/catalog/locales/`
- Add OGC API - Features to i18n files
- Get `en-georchestra.json` and `fr-georchestra.json` from georchestra
- `src/main/resources/catalog/style/gn-less`: remove position fixed from `.gn-top-bar` to get gn header after georchestra's header.
- `gn_admin_default.less`: Same (header position)
- `gn_navbar_default.less`: Same (header position)
- `src/main/resources/catalog/views/module.js`: Add 3DTiles
- `src/main/resources/catalog/views/georchestra/`: Get files from georchestra
- *workers*
- *wro4j*
- .gitignore
- add idea and settings to it
- pom.xml
- Implement JDK 11 specific things, add georchestra-integration module, set db-type

## Process used

### 4.2.4 -> 4.2.7

A branch has been created from core-geonetwork 4.2.7 tag and merged into `georchestra-gn4.2.x`.
All conflicts files have been resolved with taking gn changes and not grochestra's one.
Then changes have been reimplemented.

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
49 changes: 49 additions & 0 deletions georchestra-migration/upgrade_geonetwork.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# Upgrading geonetwork

those are generally the steps to check/remember to do when upgrading
geonetwork, be it for a major or a minor version, eg more or less applies for
3.8->4.0, 4.0->4.2, or 4.2.2->4.2.4.

## in the geonetwork data directory

* several files should be updated with the ones from the webapp:
* the configuration of the elasticsearch index
* the schemas
* the formatters used by the complete view
```
/srv/data/geonetwork# rm -Rf config/schema_plugins data/formatter
/srv/data/geonetwork# cp -r /srv/tomcat/georchestra/webapps/geonetwork/WEB-INF/data/config/schema_plugins config/
/srv/data/geonetwork# cp -r /srv/tomcat/georchestra/webapps/geonetwork/WEB-INF/data/config/index config/
/srv/data/geonetwork# cp -r /srv/tomcat/georchestra/webapps/geonetwork/WEB-INF/data/data/formatter data/
```
* make sure those files can be overwritten by the user running tomcat/the webapp
```
/srv/data/geonetwork# chown -R tomcat:tomcat data/formatter config/schema_plugins
```
* if you use the https://github.com/georchestra/geonetwork_minimal_datadir
repository, generally those changes are already available in the latest
branch of the repository
* drop the `wro4j-cache.mv.db` file (that's equivalent to the 'clear js & css
cache' button in the tools pane of the admin ui, but sometimes this page
might not be accessible when upgrading from a previous version)
* restart/reload the webapp
* check the logs, because if there are database schema changes to apply, those
should be done at the first start of the new version of the webapp

## in the geonetwork admin UI

* connected as admin, go to the *Tools* page in the *Admin console* menu
* hit *delete index and reindex* button. This will reset the elasticsearch
index, update its configuration (the `records.json` file in the
`config/index/` subdirectory of the geonetwork data directory) and rebuild
the index
* go in the *user interface* pane of the *Settings* page in the *Admin console* menu
* hit the *Json form* button and *Current configuration*. This will show you
what differs from the default configuration provided by the project.
* make a note of all those changes that were done previously, hit the 'Reset
configuration' button. This will make sure the UI configuration uses the
default provided by the georchestra project for its geonetwork fork.
* then, via the *Configuration form* tab, you can report the modifications you
previously did in the UI config to your own taste.
* make sure that facets in search results do still work, and that you still have
all your metadata :)

0 comments on commit 2dad0f8

Please sign in to comment.