-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
76f1a18
commit 0a08bb6
Showing
3 changed files
with
114 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1 +1,62 @@ | ||
# fair-ds-oc1-docs | ||
# FAIR-DS Demonstrator: Copernicus Data SpaceMachine Learning - ECOMETRICS | ||
|
||
The ECOMETRICS app is a demonstrator for the FAIR-DS project. | ||
This document presents the software architecture. | ||
You can find more information about the app itself in the [FAIR-DS wiki](https://fair-ds4nfdi.github.io/wiki/). | ||
|
||
## Architecture | ||
|
||
There are four components to the ECOMETRICS app demonstrator: the Data Spaces, the Geo Engine instance, the machine learning in Jupyter and the ECOMETRICS dashboard. | ||
There are two kinds of users: a Data Scientist that trains and refines a model and a user that select area and time of interest and performs an analysis. | ||
The following diagram shows the architecture of the ECOMETRICS app. ![ECOMETRICS architecture](./assets/architecture.svg) | ||
|
||
```mermaid | ||
flowchart LR | ||
subgraph DataSpaces | ||
NFDI[NFDI 4 Biodiversity] --> Aruna | ||
Copernicus[Copernicus Data Space Ecosystem] --> Aruna | ||
end | ||
subgraph GeoEngine | ||
Aruna[Data Connectors - Aruna & STAC] --> ProcessingEngine[Processing Engine] | ||
ProcessingEngine --> API[API] | ||
end | ||
API --> Dashboards[ECOMETRICS Dashboards] | ||
API --> PythonLibrary[Python Library] | ||
User[User] -->|Select area/time and analyze| Dashboards | ||
DataScientist[Data Scientist] -->|Train and refine model| PythonLibrary | ||
PythonLibrary --> Jupyter[Jupyter Notebook] | ||
PythonLibrary --> ScikitLearn[Scikit-Learn] | ||
PythonLibrary --> ONNX[ONNX] | ||
Dashboards -->|Show insights| User | ||
``` | ||
|
||
### Data Spaces | ||
|
||
The demonstrator connects to the Copernicus Data Space and the NFDI4Biodiversity Data Space. | ||
The NFDI4Biodiversity provides training labels for the machine learning model. | ||
The Copernicus Data Space provides the satellite data. | ||
|
||
### Geo Engine | ||
|
||
Geo Engine has a data connector for each of the Data Spaces. | ||
The connectors are able to browser metadata and access the raw data for analysis. | ||
They map the files to the Geo Engine data model that supports temporal and spatial queries. | ||
The processing engine harmonizes and enriches the data and makes it ready for machine learning. | ||
Standardized and custom API methods make the data available. | ||
|
||
### Machine Learning in Jupyter | ||
|
||
The machine learning notebook access the Geo Engine API to retrieve the data. | ||
It uses Sci-kit Learn to train and refine a model. | ||
The model is stored in the ONNX format and uploaded to the Geo Engine where it is registered and can be used as an operator. | ||
|
||
### ECOMETRICS Dashboard | ||
|
||
The ECOMETRICS Dashboard is an easy-to-use web app that build upon the Geo Engine UI toolkit. | ||
The dashboard has an interactive map and lets the user select an area and time of interest. | ||
It has an analysis functionality that triggers the Geo Engine to run the machine learning model. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,51 @@ | ||
flowchart TB | ||
%% Data Spaces | ||
subgraph DataSpaces["Data Spaces"] | ||
nfdi["NFDI 4 BIODIVERSITY"] | ||
copernicus["Copernicus Data Space Ecosystem"] | ||
end | ||
|
||
%% Geo Engine Components | ||
subgraph GeoEngine["Geo Engine"] | ||
subgraph DataConnectors["Data Connectors"] | ||
aruna["ARUNA"] | ||
stac["STAC"] | ||
end | ||
processingEngine["Processing Engine"] | ||
api["API"] | ||
|
||
subgraph PythonLibrary["Python Library"] | ||
pythonLib["Python Library"] | ||
scikit["Scikit-learn"] | ||
onnx["ONNX"] | ||
jupyter["Jupyter"] | ||
end | ||
end | ||
|
||
%% ECOMETRICS Dashboards | ||
subgraph Dashboards["ECOMETRICS Dashboards"] | ||
dashboard["Dashboard"] | ||
end | ||
|
||
%% Actors | ||
user["User"] | ||
dataScientist["Data Scientist"] | ||
|
||
%% Connections within Geo Engine | ||
nfdi --> aruna | ||
copernicus --> stac | ||
aruna --> processingEngine | ||
stac --> processingEngine | ||
processingEngine --> api | ||
api --> dashboard | ||
api --> pythonLib | ||
pythonLib --> scikit | ||
pythonLib --> onnx | ||
pythonLib --> jupyter | ||
|
||
%% User Interactions | ||
user --> dashboard:::interaction["Select area/time and analyze"] | ||
dataScientist --> pythonLib:::interaction["Train and refine model"] | ||
|
||
%% Styling for better readability (optional) | ||
classDef interaction fill:#f9f,stroke:#333,stroke-width:2px; |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.