Feature: add rules engine #481

Guts · 2024-04-05T15:31:58Z

This PR adds a mechanism to allow QGIS profile editors to condition their deployment with a minimalist system of rules that can be edited in the profile.json file.

Featured changes

add a minimal rules engine
add a demonstration profile which is deployed only on Linux

Example

{
  "$schema": "https://raw.githubusercontent.com/Guts/qgis-deployment-cli/main/docs/schemas/profile/qgis_profile.json",
  "name": "QDT Only Linux",
  "folder_name": "qdt_only_linux",
  "description": "Demonstrating a QDT profile that's deployed only on Linux.",
  "author": "Julien Moura",
  "email": "infos+qdt@oslandia.com",
  "qgisMinimumVersion": "3.34.0",
  "qgisMaximumVersion": "3.99.10",
  "version": "1.0.0",
  "rules": [
    {
      "name": "Environment",
      "description": "Profile is configured to run only on Linux.",
      "conditions": {
        "all": [
          {
            "path": "$.environment.operating_system_code",
            "value": "linux",
            "operator": "equal"
          }
        ]
      }
    }
  ]
}

Other changes

It also initiates a change in the documentation organization to comply with https://documentation.divio.com/.

Related to

[Feature Request]: add conditions on profile deployment #343
Feature: retrieve win32 user groups #393 will add the ability to condition the deployment to Active Directory groups

codecov-commenter · 2024-04-05T15:34:42Z

Codecov Report

Attention: Patch coverage is 67.92453% with 17 lines in your changes are missing coverage. Please review.

Project coverage is 69.97%. Comparing base (3b1cd7a) to head (0c3bc85).
Report is 2 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #481      +/-   ##
==========================================
- Coverage   70.06%   69.97%   -0.10%     
==========================================
  Files          46       47       +1     
  Lines        2856     2904      +48     
  Branches      624      630       +6     
==========================================
+ Hits         2001     2032      +31     
- Misses        670      684      +14     
- Partials      185      188       +3

Flag	Coverage Δ
unittests	`69.49% <67.92%> (-0.09%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
..._deployment_toolbelt/utils/computer_environment.py	`100.00% <100.00%> (ø)`
qgis_deployment_toolbelt/profiles/qdt_profile.py	`78.88% <81.81%> (-0.43%)`	⬇️
...loyment_toolbelt/jobs/job_profiles_synchronizer.py	`42.85% <46.42%> (+1.02%)`	⬆️

florentfgrs · 2024-04-17T16:19:06Z

With these rules, is it possible to exclude one or more profiles? For example, I want to deploy all the profiles on my repo except foo and bar.

jmkerloch

Some remarks for documentation and use case. I hope I will find time for test this week.

docs/schemas/profile/qgis_profile.json

docs/schemas/profile/rules.json

qgis_deployment_toolbelt/jobs/job_profiles_synchronizer.py

jmkerloch

Operators in json schema must be updated

docs/schemas/profile/rules.json

jmkerloch

LGTM !

Very nice improvement. Can't wait to see new variable supported for rules !

sonarqubecloud · 2024-04-18T15:37:44Z

Quality Gate passed

Issues
4 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

Following #481, this PR fixes the case where a profile is not installed because of not matching defined rules but QDT is failing because of Splash Screen job tries to log a `profile.name` that does not exist in that case: ```python 2024-04-18 15:40:56||ERROR||check_path||check_path_exists||145||C:\Users\runneradmin\AppData\Roaming\QGIS\QGIS3\profiles\QDT Only Linux\QGIS\QGIS3.ini doesn't exist. Attempt 2/2. Game over. 2024-04-18 15:40:56||INFO||qgis_ini_handler||__init__||117||The specified file does not exist: C:\Users\runneradmin\AppData\Roaming\QGIS\QGIS3\profiles\QDT Only Linux\QGIS\QGIS3.ini. 2024-04-18 15:40:56||INFO||job_splash_screen||run||135||No profile.json found for profile 'C:\Users\runneradmin\.cache\qgis-deployment-toolbelt\repositories\qdt-demo-scenario\examples\profiles\only_linux' 2024-04-18 15:40:56||ERROR||bouncer||exit_cli_error||43||'NoneType' object has no attribute 'name' Traceback (most recent call last): File "d:\a\qgis-deployment-cli\qgis-deployment-cli\qgis_deployment_toolbelt\commands\deployment.py", line 205, in run job.run() File "d:\a\qgis-deployment-cli\qgis-deployment-cli\qgis_deployment_toolbelt\jobs\job_splash_screen.py", line 143, in run f"{profile_installed.name} ({profile_installed.path_in_qgis})" AttributeError: 'NoneType' object has no attribute 'name' 2024-04-18 15:40:56||ERROR||bouncer||exit_cli_error||44||Please, read the full detailed log: C:\Users\runneradmin\.cache\qgis-deployment-toolbelt\logs\QGISDeploymentToolbelt_0.33.0.log 'NoneType' object has no attribute 'name' 2024-04-18 15:40:57||WARNING||shortcuts||__init__||110||Executable does not exist: C:\Program Files\QGIS 3.34.4\bin\qgis-ltr-bin.exe. Shortcuts might not work. Check and fix your scenario. 2024-04-18 15:40:57||WARNING||shortcuts||__init__||110||Executable does not exist: C:\Program Files\QGIS 3.34.4\bin\qgis-ltr-bin.exe. Shortcuts might not work. Check and fix your scenario. 2024-04-18 15:40:57||WARNING||job_splash_screen||run||162||Profile qdt_demo -C:\Users\runneradmin\AppData\Roaming\QGIS\QGIS3\profiles\qdt_demo\images\splash.png dimensions (1000, 479) do not comply with dimensions recomended by QGIS for splash screen: 600x300. 2024-04-18 15:40:57||ERROR||check_path||check_path_exists||145||C:\Users\runneradmin\AppData\Roaming\QGIS\QGIS3\profiles\QDT Only Linux\QGIS\QGIS3.ini doesn't exist. Attempt 2/2. Game over. 2024-04-18 15:40:57||ERROR||bouncer||exit_cli_error||43||'NoneType' object has no attribute 'name' Traceback (most recent call last): File "d:\a\qgis-deployment-cli\qgis-deployment-cli\qgis_deployment_toolbelt\commands\deployment.py", line 205, in run job.run() File "d:\a\qgis-deployment-cli\qgis-deployment-cli\qgis_deployment_toolbelt\jobs\job_splash_screen.py", line 143, in run f"{profile_installed.name} ({profile_installed.path_in_qgis})" AttributeError: 'NoneType' object has no attribute 'name' 2024-04-18 15:40:57||ERROR||bouncer||exit_cli_error||44||Please, read the full detailed log: C:\Users\runneradmin\.cache\qgis-deployment-toolbelt\logs\QGISDeploymentToolbelt_0.33.0.log 'NoneType' object has no attribute 'name' ```

This PR adds date information to rules context allowing to condition a profile deployment based on current year, month or day. In the following example, the profile is installed only if QDT runs on a Linux operating system and if the current year >= 2023: ```json { "$schema": "https://raw.githubusercontent.com/Guts/qgis-deployment-cli/main/docs/schemas/profile/qgis_profile.json", "name": "QDT Only Linux", "folder_name": "qdt_only_linux", "description": "Demonstrating a QDT profile that's deployed only on Linux.", "author": "Julien Moura", "email": "infos+qdt@oslandia.com", "qgisMinimumVersion": "3.34.0", "qgisMaximumVersion": "3.99.10", "version": "1.0.0", "rules": [ { "name": "Environment", "description": "Profile is configured to run only on Linux.", "conditions": { "all": [ { "path": "$.environment.operating_system_code", "value": "linux", "operator": "equal" }, { "path": "$.date.current_year", "value": 2023, "operator": "greater_than_inclusive" } ] } } ] } ``` Depends on #481

…matically in jobs (#487) This PR is a following up of #481 and #486. After some real-life testing of the profile deployment rules, it became clear that the initial implementation was clumsy. By placing the filtering of profiles based on rules in the qprofiles-synchronizer job alone, we forget that other jobs based on the content of downloaded profiles also need to respect the rules: qplugins-synchronizer, etc. This PR fixes this by moving the rules filtering logic right into the generic job class and make sure that every job which relies on downloaded profiles use the same method to filter on rules.

This PR is a following up of #481. It's mainly a code structure improvement which aims to improve readability, testing and maintainability: - create an object - move it from utils to profiles since it's only related to QDT profiles management - add methods to export as dict and JSON - add unit test - use it in documentation and GenericJob.filter_on_rules Next (spoiler alert): a CLI command to generate the rules context to make the rules writing more handy

github-actions bot added documentation Improvements or additions to documentation enhancement New feature or request dependencies Project dependencies. jobs Scenarios and jobs quality Tests, project resiliency, etc. labels Apr 5, 2024

Guts force-pushed the feature/add-rules-engine branch from f6d6331 to 5ab62ad Compare April 5, 2024 20:27

Guts self-assigned this Apr 5, 2024

Guts requested a review from jmkerloch April 15, 2024 16:22

jmkerloch reviewed Apr 18, 2024

View reviewed changes

jmkerloch requested changes Apr 18, 2024

View reviewed changes

docs/schemas/profile/rules.json Show resolved Hide resolved

Guts mentioned this pull request Apr 18, 2024

Feature: add datetime to rules context #484

Merged

Guts requested a review from jmkerloch April 18, 2024 15:27

jmkerloch approved these changes Apr 18, 2024

View reviewed changes

Guts added 8 commits April 18, 2024 17:37

deps: add python-rule-engine for profiles conditional sync

cea9509

profile: add rules section to the model

04a712c

docs: add a sample profile with conditional deployment rule

59e6823

dev: simple script to tests rules engine

88b8879

feature: add profile's rules checks based on execution environment

736c7d1

fix: make sure that environment dict is cross-platform

4ee4352

fix: typo in JSON Schema description

f298c8e

fix: operators were based on JSON rules and not on Python implementation

9a8bb42

Guts force-pushed the feature/add-rules-engine branch from 0c3bc85 to 9a8bb42 Compare April 18, 2024 15:37

Guts enabled auto-merge April 18, 2024 15:37

Guts merged commit f8620f3 into main Apr 18, 2024
28 checks passed

Guts deleted the feature/add-rules-engine branch April 18, 2024 15:39

Guts mentioned this pull request Apr 18, 2024

fix(logs): unreachable profile-attributes in logs #486

Merged

Guts mentioned this pull request Apr 23, 2024

Refacto: move profiles rules filter to generic class and use it systematically in jobs #487

Merged

Guts mentioned this pull request Apr 23, 2024

Refacto: move rules context into a structured object #491

Merged

Guts linked an issue Apr 23, 2024 that may be closed by this pull request

[Feature Request]: add conditions on profile deployment #343

Closed

1 task

Guts mentioned this pull request Apr 23, 2024

feature(rules): add user name and groups to rules context #489

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature: add rules engine #481

Feature: add rules engine #481

Guts commented Apr 5, 2024 •

edited

Loading

codecov-commenter commented Apr 5, 2024 •

edited

Loading

florentfgrs commented Apr 17, 2024

jmkerloch left a comment

jmkerloch left a comment

jmkerloch left a comment

sonarqubecloud bot commented Apr 18, 2024

Feature: add rules engine #481

Feature: add rules engine #481

Conversation

Guts commented Apr 5, 2024 • edited Loading

Featured changes

Example

Other changes

Related to

codecov-commenter commented Apr 5, 2024 • edited Loading

Codecov Report

florentfgrs commented Apr 17, 2024

jmkerloch left a comment

Choose a reason for hiding this comment

jmkerloch left a comment

Choose a reason for hiding this comment

jmkerloch left a comment

Choose a reason for hiding this comment

sonarqubecloud bot commented Apr 18, 2024

Quality Gate passed

Guts commented Apr 5, 2024 •

edited

Loading

codecov-commenter commented Apr 5, 2024 •

edited

Loading