Merge remote-tracking branch 'origin/main' into extra-markers

* origin/main:
  Upgrade minimal Cleo to 2.2.1
  make `allow-prereleases` a tri-state setting to really forbid pre-releases if the setting is `false` and keep default behavior to allow pre-releases only if necessary
  feat: add confirmation step
  feat(cli): add info messages about applied migration
  docs: add information about --migrate
  feat(cli): add support for --local for --migrate
  feat(cli): add --migration option to config command
  feat(config): add ConfigSourceMigration
  feat(config): provide method to remove empty config category
  feat(config): add get_property() to ConfigSource
  Ignore http credentials with empty usernames
  Fix regression when using empty username/password
  fix index error for yanked releases without dependencies
  ignore installed packages during solving
  vcs: use peeled ref when retrieving tag revision (#9849)
  chore: update json fixtures and improve generate script so that it produces the same results on Linux and Windows
  installer: add option to install without re-resolving (just by evaluating locked markers) (#9427) - introduce "installer.re-resolve" config option (default: True) - if the config option is set to False and the lock file is at least version 2.1, the installer will not re-resolve but evaluate locked markers
  locker: lock transitive marker and groups for each package (#9427)
  fix: do not ignore local config for implicit PyPI source (#9816)
  Cleanup, linting, typing (#9839)

# Conflicts:
#	src/poetry/installation/installer.py
#	src/poetry/puzzle/solver.py

.pre-commit-config.yaml

@@ -27,7 +27,7 @@ repos:
       - id: validate_manifest
 
   - repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: v0.5.4
+    rev: v0.7.3
     hooks:
       - id: ruff
       - id: ruff-format

docs/cli.md

@@ -630,6 +630,7 @@ Without `--` this command will fail if `${GITLAB_JOB_TOKEN}` starts with a hyphe
 * `--unset`: Remove the configuration element named by `setting-key`.
 * `--list`: Show the list of current config variables.
 * `--local`: Set/Get settings that are specific to a project (in the local configuration file `poetry.toml`).
+* `--migrate`: Migrate outdated configuration settings.
 
 ## run
 

docs/configuration.md

@@ -114,6 +114,21 @@ This also works for secret settings, like credentials:
 export POETRY_HTTP_BASIC_MY_REPOSITORY_PASSWORD=secret
 ```
 
+## Migrate outdated configs
+
+If poetry renames or remove config options it might be necessary to migrate explicit set options. This is possible
+by running:
+
+```bash
+poetry config --migrate
+```
+
+If you need to migrate a local config run:
+
+```bash
+poetry config --migrate --local
+```
+
 ## Default Directories
 
 Poetry uses the following default directories:
@@ -269,6 +284,21 @@ Use parallel execution when using the new (`>=1.1.0`) installer.
 Set the maximum number of retries in an unstable network.
 This setting has no effect if the server does not support HTTP range requests.
 
+### `installer.re-resolve`
+
+**Type**: `boolean`
+
+**Default**: `true`
+
+**Environment Variable**: `POETRY_INSTALLER_RE_RESOLVE`
+
+*Introduced in 2.0.0*
+
+If the config option is _not_ set and the lock file is at least version 2.1
+(created by Poetry 2.0 or above), the installer will not re-resolve dependencies
+but evaluate the locked markers to decide which of the locked dependencies have to
+be installed into the target environment.
+
 ### `solver.lazy-wheel`
 
 **Type**: `boolean`

docs/dependency-specification.md

@@ -601,3 +601,17 @@ markers = "platform_python_implementation == 'CPython'"
 The same information is still present, and ends up providing the exact
 same specification. It's simply split into multiple, slightly more readable,
 lines.
+
+### Handling of pre-releases
+
+Per default, Poetry will prefer stable releases and only choose a pre-release
+if no stable release satisfies a version constraint. In some cases, this may result in
+a solution containing pre-releases even if another solution without pre-releases exists.
+
+If you want to disallow pre-releases for a specific dependency,
+you can set `allow-prereleases` to `false`. In this case, dependency resolution will
+fail if there is no solution without choosing a pre-release.
+
+If you want to prefer the latest version of a dependency even if it is a pre-release,
+you can set `allow-prereleases` to `true` so that Poetry makes no distinction
+between stable and pre-release versions during dependency resolution.

docs/repositories.md

@@ -511,6 +511,17 @@ You can prevent this by adding double dashes to prevent any following argument f
 poetry config -- http-basic.pypi myUsername -myPasswordStartingWithDash
 ```
 
+{{% note %}}
+In some cases like that of [Gemfury](https://gemfury.com/help/errors/repo-url-password/) repositories, it might be
+required to set an empty password. This is supported by Poetry.
+
+```bash
+poetry config http-basic.foo <TOKEN> ""
+```
+
+**Note:** Usernames cannot be empty.  Attempting to use an empty username can result in an unpredictable failure.
+{{% /note %}}
+
 ## Certificates
 
 ### Custom certificate authority and mutual TLS authentication

pyproject.toml

@@ -34,7 +34,7 @@ python = "^3.9"
 poetry-core = { git = "https://github.com/python-poetry/poetry-core.git", branch = "main" }
 build = "^1.2.1"
 cachecontrol = { version = "^0.14.0", extras = ["filecache"] }
-cleo = "^2.1.0"
+cleo = "^2.2.1"
 dulwich = "^0.22.1"
 fastjsonschema = "^2.18.0"
 importlib-metadata = { version = ">=4.4", python = "<3.10" }
@@ -98,8 +98,6 @@ extend-exclude = [
     # External to the project's coding standards
     "tests/fixtures/git/*",
     "tests/fixtures/project_with_setup*/*",
-    "tests/masonry/builders/fixtures/pep_561_stub_only*/*",
-    "tests/utils/fixtures/setups/*",
 ]
 fix = true
 line-length = 88

src/poetry/config/config.py

@@ -123,6 +123,7 @@ class Config:
             "max-retries": 0,
         },
         "installer": {
+            "re-resolve": True,
             "parallel": True,
             "max-workers": None,
             "no-binary": None,
@@ -297,6 +298,7 @@ def _get_normalizer(name: str) -> Callable[[str], Any]:
             "virtualenvs.options.no-pip",
             "virtualenvs.options.system-site-packages",
             "virtualenvs.use-poetry-python",
+            "installer.re-resolve",
             "installer.parallel",
             "solver.lazy-wheel",
             "system-git-client",

src/poetry/config/config_source.py

@@ -1,13 +1,99 @@
 from __future__ import annotations
 
+import dataclasses
+import json
+
 from abc import ABC
 from abc import abstractmethod
+from typing import TYPE_CHECKING
 from typing import Any
 
+from cleo.io.null_io import NullIO
+
+
+if TYPE_CHECKING:
+    from cleo.io.io import IO
+
+
+UNSET = object()
+
+
+class PropertyNotFoundError(ValueError):
+    pass
+
 
 class ConfigSource(ABC):
+    @abstractmethod
+    def get_property(self, key: str) -> Any: ...
+
     @abstractmethod
     def add_property(self, key: str, value: Any) -> None: ...
 
     @abstractmethod
     def remove_property(self, key: str) -> None: ...
+
+
+@dataclasses.dataclass
+class ConfigSourceMigration:
+    old_key: str
+    new_key: str | None
+    value_migration: dict[Any, Any] = dataclasses.field(default_factory=dict)
+
+    def dry_run(self, config_source: ConfigSource, io: IO | None = None) -> bool:
+        io = io or NullIO()
+
+        try:
+            old_value = config_source.get_property(self.old_key)
+        except PropertyNotFoundError:
+            return False
+
+        new_value = (
+            self.value_migration[old_value] if self.value_migration else old_value
+        )
+
+        msg = f"<c1>{self.old_key}</c1> = <c2>{json.dumps(old_value)}</c2>"
+
+        if self.new_key is not None and new_value is not UNSET:
+            msg += f" -> <c1>{self.new_key}</c1> = <c2>{json.dumps(new_value)}</c2>"
+        elif self.new_key is None:
+            msg += " -> <c1>Removed from config</c1>"
+        elif self.new_key and new_value is UNSET:
+            msg += f" -> <c1>{self.new_key}</c1> = <c2>Not explicit set</c2>"
+
+        io.write_line(msg)
+
+        return True
+
+    def apply(self, config_source: ConfigSource) -> None:
+        try:
+            old_value = config_source.get_property(self.old_key)
+        except PropertyNotFoundError:
+            return
+
+        new_value = (
+            self.value_migration[old_value] if self.value_migration else old_value
+        )
+
+        config_source.remove_property(self.old_key)
+
+        if self.new_key is not None and new_value is not UNSET:
+            config_source.add_property(self.new_key, new_value)
+
+
+def drop_empty_config_category(
+    keys: list[str], config: dict[Any, Any]
+) -> dict[Any, Any]:
+    config_ = {}
+
+    for key, value in config.items():
+        if not keys or key != keys[0]:
+            config_[key] = value
+            continue
+        if keys and key == keys[0]:
+            if isinstance(value, dict):
+                value = drop_empty_config_category(keys[1:], value)
+
+            if value != {}:
+                config_[key] = value
+
+    return config_

src/poetry/config/dict_config_source.py

@@ -3,6 +3,7 @@
 from typing import Any
 
 from poetry.config.config_source import ConfigSource
+from poetry.config.config_source import PropertyNotFoundError
 
 
 class DictConfigSource(ConfigSource):
@@ -13,6 +14,19 @@ def __init__(self) -> None:
     def config(self) -> dict[str, Any]:
         return self._config
 
+    def get_property(self, key: str) -> Any:
+        keys = key.split(".")
+        config = self._config
+
+        for i, key in enumerate(keys):
+            if key not in config:
+                raise PropertyNotFoundError(f"Key {'.'.join(keys)} not in config")
+
+            if i == len(keys) - 1:
+                return config[key]
+
+            config = config[key]
+
     def add_property(self, key: str, value: Any) -> None:
         keys = key.split(".")
         config = self._config

src/poetry/config/file_config_source.py

@@ -8,6 +8,8 @@
 from tomlkit import table
 
 from poetry.config.config_source import ConfigSource
+from poetry.config.config_source import PropertyNotFoundError
+from poetry.config.config_source import drop_empty_config_category
 
 
 if TYPE_CHECKING:
@@ -30,6 +32,20 @@ def name(self) -> str:
     def file(self) -> TOMLFile:
         return self._file
 
+    def get_property(self, key: str) -> Any:
+        keys = key.split(".")
+
+        config = self.file.read() if self.file.exists() else {}
+
+        for i, key in enumerate(keys):
+            if key not in config:
+                raise PropertyNotFoundError(f"Key {'.'.join(keys)} not in config")
+
+            if i == len(keys) - 1:
+                return config[key]
+
+            config = config[key]
+
     def add_property(self, key: str, value: Any) -> None:
         with self.secure() as toml:
             config: dict[str, Any] = toml
@@ -62,6 +78,10 @@ def remove_property(self, key: str) -> None:
 
                 current_config = current_config[key]
 
+            current_config = drop_empty_config_category(keys=keys[:-1], config=config)
+            config.clear()
+            config.update(current_config)
+
     @contextmanager
     def secure(self) -> Iterator[TOMLDocument]:
         if self.file.exists():

src/poetry/console/application.py

@@ -16,7 +16,6 @@
 from cleo.events.event_dispatcher import EventDispatcher
 from cleo.exceptions import CleoError
 from cleo.formatters.style import Style
-from cleo.io.null_io import NullIO
 
 from poetry.__version__ import __version__
 from poetry.console.command_loader import CommandLoader
@@ -322,13 +321,10 @@ def configure_installer_for_command(command: InstallerCommand, io: IO) -> None:
         )
         command.set_installer(installer)
 
-    def _load_plugins(self, io: IO | None = None) -> None:
+    def _load_plugins(self, io: IO) -> None:
         if self._plugins_loaded:
             return
 
-        if io is None:
-            io = NullIO()
-
         self._disable_plugins = io.input.has_parameter_option("--no-plugins")
 
         if not self._disable_plugins:

src/poetry/console/commands/add.py

@@ -219,7 +219,7 @@ def handle(self) -> int:
 
         requirements = self._determine_requirements(
             packages,
-            allow_prereleases=self.option("allow-prereleases"),
+            allow_prereleases=self.option("allow-prereleases") or None,
             source=self.option("source"),
         )