7.0.0 (31 January 2023)

Announcement

At long last, the interminable wait is over. New Curator versions are being released. Yes, plural.
Curator versions 6, 7, and 8 are all coming out at once.

• This release is a simplified release for only pip and Docker. It only works
  with Elasticsearch 7.x and is functionally identical to 5.8.4

Breaking Changes

• Curator is now version locked. Curator v7.x will only work with Elasticsearch v7.x
• Going forward, Curator will only be released as a tarball via GitHub, as an sdist or
  wheel via pip on PyPI, and to Docker Hub. There will no longer be RPM, DEB, or Windows
  ZIP releases. I am sorry if this is inconvenient, but one of the reasons the development and
  release cycle was delayed so long is because of how painfully difficult it was to do releases.
• Curator will only work with Python 3.8+, and will more tightly follow the Python version releases.

New

• Python 3.11.1 is fully supported, and all versions of Python 3.8+ should be fully supported.
• Use hatch and hatchling for package building & publishing
• Because of hatch and pyproject.toml, the release version still only needs to be tracked
  in curator/_version.py.
• Maintain the barest setup.py for building a binary version of Curator for Docker using
  cx_Freeze.
• Remove setup.cfg, requirements.txt, MANIFEST.in, and other files as functionality
  is now handled by pyproject.toml and doing pip install . to grab dependencies and
  install them. YAY! Only one place to track dependencies now!!!
• Preliminarily updated the docs.
• Migrate towards pytest and away from nose tests.
• Scripts provided now that aid in producing and destroying Docker containers for testing. See
  docker_test/scripts/create.sh. To spin up a numbered version release of Elasticsearch, run
  docker_test/scripts/create.sh 7.17.8. It will download any necessary images, launch them,
  and tell you when it's ready, as well as provide REMOTE_ES_SERVER environment variables for
  testing the reindex action, e.g.
  REMOTE_ES_SERVER="172.16.0.1:9201" pytest --cov=curator. These tests are skipped
  if this value is not provided. To clean up afterwards, run docker_test/scripts/destroy.sh
• Add filter by size feature. #1612 (IndraGunawan)
• Update Elasticsearch client to 7.17.8

Security Fixes

• Use urllib3 1.26.5 or higher #1610 (tsaarni) — This dependency is now fully handled by the
  elasticsearch7 module and not a separate urllib3 import.

6.0.0 (31 January 2023)

Announcement

At long last, the interminable wait is over. New Curator versions are being released. Yes, plural.
Curator versions 6, 7, and 8 are all coming out at once.

• This release is a simplified release for only pip and Docker. It only works
  with Elasticsearch 6.x and is functionally identical to 5.8.4

Breaking Changes

• Curator is now version locked. Curator v6.x will only work with Elasticsearch v6.x
• Going forward, Curator will only be released as a tarball via GitHub, as an sdist or
  wheel via pip on PyPI, and to Docker Hub. There will no longer be RPM, DEB, or Windows
  ZIP releases. I am sorry if this is inconvenient, but one of the reasons the development and
  release cycle was delayed so long is because of how painfully difficult it was to do releases.
• Curator will only work with Python 3.8+, and will more tightly follow the Python version releases.

New

• Python 3.11.1 is fully supported, and all versions of Python 3.8+ should be fully supported.
• Use hatch and hatchling for package building & publishing
• Because of hatch and pyproject.toml, the release version still only needs to be tracked
  in curator/_version.py.
• Maintain the barest setup.py for building a binary version of Curator for Docker using
  cx_Freeze.
• Remove setup.cfg, requirements.txt, MANIFEST.in, and other files as functionality
  is now handled by pyproject.toml and doing pip install . to grab dependencies and
  install them. YAY! Only one place to track dependencies now!!!
• Preliminarily updated the docs.
• Migrate towards pytest and away from nose tests.
• Scripts provided now that aid in producing and destroying Docker containers for testing. See
  docker_test/scripts/create.sh. To spin up a numbered version release of Elasticsearch, run
  docker_test/scripts/create.sh 6.8.23. It will download any necessary images, launch them,
  and tell you when it's ready, as well as provide REMOTE_ES_SERVER environment variables for
  testing the reindex action, e.g.
  REMOTE_ES_SERVER="172.16.0.1:9201" pytest --cov=curator. These tests are skipped
  if this value is not provided. To clean up afterwards, run docker_test/scripts/destroy.sh
• Add filter by size feature. #1612 (IndraGunawan)
• Update Elasticsearch client to 6.8.2

Security Fixes

• Use urllib3 1.26.5 or higher #1610 (tsaarni) — This dependency is now fully handled by the
  elasticsearch7 module and not a separate urllib3 import.

5.8.4 (27 April 2021)

Announcement

• Because Python 2.7 has been EOL for over a year now, many projects are no
  longer supporting it. This will also be the case for Curator as its
  dependencies cease to support Python 2.7. With boto3 having announced it
  is ceasing support of Python 2.7, deprecated as of 15 Jan 2021, and fully
  unsupported on 15 Jul 2021, Curator will follow these same dates. This
  means that you will need to use an older version of Curator to continue
  using Python 2.7, or upgrade to Python 3.6 or greater.

Breaking

• Normally I would not include breaking changes, but users have asked for
  Click v7, which changes actions to require hyphens, and not underscores.
  Options can still have underscores, but actions can't--well, not strictly
  true. You can have underscores, but Click v7 will convert them to hyphens.
  This should only affect users of the Curator CLI, and not YAML file
  users, and only the actions: show-indices, show-snapshots,
  delete-indices, delete-snapshots. The actual actions are still named
  with underscores, and the code has been updated to work with the hyphenated
  action names.

New

• Now using elasticsearch-py version 7.12.0
• Adding testing for Python 3.9
• Removing testing on Python 3.6
• Tested Elasticsearch versions now include 7.12.0, 7.11.2, 7.10.2, 7.9.3,
  7.8.1, 6.8.15, 5.6.16
• Changing requirements.txt as follows:
  • boto3-1.17.57
  • certifi-2020.12.5
  • click-7.1.2
  • elasticsearch-7.12.0
  • pyyaml-5.4.1
  • requests-2.25.1
  • requests-aws4auth-1.0.1
  • six-1.15.0
  • urllib3-1.26.4
  • voluptuous-0.12.1

Bug Fixes

• Alias integration tests needed updating for newer versions of Elasticsearch
  that include ILM.
• Click 7.0 now reports an exit code of 1 for schema mismatches where it
  yielded a -1 in the past. Tests needed updating to correct for this.

Security

• Address multiple pyyaml vulnerabilities by bumping to version 5.4.1.
  Contributed in #1596 (tsaarni)

5.8.3 (25 November 2020)

New

• Determined to test the last 2 major version's final patch releases, plus
  the last 5 minor releases in the current major version. Travis CI testing
  needs to go faster, and this should suffice. For now, this means versions
  5.6.16, 6.8.13, 7.6.2, 7.7.1, 7.8.1, 7.9.3, and 7.10.0

Bug Fixes

• Caught a few stale merge failures, and asciidoc documentation problems
  which needed fixing in the 5.8 branch, which necessitate this tiny bump
  release. No code changes between 5.8.2 and 5.8.3.

5.8.2 (24 November 2020)

Announcement

• No, Curator isn't going away. But as you can tell, it's not as actively
  developed as it once was. I am gratified to find there are still users who
  make it a part of their workflow. I intend to continue development in my
  spare time. Curator is now a labor of love, not an engineering project I
  do during work hours.

New

• Testing changes. Only last ES version of 5.x and 6.x are tested, plus the
  releases of 7.x since 7.2.
• http_auth is now deprecated. You can continue to use it, but it will go
  away in the next major release. Moving forward, you should use username
  and password. This should work in curator, curator_cli, and
  es_repo_mgr.
• Removed tests for all 5.x branches of Elasticsearch but the final (5.6).
• Added tests for missing 7.x branches of Elasticsearch
• Remove tests for Python 3.5
• Fix hang of Shrink action in ES 7.x in #1528 (jclegras)
• Add ecs as a logformat option in #1529 (m1keil)

Bug Fixes

• Lots of code cleanup, trying to go PEP-8. All tests are still passing, and
  the APIs are not changed (yet—-that comes in the next major release).
• Dockerfile has been updated to produce a working version with Python 3.7
  and Curator 5.8.1
• Pin (for now) Elasticsearch Python module to 7.1.0. This will be updated
  when an updated release of the module fixes the cluster.state API call
  regression at elastic/elasticsearch-py#1141
• Fix client.tasks.get API call to be client.tasks.list when no index
  name is provided. See
  elastic/elasticsearch-py#1110
• Pin some pip versions to allow urllib3 and boto to coexist. See #1562
  (sethmlarson).

Documentation

• Add Freeze/Unfreeze documentation in #1497 (lucabelluccini)
• Update compatibility matrix in #1522 (jibsonline)

5.8.1 (25 September 2019)

Bug Fixes

• LD_LIBRARY_PATH will now be set in /usr/bin/curator and the
  associated scripts rather than set in /etc/ld.so.conf.d

Other

• Unsaved logging change in utils.py that got missed is merged.

5.8.0 (24 September 2019)

Some big changes here, with more coming soon.

New

• Require elasticsearch-py version 7.0.4
• Official support for Python 3.7 — In fact, the pre-built packages are built
  using Python 3.7 now.
• Packages bundle OpenSSL 1.1.1c, removing the need for system OpenSSL
• Certifi 2019.9.11 certificates included.
• New client configuration option: api_key - used in the X-Api-key header in
  requests to Elasticsearch when set, which may be required if ReadonlyREST
  plugin is configured to require api-key. Requested in #1409 (vetler)
• Add skip_flush option to the close action. This should be useful
  when trying to close indices with unassigned shards (e.g. before restore).
  Raised in #1412. (psypuff)
• Use RequestsHttpConnection class, which permits the use of
  HTTP_PROXY and HTTPS_PROXY environment variables. Raised in #510
  and addressed by #1259 (raynigon) in August of 2018. Subsequent changes,
  however, required some adaptation, and re-submission as a different PR.
  (untergeek)
• ignore_existing option added to CreateIndex. Will not raise an
  error if the index to be created already exists. Raised by (breml) in
  #1352. (untergeek)
• Add support for freeze and unfreeze indexes using curator. Requires
  Elasticsearch version 6.6 or greater with xpack enabled. Requested in issue
  #1399 and rasied in PR #1454. (junmuz)
• Allow the close action to ignore synced flush failures with the new
  ignore_sync_failures option. Raised in #1248. (untergeek)

Bug Fixes

• Fix kibana filter to match any and all indices starting with .kibana.
  This addresses #1363, and everyone else upgrading to Elasticsearch 7.x.
  Update documentation accordingly. (untergeek)
• Fix reindex post-action checks. When the filters do not return documents
  to be reindexed, the post-action check to ensure the target index exists
  is not needed. This new version will skip that validation if no documents
  are processed (issue #1170). (afharo)
• Prevent the empty filtertype from incorrectly matching against closed
  indices #1430 (heyitsmdr)
• Fix index_size function to be able to report either for either the
  total of all shards (default) or just primaries. Added as a keyword
  arg to preserve existing behavior. This was needed to fix sizing
  calculations for the Shrink action, which should only count primaries.
  Raised in #1429 (untergeek).
• Fix allow_ilm_indices to work with the rollover action. Reported in
  #1418 (untergeek)
• Update the client connection logic to be cleaner and log more verbosely in
  an attempt to address issues like #1418 and others like it more effectively
  as other failures have appeared to be client failures because the last
  log message were vague indications that a client connection was attempted.
  This is a step in the right direction, as it explicitly exits with a 1 exit
  code for different conditions now. (untergeek)
• Catch snapshots without a timestring in the name causing a logic error when
  using the count filter and use_age with source: name. Reported
  by (nerophon) in #1366. (untergeek)
• Ensure authentication (401), authorization (403), and other 400 errors are
  logged properly. Reported by (rfalke) in #1413. (untergeek)
• Fix crashes in restore of "large" number of indices reported by breml in
  #1360. (anandsinghkunwar)
• Do an empty list check before querying indices for field stats. Fixed by
  (CiXiHuo) in #1448.
• Fix "Correctly report task runtime in seconds" while reindexing. Reported
  by (jkelastic) in #1335

Documentation

• Grammar correction of ilm.asciidoc #1425 (SlavikCA)
• Updates to reflect changes to Elasticsearch 7 documentation #1426 and #1428
  (lcawl) and (jrodewig)

5.7.6 (6 May 2019)

Security Fix

Evidently, there were some upstream dependencies which required vulnerable
versions of urllib3 and requests. These have been addressed.

• CVE-2018-20060, CVE-2019-11324, CVE-2018-18074 are addressed by this
  update. Fixed in #1395 (cburgess)

Bug Fixes

• Allow aliases in Elasticsearch versions >= 6.5.0 to refer to more than one
  index, if is_write_index is present and one index has it set to True.
  Requested in #1342 (untergeek)

5.7.5 (26 April 2019)

This has to be a new record with 5 releases in 3 days, however, as a wonderful
aside, this release is the Curator Haiku release (if you don't know why, look
up the structure of a Haiku).

Bug Fix

• Persistent ILM filter error has finally been caught. Apparently, in Python,
  a list of lists [[]] will evaluate as existing, because it has one
  array element, even if that element is empty. So, this is my bad, but it is
  fixed now. (untergeek)

5.7.4 (25 April 2019)

Bug Fix

• ILM filter was reading from full index list, rather than the working list
  Reported in #1389 (untergeek)