extend circleci orb support

[bob-rohan]

Tell us more.

Currently using circleci manager to match dockerfile and orb datasources. This will match executors defined as per https://github.com/tlbraams/renovate-circleci-executors/blob/main/.circleci/config.yml

executors:
  my-executor:
    docker:
      - image: cimg/ruby:3.0.3-browsers

However when CircleCI is configured via folders these dependencies are not identified.

.circleci
├── commands
│   ├── @common.yml
├── config.yml
├── config_default.yml
├── jobs
│   ├── @common.yml
└── workflows
    ├── test.yml
    └── validate.yml

Or folders in custom orbs

.
├── dist
│   └── orb.yml
├── renovate.json
├── src
│   ├── @orb.yml
│   ├── commands
│   │   └── compute.yml
│   ├── executors
│   │   ├── default.yml
│   │   ├── default_machine.yml
│   │   └── python.yml
│   └── jobs
│       ├── run.yml

In the above example, executors/default.yml

description: >
  default docker executor.
docker:
  - image: 'cimg/base:2023.08'
parameters:

PR is not identified for cimg/base:2023.08 by Renovate for above folder decompose CircleCI setup.

[bob-rohan]

Default circleci file match was updated during testing

  "circleci": {
    "fileMatch": [
      "(^|/)src/commands/.+\\.ya?ml$",
      "(^|/)src/executors/.+\\.ya?ml$",
      "(^|/)src/jobs/.+\\.ya?ml$",
      "(^|/)src/.+\\.ya?ml$",
      "(^|/)dist/.+\\.ya?ml$"
    ]
  },

[rarkins]

The default circleci fileMatch is this:

{
  "fileMatch": [
    "(^|/)\\.circleci/.+\\.ya?ml$"
  ]
}

If you are using custom file patterns then you need to extend it just like you seem to have discovered. The patterns you identify above don't seem to have any type of file naming convention which would be safe to use as a default fileMatch, so I don't see any change to be done here to Renovate's defaults

[bob-rohan]

The above folder naming are CircleCI dictated, not custom. They replace CircleCI defined yaml attributes they recognise, such as executors, with folders. It's a way to structure code with larger circleci config projects.

There are two ways these folders can be used, inside orb definitions, and outside orb definitions.

Inside orb definitions these folders are found under top level src folder. I can see how these names might overlap with non-circleci config were they to be added to the default file match for the circleci manager. However that was not the aim of my suggestion.

Outside orb definitions these folders would be found under top level .circleci folder. I think these would be fine to add to the default file match, but again, was not the aim of my suggestion.

I've extended the filematch to include these folders, as stated above, however the docker image dependencies found in executors are not included. To give feature parity with how these same dependencies are identified in a non-folder based CI setup would be a welcome extension.

Non-folder based setup - dependencies found by renovate

# .circleci/config.yml
executors:
  my-executor:
    docker:
      - image: cimg/ruby:3.0.3-browsers

Folder based setup - dependencies not found by renovate

# src/executors/my-executor.yml (orb definition)
# .circleci/executors/my-executor.yml (not orb)
docker:
  - image: cimg/ruby:3.0.3-browsers

[rarkins]

Does circleci have a reference for these additional file formats?

[bob-rohan]

https://github.com/CircleCI-Public/fyaml

https://circleci.com/docs/how-to-use-the-circleci-local-cli/

This CLI pack command (separate to circleci orb pack described above) allows you to create a single YAML file from several separate files (based on directory structure and file contents). The pack command implements FYAML
, a scheme for breaking YAML documents across files in a directory tree. This is useful for breaking up source code for large orbs and allows custom organization of your orbs' YAML configuration.

[rarkins]

If this allows any structure then I think it needs a custom manager