Allow to define custom presets in the global config for self-hosted users

[felipecrs]

Tell us more.

Currently, presets need to be defined as a JSON file in some repository or in some HTTP server.

I would like to propose an alternative: defining presets in the global config, which would be useful for self-hosted users.

For example, here's how a config.js could look like:

// config.js

customPresets: {
  groupAll: {
    packageRules: [
      {
        description: "Group all dependencies into a single PR",
        matchPackageNames: ["*"],
        groupName: "dependencies",
      },
    ];
  }
}

And here's how a repository's renovate.json could look like:

{
    "$schema": "https://docs.renovatebot.com/renovate-schema.json",
    "extends": [
        "config:recommended",
        "custom:groupAll"
    ]
}

Where custom: prefix would indicate when a preset should be loaded from customPresets, to avoid risking it overriding some built-in preset (even in future).

And I believe that would be a better approach for some users for two main reasons:

1. It would avoid Renovate from having to perform network requests to fetch the presets. They'd be readily available for reuse.
2. Those presets would be more easily testable, avoiding having commit and merge changes to the repository that hosts them.

What do you think?

Originally #40117.

[jamietanna]

Those presets would be more easily testable

Out of interest, why do you feel this would make them more testable? I'd see a "merge to the renovate-config repo" as a quicker process than i.e. "merge to the config.js and wait for our Renovate container to be deployed"

[felipecrs]

In our case, we have both the config.js and the presets in a single repository named renovate-configs.

We use Jenkins for running Renovate, which first clones our renovate-configs to fetch the pipeline definitions and the config.js.

We don't build the Renovate image with the config.js, instead we just mount it when running the image.

And we test changes by running Renovate locally. We just call the Renovate docker image the way the pipeline would.

Therefore, with that improvement we would be able to test changes to both config.js and presets without having to merge anything.

[jamietanna]

avoid Renovate from having to perform network requests to fetch the presets

Have you got any stats indicating that the HTTP lookups are causing a bottleneck for you? The Renovate cache should also stop them needing re-fetching too often

[felipecrs]

No, I have not collected stats to proof my hypothesis. And yes, Renovate already caches in memory the GET requests for the presets.

However, let's do a math:

1. We run Renovate for 200 repositories, one Renovate execution per repository.
2. Each have slightly different schedules, but let's consider they all run on every hour.
3. The preset GET requests are cached in memory, i.e. Renovate will still fetch the presets once per execution.
4. Each repository extends from 4~5 presets (counting recursive preset)

In this situation, Renovate will make at least 800 requests per hour for fetching these presets. It grows as we add more repositories.

[felipecrs]

Just FYI, I also considered suggesting allowing presets to be fetched from file:// URLs (similar to HTTP). But I have found the customPresets option to be nicer.

Because it is in the config.js, I can still read the presets from separate files before adding them to the customPresets object.

[viceice]

I would not block this suggestion

[felipecrs]

Thank you! Any major concerns with my suggested implementation? If not, maybe you can reopen that PR. But I can submit a new one too, if you prefer.

[jamietanna]

Given we'll soon have globalExtends supporting file:// (#39304) we could also do this for this suggestion?

[felipecrs]

Yes, that'd work too. But defining the preset itself in the global config has some advantages:

1. Easier code reuse and type-check. It's JavaScript instead of JSON.
2. Will not require renovate.json for repositories to have server-dependent file paths (e.g. "extends": ["file:///server/dependent/path/to/preset.json"]).
   If the file structure on server ever changes, it will not be necessary to refactor the renovate.jsons.

@jamietanna, maybe you can even consider my solution instead of #39304. It probably solves the issue you're trying to solve as well.

You can even use it to load external files if you wish:

// config.js
// ...
customPresets: {
  groupAll: JSON.parse(fs.readFileSync("/path/to/preset/groupAll.json"))
}
// ...

Just a thought, you guys decide.

[felipecrs]

@jamietanna, would you mind if I re-propose a PR like #40117?

[felipecrs]

I re-proposed this as:

• feat: allow to define custom presets in self-hosted config #44673