Support passing parameters (`vars:`) to runbooks called from the `needs:` section

[k2tzumi]

Summary

I'd like to propose allowing parameters (e.g. vars:) to be passed to runbooks invoked from the needs: section, so that a single "fixture / provisioning" runbook can be reused across many scenarios with different inputs.

Currently needs: only accepts a key-to-path mapping:

needs:
  prebook: path/to/prebook.yml
  prebook2: path/to/prebook2.yml

There is no way to parameterize the called runbook, so variations of the same fixture have to be created as separate runbook files.

Motivation / Use case

I want to use needs: runbooks as common fixture / provisioning templates to minimize the amount of dependent data each scenario has to own. This is primarily for regression testing, where I don't want to keep increasing the number of test accounts.

Concretely, my goals are:

• Reduce total test execution time. Shared, expensive setup should run only once.
• Make dependencies easy to manage. The dependent data effectively becomes the identifier of the needs:, which is easier to reason about than duplicating similar data everywhere.
• Register SaaS provisioning data. Many features need to be verified; the scenarios themselves have few dependencies, but provisioning data is always required and is expensive to create (authentication & authorization data must be set up).
• Avoid mass-producing near-identical data/runbooks.

Why the existing options don't fully cover this

needs:

needs: already deduplicates the same runbook across scenarios (via lo.Uniq in sortWithNeeds), so "run the shared setup only once" is effectively achieved and I don't have to write any "already executed?" guard in my scenarios. This is exactly why needs: is attractive for shared/expensive provisioning.

However, because needs: takes only map[string]string (key → path), I cannot pass vars:. When I need data variations, I'm forced to duplicate the runbook, which leads to a lot of near-identical runbooks.

include:

include: does support vars:, but it runs every time and is not deduplicated. To reuse a shared fixture with include:, I'd have to implement an "is it already executed?" check inside my scenarios myself, which is exactly the boilerplate I want to avoid.

So there is a gap between the two mechanisms:

Need	needs:	include:
Run shared setup only once (dedup)	✅ automatic	❌ must guard manually
Pass parameters (vars:)	❌ not supported	✅ supported

The goal of this request is to close that gap: keep the "run once / declarative dependency" behavior of needs:, while allowing parameterization like include:.

Proposed idea

Allow a vars:-like input for needs: entries, e.g. (syntax is just for illustration):

needs:
  admin_account:
    path: path/to/provision_account.yml
    vars:
      role: admin
  member_account:
    path: path/to/provision_account.yml
    vars:
      role: member

Bound values would still be referenced via needs.<key>.* as today.

Points to discuss

I understand needs: and runbook-to-runbook interaction is a sensitive area (the Include + needs: combination in #1002 was reverted once and reworked in #1006, and #1017 addressed result overwriting), so I'd like to raise the following design points:

• Dedup key design: should two needs: entries that point to the same path but have different vars: be treated as distinct executions? (i.e. how should lo.Uniq in sortWithNeeds account for vars:?)
• Result structure: impact on needs.<key> results when the same path is run multiple times with different parameters.
• Backward compatibility: the existing needs: { key: path } shorthand should keep working unchanged.

[k1LoW]

Thank you for your suggestion.

needs:, as its name suggests, "requires (wait) the prior completion of a specified runbook," and it shouldn't have any influence on the preceding runbook.
The same design principle applies as GitHub Actions' needs: section.

Your proposal is affecting the behavior of the preceding runbook by vars.

[k2tzumi]

Thanks for the detailed and well-researched proposal — the comparison table between needs: and include: captures the gap precisely.

After digging into the code, I think the key insight is this: the difficulty isn't really about adding a vars: field — it's that vars: conflicts with the mental model needs: is built on.

Why extending needs: is hard

needs: is fundamentally a dependency-graph (DAG) model. Dedup is not decided at result-lookup time — it's decided before execution, when operator instances are shared by path in traverseOperators (opn.om[path]). resolveNeeds / sortWithNeeds then flatten that graph and lo.Uniq dedups by operator instance identity.

That path-as-identity assumption is exactly what vars: breaks, and it cascades into three coupled problems:

1. Dedup key: same path + different vars must become distinct nodes, so the identity key has to move from path to path + vars.
2. Result store: nm is keyed by path and written with TrySet (first-write-wins), so running the same path with different vars would collide.
3. Ordering: to build a dedup key before the topological sort, vars: must be known before execution. If vars: is allowed to be dynamic (evaluated at runtime), the key can't be computed at sort time at all — and since needs: runs before the caller, there's no point in time where the caller's dynamic values could even be applied.

In short: "parameterized" and "run-once dedup" are in tension inside the needs: (DAG) model.

A different mental model: load & share a Fixture (memoization)

I think this request actually wants a third mechanism, not an extension of needs:. The natural model is memoization of a fixture call:

fixture(path, vars) -> result
  - if (path, vars) was seen before  -> return the cached result (do not re-run)
  - if (path, vars) is new           -> run once, cache, and return

This is just a memoized pure-function call, and it dissolves every problem above:

• vars: is a first-class argument, not something fighting node identity.
• The cache key is naturally (path, vars) — exactly the "same path + same vars → reuse the stored result" behavior described in the proposal.
• "Which point-in-time vars:?" becomes well-defined: the value evaluated at call time.
• No collision, because it's a plain cache get/set instead of first-write-wins.
• It doesn't need to enter the sortWithNeeds / topological-sort world at all, which is where most of the coupling lives.

Implementation-wise, runn already has a global shared store (kv) that spans runbooks, which looks like a natural home for such a cache:

key := "fixture:" + path + hash(vars)
if v, ok := kv.Get(key); ok {
    return v            // cache hit -> shared, not re-run
}
v := run(path, vars)    // run once
kv.Set(key, v)
return v

This keeps it a separate layer from needs:'s nm (path-keyed, TrySet), so the two mechanisms don't interfere.

How this answers the three discussion points

• Dedup key design → the cache key is (path, vars). Same path + same vars is shared (run once, even across scenarios); same path + different vars is a distinct execution.
• Result structure → each (path, vars) gets its own cached result, still referenced via the local key (fixture.<key>.* / needs.<key>.*-style), so different parameterizations don't overwrite each other.
• Backward compatibility → this is a new mechanism, so the existing needs: { key: path } shorthand stays exactly as-is. (If desired, the shorthand-with-vars: sugar could later be layered on top of the same cache.)

Suggested framing

Rather than "needs: gains vars:", I'd suggest positioning this as three complementary tools:

Mechanism	Behavior
include:	run every time, parameterizable
needs:	declarative dependency, run once (dedup by path)
Fixture (new)	parameterizable and memoized/shared, dedup by (path, vars)

One thing worth deciding up front: should vars: be restricted to static values? Static values keep the cache key computable and the semantics clean; allowing runtime-evaluated vars: is where the "which point in time?" ambiguity creeps back in.

Curious what you (and @k1LoW) think about approaching it as a fixture/memoization mechanism rather than extending needs: directly.

[k1LoW]

@k2tzumi
Can your use case be solved using the following pattern?

# login.yml (parameterized template)
vars:
  role: null
steps:
  # login logic using vars.role

# admin.yml (thin wrapper)
steps:
  login:
    include:
      path: login.yml
      vars:
        role: admin

# member.yml (thin wrapper)
steps:
  login:
    include:
      path: login.yml
      vars:
        role: member

# viewer.yml (thin wrapper)
steps:
  login:
    include:
      path: login.yml
      vars:
        role: viewer

# scenario.yml
needs:
  admin: admin.yml
  member: member.yml
  viewer: viewer.yml

flowchart TD
    scenario["scenario.yml"]
    admin["admin.yml"]
    member["member.yml"]
    viewer["viewer.yml"]
    login["login.yml"]

    scenario -->|needs| admin
    scenario -->|needs| member
    scenario -->|needs| viewer

    admin -->|"include (vars: role=admin)"| login
    member -->|"include (vars: role=member)"| login
    viewer -->|"include (vars: role=viewer)"| login

Loading

[k2tzumi]

Thanks @k1LoW — yes, this pattern works and achieves the "run once via needs:" behavior I want, so functionally it solves the use case. 🙏

The remaining pain point for me is that it requires creating a dedicated wrapper file (admin.yml, member.yml, viewer.yml, …) for every parameter variation. Since these are fixtures, the set of "which dataset do I need" tends to grow per scenario, and I'd really like to be able to declare that dataset on the caller side rather than pre-materializing a file for each combination. As the number of roles/parameters/scenarios grows, the wrapper files multiply into a lot of near-identical YAML — which is exactly the duplication I was hoping to avoid.

So comparing the two directions:

• Wrapper + needs: (this comment): works today, no code changes needed, but requires one static file per variation and the caller can't declare the dataset inline.
• Fixture / memoization mechanism (my earlier comment): the caller passes vars: directly at call time, dedup is keyed by (path, vars), and no per-variation wrapper files are needed. It stays a separate layer from needs:'s path-keyed store, so it doesn't change needs:' semantics.

For my regression-testing use case (many roles/permission combinations, provisioning data defined close to the scenario), the caller-side declaration of the fixture + memoization approach fits much better and avoids the file explosion.

That said, this would mean introducing one more new concept alongside include: and needs:, so there's a trade-off in the extra learning and maintenance cost it brings. Given that cost, I'd love your honest take: do you think it's worth considering introducing a new concept like fixture / memoization at all? Or do you feel the current landing point — "no new mechanism, but N wrapper files" — is preferable?

[k1LoW]

@k2tzumi Thanks for the follow-up. Now I understand what you're really asking for. The wrapper approach is functionally equivalent to what you propose, so this reads to me as syntax sugar for reuse, not a missing capability. Since runn can already achieve what you want via wrappers, I am cautious about adding a new mechanism just to remove YAML boilerplate.

That said, from a design perspective, if we were to close this gap, it belongs on the include: side, not needs:.

• needs: declares "this runbook must have already run" (same principle as GitHub Actions needs:). The caller does not shape how the callee runs. Passing vars: to it breaks that principle, no matter what we name the mechanism.
• include: is already the "caller shapes callee" tool, and it already takes vars:. vars: values are evaluated at call time against the caller's store, so a dedup key of (path, evaluated_vars) would fit naturally on this side. A dedup attribute on include: (e.g. include.once: true) is a small, well-scoped extension of an existing concept, and I can see some standalone value there for include: itself.

My position has two parts.

1. Extending needs: for this is a no. That would break its semantics.
2. Extending include: with a dedup attribute is worth considering on its own merits, though I want to keep the bar high given the wrapper pattern already works.

Before deciding, I'd like to explore a few design points with you. On scope, I don't think there's much room for choice. The dedup should cover all scenarios in a single runn run, matching needs:. That leaves two open points.

1. Execution timing. include: runs at step time, so include.once would give you "run once across scenarios" but only when the step is reached. That misses the "provisioning finishes before all scenarios start" property of needs:. Does that gap matter for your regression-testing case?
2. Beyond fixtures. Do you see uses for include.once outside the shared-fixture pattern? A feature that only serves one use case is a signal to look for a smaller solution instead.

Curious how you think about each.