[FEATURE] Expose N-best hypotheses from offline transducer modified beam search

[mitsu-h]

Describe the feature

I would like sherpa-onnx to expose N-best hypotheses for offline transducer models when using modified_beam_search.

Currently, modified_beam_search keeps multiple hypotheses internally according to max_active_paths, but only the single best hypothesis is returned to the user.

A possible API would be to add an option such as:

num_return_paths=5

separately from:

max_active_paths=8

where:

• max_active_paths controls the search width
• num_return_paths controls how many final hypotheses are returned

The default could remain num_return_paths=1 to preserve the current behavior.

For backward compatibility, existing fields such as:

result.text
result.tokens
result.timestamps

could continue to represent the 1-best result, while N-best results could be exposed separately, for example:

result.hypotheses

with each hypothesis containing fields such as:

hyp.text
hyp.tokens
hyp.timestamps
hyp.ys_log_probs
hyp.score

The exact public API is open to discussion. In particular, I would appreciate guidance on the preferred representation across the different language bindings.

For the initial implementation, I suggest limiting the scope to:

• offline recognition
• transducer models
• modified_beam_search

This proposal is not intended to add a new lattice-based decoding algorithm such as icefall's fast_beam_search_nbest.

Motivation / Use Case

The main use case is downstream ASR post-processing where having several plausible hypotheses is significantly more useful than having only the 1-best result.

Examples include:

• ASR error correction
• named entity correction
• external LM / LLM reranking
• domain-specific post-processing

For example, the correct named entity may already exist as the 2nd or 3rd hypothesis even when the 1-best hypothesis is incorrect. If only the 1-best result is exposed, downstream correction systems cannot make use of that information.

This would be particularly useful for deployment scenarios where sherpa-onnx is used as the inference runtime, while additional lightweight correction or reranking is performed outside the ASR decoder.

Additional context

Looking at the current implementation, this seems potentially smaller than implementing a new N-best decoder.

OfflineTransducerModifiedBeamSearchDecoder::Decode() currently selects the final result with:

Hypothesis hyp = cur[i].GetMostProbable(true);

However, Hypotheses already implements:

std::vector<Hypothesis> GetTopK(int32_t k, bool length_norm) const;

so the decoder already has a mechanism for retrieving the top-K final hypotheses.

Also, Hypotheses::Add() merges hypotheses with identical token sequences using log-sum-exp, so these would naturally be N-best distinct token sequences rather than duplicate search paths.

Related issue:

• Availability of different beam search as icefall #465 — Availability of different beam search as icefall

That issue also mentions fast_beam_search_nbest, but this request is narrower: expose the final hypotheses already retained by modified_beam_search, without introducing a new decoding method or lattice dependency.

It may also be useful to expose the ranking score for each hypothesis. Since Hypothesis already contains log_prob, lm_log_prob, and TotalLogProb(), I would be interested in the maintainers' preference for the public score semantics, especially when LM or context-biasing scores are involved.

If this direction makes sense, I would be happy to work on an implementation.