Research Funding as a Decision Problem Under Heavy-Tailed Uncertainty

Abstract

Heavy-tailed impact distributions, intrinsic uncertainty, and the high costs of proposal-based peer review increasingly challenge research funding decisions. Using large-scale bibliometric data, we show that past scientific performance provides statistically meaningful, though imperfect, information about future productivity and impact across multiple dimensions. An aggregated, percentile-normalised proxy signal captures this predictive structure robustly across research domains.

We analyse deterministic and stochastic funding allocation mechanisms under impact-based objectives and find that both converge to highly concentrated allocations that favour a small number of top-performing researchers. To address the limitations of pure exploitation, we introduce a biased lottery framework based on a regularised decision-theoretic objective that explicitly balances exploration and exploitation while accounting for practical funding constraints. Our results suggest that biased lottery mechanisms offer a transparent, efficient, and scalable alternative to conventional peer review in environments characterised by heavy-tailed scientific returns. Additionally, we provide a web application, available at http://scilottery.biocomputingunit.es, that implements the deterministic allocation method presented in this work.

1 Introduction

The allocation of public research funds is a central instrument of science policy, intended to maximise the long-term societal return of scientific activity. In most contemporary systems, this allocation relies predominantly on peer review, in which expert panels assess proposals and applicants to identify those most likely to produce impactful research. Despite its widespread adoption and clear merits, peer review operates under severe uncertainty: scientific outcomes are inherently unpredictable, and the eventual impact of funded research often differs substantially from expectations formed at the time of evaluation.

From a policy perspective, the goal of research funding is not merely to sustain activity, but to maximise societal return, broadly understood as the production of knowledge, methods, and discoveries that advance science. Although societal impact is difficult to measure directly and often unfolds over long time scales, bibliometric indicators are commonly used as practical proxies for scientific output and influence within defined fields and time windows (Wilsdon et al., 2015; Sinatra et al., 2016). Measures of productivity, citation impact, and the occurrence of highly influential publications capture complementary aspects of scientific performance and remain central—explicitly or implicitly—to many funding decisions.

A natural assumption underlying these practices is that researchers who have performed well in the recent past are, on average, more likely to perform well in the near future, although high-impact contributions can occur at any point in their scientific careers (Sinatra et al., 2016). This intuition motivates both performance-based funding schemes and peer-review judgments informed by publication records. However, it raises two closely related but unresolved questions. First, to what extent are bibliometric indicators computed over a recent period actually predictive of future performance over a comparable horizon? Second, if such predictability exists, how should it be used to allocate limited research funds in a setting where scientific impact is highly skewed, with rare and weakly predictable breakthroughs accounting for a disproportionate share of total influence? The first question has been examined in a growing body of work showing that scientific careers exhibit measurable persistence and cumulative advantage dynamics. Longitudinal analyses of publication and citation records reveal statistically significant temporal correlations between productivity and impact (Petersen et al., 2012), while large-scale studies of career trajectories demonstrate substantial heterogeneity yet identifiable regularities in performance over time (Way et al., 2017). At the same time, research on the predictability of scientific impact indicates that although early signals contain informative structure, individual breakthroughs remain only partially foreseeable due to intrinsic stochasticity and heavy-tailed citation distributions (Wang et al., 2013; Sinatra et al., 2016). These findings suggest that past performance provides a meaningful—but imperfect—basis for forecasting future outcomes, reinforcing the need to distinguish between statistical predictability at the aggregate level and the irreducible uncertainty surrounding exceptional contributions.

In this work, we address these questions by framing research funding as a decision problem under heavy-tailed uncertainty (Wang et al., 2013). We separate the empirical assessment of predictability from the normative issue of optimal allocation. Using large-scale bibliometric data, we quantify how past performance predicts future outcomes across multiple dimensions and examine the implications of this predictability for funding policies designed to maximize scientific impact. We show that impact-based optimization — whether implemented through deterministic rules or stochastic mechanisms — naturally yields highly concentrated allocations. Building on this result, we introduce a biased lottery framework derived from a regularized decision-theoretic objective that explicitly balances the exploitation of predictive signals with exploration under irreducible uncertainty.

This approach provides a transparent and scalable alternative to conventional proposal-based peer review (Fang and Casadevall, 2016; De Peuter and Conix, 2022), which becomes increasingly unreliable as funding rates decline. At such paylines, panels are effectively asked to make fine-grained distinctions among many highly meritorious proposals, despite strong evidence that reviewers have limited statistical power to accurately stratify applications within the top tier (Fang et al., 2016) or provide consistent scores (Marsh et al., 2008; Bornmann, 2011). In this regime, small random fluctuations in scoring and the influence of individual reviewers can determine outcomes (Kaplan et al., 2008), magnifying arbitrariness and exacerbating bias related to factors such as seniority, institutional prestige, gender, race, or field proximity (Fang and Casadevall, 2016; Guthrie et al., 2019).

2 Methods

We analyzed large-scale bibliometric data obtained from the OpenAlex database through its public API. The study focuses on individual researchers affiliated with scientifically homogeneous research institutes, selected to minimize heterogeneity in publication and citation practices across disciplines. This approach helps to standardize the analyses independently of the research topic, an alternative to the approach discussed in Radicchi et al. (2008). All analyses were conducted at the researcher level.

For each researcher, bibliometric indicators were computed over two consecutive, non-overlapping five-year windows: a reference period $[Y-5,Y-1]$ and a future period $[Y,Y+4]$. Within each window, we extracted three complementary indicators of scientific performance: productivity, measured as the number of publications; average citation impact, measured as citations per publication; and maximum citation impact, defined as the highest number of citations received by any single publication. Citations were counted only within the same temporal window as the corresponding publications. Researchers with fewer than two publications in the reference period were excluded to ensure minimally stable performance estimates.

To enable meaningful comparisons across researchers operating in related but non-identical scientific contexts, all indicators were normalised using within-institute percentile ranks. For each institute and time window, raw bibliometric measures were replaced by their empirical percentile ranks and rescaled to the interval $[0,1]$. This normalisation removes scale effects associated with field-specific publication volumes and citation dynamics, ensuring that all indicators are interpreted as relative standing within a homogeneous scientific context.

In addition to individual indicators, we defined an aggregated past-performance signal by averaging percentile-normalised productivity, the average citation impact, and the maximum citation impact over the reference period. This composite proxy provides a compact, field-normalised summary of recent scientific performance and serves as the primary predictive signal in subsequent analyses.

Predictability of future scientific performance was assessed by comparing percentile-normalised indicators measured in the reference period with their counterparts in the coming period. We combined exploratory visualisation, cross-period rank correlations, dimensionality reduction, and rank-based regression models to quantify performance stability over time. Rank-based methods were used throughout to focus on relative ordering rather than absolute magnitudes, ensuring robustness to heavy-tailed distributions and extreme observations and reflecting the comparative nature of evaluation and funding decisions. Full methodological details and additional analyses are provided in the Supplementary Material.

2.1 Deterministic allocation model

We introduce a deterministic funding allocation rule that combines exploitation of predictive performance signals with an explicit exploration component that accounts for irreducible uncertainty and heavy-tailed scientific returns.

Let $B$ denote the total available budget to be distributed among $N$ eligible researchers. Each researcher $i$ is assigned a non-negative performance score $s_{i}$, derived from reference-period indicators and predictive modelling, with larger values indicating higher expected future performance. An allocation $\mathbf{b}=(b_{1},\dots,b_{N})$ satisfies $\sum_{i}b_{i}=B$ and $b_{i}\geq 0$.

The allocation rule can be interpreted as an approximation to a utility-maximisation problem that balances predictable returns with the option value of rare, high-impact outcomes under heavy-tailed uncertainty,

$$U(\mathbf{b})=(1-\alpha)\sum_{i=1}^{N}b_{i}s_{i}+\alpha\,\mathrm{Tail}(\mathbf{b}),$$

where $\alpha\in[0,1]$ controls the trade-off between exploitation of predictive signals and preservation of exploration. The first term corresponds to expected return under the predictive model, while the second captures the value of maintaining broad support to increase exposure to unpredictable but potentially transformative contributions. In the deterministic setting, this tail component is approximated through diversification rather than explicit stochastic modelling.

Operationally, the total budget $B$ is split into an exploration component $B_{\mathrm{explore}}=\alpha B$ and an exploitation component $B_{\mathrm{exploit}}=(1-\alpha)B$. The exploration budget is distributed broadly across researchers while allowing a controlled dependence on predicted performance,

$$b^{\mathrm{explore}}_{i}=B_{\mathrm{explore}}\left(\lambda\frac{1}{N}+(1-\lambda)\frac{s_{i}}{\sum_{j=1}^{N}s_{j}}\right),$$

where $\lambda\in[0,1]$ interpolates between a uniform allocation ($\lambda=1$) and one proportional to the performance scores $s_{i}$ ($\lambda=0$). Larger values of $\lambda$ preserve broad participation while retaining weak performance sensitivity.

In contrast, the exploitation budget is allocated by concentrating resources on higher-scoring researchers using a parametric concentration rule. In particular, the exploitation component takes the form

$$b_{i}^{\mathrm{exploit}}=B_{\mathrm{exploit}}\frac{s_{i}^{\gamma}}{\sum_{j=1}^{N}s_{j}^{\gamma}},$$

where $\gamma>0$ controls the degree of concentration, ranging from egalitarian ($\gamma<1$) to strongly concentrated allocations ($\gamma\gg 1$). The total allocation to each researcher is given by $b_{i}=b_{i}^{\mathrm{explore}}+b_{i}^{\mathrm{exploit}}$.

The resulting policy can be interpreted as a portfolio rule: a fraction $\alpha$ of the budget maintains exposure to unpredictable, high-impact outcomes, while the remaining fraction is invested based on predicted performance, with a tunable concentration. Optional lower and upper bounds on individual allocations can be imposed to reflect policy or operational constraints.

The allocation rule is governed by a small set of interpretable hyperparameters controlling exploration, uniformity, and concentration. These parameters can be fixed by policy design or optimised empirically via backtesting on historical data. Full mathematical details, alternative formulations, and implementation specifics are provided in the Supplementary Material.

2.2 Stochastic allocation model

The deterministic allocation rule provides a transparent baseline but treats uncertainty only implicitly. Because scientific returns are heavy-tailed and only partially predictable, rare and disproportionate contributions play a dominant role in aggregate impact. We therefore introduce a stochastic allocation mechanism that explicitly incorporates irreducible uncertainty through randomised selection, while remaining grounded in standard decision-theoretic principles.

Let $s_{i}\geq 0$ denote the predicted performance score of researcher $i$. In the stochastic framework, the decision variable is a probability distribution $p=(p_{1},\dots,p_{N})$ over researchers, where $p_{i}$ represents the probability that researcher $i$ is selected for funding. We consider a regularised expected-utility objective of the form

$$\max_{p}\;(1-\alpha)\sum_{i=1}^{N}p_{i}s_{i}-\alpha\,\tau\,\mathrm{KL}\!\left(p\,\|\,u\right),$$

subject to $\sum_{i}p_{i}=1$ and $p_{i}\geq 0$, where $u$ denotes the uniform distribution, $\mathrm{KL}(\cdot\|\cdot)$ is the Kullback–Leibler divergence, $\alpha\in[0,1]$ controls the exploitation–exploration trade-off, and $\tau>0$ is a temperature parameter. The first term favors concentration on high-scoring researchers, while the regularization term penalizes excessive concentration and preserves exposure to unpredictable outcomes.

This optimization admits a closed-form solution given by a Gibbs (softmax) distribution,

$$p_{i}=\frac{\exp\!\left(\dfrac{(1-\alpha)s_{i}}{\alpha\tau}\right)}{\sum_{j=1}^{N}\exp\!\left(\dfrac{(1-\alpha)s_{j}}{\alpha\tau}\right)}.$$

As $\tau\to\infty$, the policy converges to uniform random selection, whereas as $\tau\to 0^{+}$ it approaches a purely exploitative rule concentrated on the highest-scoring researchers. Intermediate values yield a principled mixture of exploitation and exploration. Importantly, this lottery is not ad hoc: it arises as the unique optimizer of a regularized expected-utility problem closely related to maximum-entropy decision making (Ortega and Braun, 2013).

To reflect the discrete nature of real funding calls, exactly $K$ researchers are selected in each round by sampling without replacement according to the probabilities $\{p_{i}\}$. Following selection, funding is allocated in two stages: a fixed seed grant ensures a minimum level of support for each selected researcher. At the same time, the remaining budget is distributed conditionally on selection using a concentration rule analogous to the deterministic model. This separation preserves stochastic exploration at the selection stage while exploiting predictive information during allocation. Full mathematical details, implementation choices, and parameter selection procedures are provided in the Supplementary Material.

3 Results

Our empirical analysis is based on a large cohort of researchers affiliated with scientifically homogeneous research institutes, constructed to enable meaningful within-field comparisons. After applying inclusion criteria and ensuring minimal research activity, the final dataset comprises approximately 4,400 researchers across 20 institutions. Full details on data collection, cohort construction, and normalisation procedures are provided in the Supplementary Material.

Across all institutions and performance dimensions, scientific output and impact exhibit substantial heterogeneity and heavy-tailed behaviour. While a small fraction of researchers accounts for a disproportionate share of publications and citations, relative scientific performance displays partial persistence across consecutive five-year periods. In particular, productivity shows strong temporal stability, whereas citation-based indicators—especially average citation impact—are markedly more variable, reflecting the stochastic nature of scientific recognition.

Despite this variability, past performance contains statistically meaningful information about future outcomes. Rank-based analyses reveal consistently positive cross-period associations for productivity, citation impact, and exceptionally influential contributions, although the strength of the predictive relationships varies across indicators. Dimensionality reduction and collinearity analyses further show that most of the informative content of past performance can be captured by a low-dimensional representation, with sustained productivity emerging as the most stable and informative individual signal. Detailed exploratory analyses, correlation matrices, principal component decompositions, and regression results are reported in the Supplementary Material.

To obtain a compact and robust summary of past scientific performance, we define an aggregated proxy signal, avgPerc1, as the average of percentile-normalised productivity, average citation impact, and maximum citation impact measured in the reference period. This composite indicator integrates sustained output, typical influence, and exceptional contributions into a single, field-normalised score.

Figure 1 shows that this aggregated proxy signal is a strong predictor of future scientific performance across all outcome dimensions considered. Higher values of avgPerc1 are associated with systematically higher median and upper-quantile outcomes in future productivity, citation impact, and aggregated performance. At the same time, substantial dispersion persists at all levels of past performance, particularly for citation-based measures, indicating that even strong aggregation cannot eliminate irreducible uncertainty or heavy-tailed effects at the individual level.

Taken together, these results establish two key empirical facts that motivate the allocation models introduced below. First, recent scientific performance provides a statistically informative but imperfect signal of future outcomes. Second, exceptional contributions remain only weakly predictable, even when conditioning on strong past performance. Any funding mechanism aimed at maximising scientific impact must therefore balance exploitation of predictable signals with explicit accommodation of uncertainty and tail risk.

Figure 1: Smoothed joint distributions of percentile-normalised future outcomes as a function of aggregated past performance avgPerc1. Each panel shows a kernel-smoothed two-dimensional histogram for a different outcome in the future period: productivity, average citation impact, maximum citation impact, and the aggregated outcome avgPerc2. The concentration of probability mass along a monotonic diagonal in all panels indicates that avgPerc1 is a strong predictor across all dimensions of future performance, despite substantial dispersion reflecting irreducible uncertainty.

3.1 Optimisation of deterministic and stochastic allocation policies

We evaluated the performance of both deterministic and stochastic allocation rules by optimising their parameters against realised scientific outcomes in the future period. In both cases, the objective function was defined as

$$U=\sum_{i=1}^{N}b_{i}\,s_{i},$$

where $b_{i}$ denotes the fraction of the total budget allocated to researcher $i$ and $s_{i}$ is the percentile-normalised composite performance indicator in the future period (avgPerc2). The total budget was fixed to $B=1$, so that allocations can be interpreted as budget shares.

For the deterministic policy, we performed a grid search over the hyperparameters $(\alpha,\lambda,\gamma)$, which control, respectively, the fraction of the exploration budget, the uniformity of the exploration component, and the concentration of the exploitation component. Across the explored parameter ranges, optimisation consistently selected $\alpha=0$ and $\lambda=0$, corresponding to a purely exploitative policy without an explicit exploration component. Under this configuration, utility increased monotonically with the concentration parameter $\gamma$, reaching its maximum at the largest value considered. This indicates that greater resource allocation to high-scoring researchers yields higher realised utility when performance is evaluated solely on future impact.

Figure 2 illustrates the resulting deterministic allocation rule for $\alpha=\lambda=0$ and representative values of $\gamma$. As $\gamma$ increases, allocations become increasingly convex and concentrated, strongly favouring researchers with the highest predicted performance.

Figure 2: Deterministic allocation curves showing the fraction of the total budget $b_{i}$ assigned to each researcher as a function of the reference-period score $s_{i}$ for $\alpha=\lambda=0$ and different values of the concentration parameter $\gamma$. Increasing $\gamma$ leads to progressively more concentrated allocations, strongly favouring researchers with the highest predicted performance.

We conducted an analogous optimisation for the stochastic allocation mechanism by maximising expected utility, estimated by averaging outcomes over multiple independent realisations of the lottery. Despite its explicit treatment of uncertainty, the optimal stochastic policy also collapses to an extreme-concentration regime. The optimal configuration selects a tiny subset of researchers ($K=5$ out of approximately 4,400), assigns a minimal seed grant, and converges to $\alpha=0$, corresponding to a purely exploitative selection policy. Conditional on selection, the remaining budget is distributed relatively evenly among the selected researchers.

These results show that both deterministic and stochastic allocation mechanisms converge toward highly concentrated, exploitative solutions when evaluated solely on average future performance. This convergence is not an artefact of the allocation rule, but a direct consequence of the objective function: under heavy-tailed outcome distributions and partial predictability, maximising expected impact systematically favours concentrating resources on a small number of top-performing researchers. Exploration and diversification do not arise endogenously from impact-only optimisation, motivating their explicit inclusion as normative policy objectives rather than emergent outcomes.

4 Discussion

This study reveals a fundamental tension in research funding: future scientific performance exhibits substantial variability and heavy-tailed behaviour, yet past performance provides a statistically meaningful—though imperfect—predictive signal. Citation-based measures, in particular, display pronounced dispersion, reflecting the inherently stochastic nature of scientific influence. At the same time, researchers with stronger recent performance tend to achieve higher productivity and impact in subsequent periods. An aggregated proxy signal combining productivity, average citation impact, and exceptional contributions captures this predictive structure more robustly than any single indicator, providing a compact summary of expected future performance under heavy-tailed uncertainty.

A second central finding is that both deterministic and stochastic allocation mechanisms converge toward extreme concentration when evaluated solely on expected future impact. Regardless of whether exploration is implemented through deterministic spreading or randomised selection, impact-based optimisation systematically favours allocating resources to a small number of top-performing researchers. This outcome is not an artefact of the allocation rules, but a direct consequence of the objective function: under partial predictability and heavy-tailed returns, concentrating resources on established excellence maximises expected output. In fact, this tendency is consistent with our empirical observation that past performance is informative about future performance: researchers who rank highly in productivity and impact in one period tend, on average, to remain among the most productive in the next. At the same time, this impact-driven concentration should be interpreted in light of the broader science-policy debate reviewed by Aagaard et al. (2020), who synthesises both the proposed benefits of concentrating funding and the arguments for dispersal across a wider population of researchers. Importantly, these perspectives are not necessarily in opposition. The concentration that emerges from our framework is not concentration “per se”, but concentration in those researchers who have already demonstrated exceptional performance. Moreover, many of the arguments favouring dispersal in the literature are motivated by objectives that are not reducible to short-term bibliometric performance—such as diversity, renewal, resilience, fairness, and the mitigation of cumulative advantage—rather than by evidence that dispersed allocations strictly maximise expected impact. Where performance-based evidence is available, Aagaard et al. (2020) highlights diminishing marginal returns to additional funding beyond a certain point; however, the same literature also suggests that, before reaching that saturation point, additional resources can generate steep gains in output and influence. Taken together, these findings suggest that impact-only optimisation naturally drives concentrated allocations, while the case for dispersion typically rests on additional policy goals that must be introduced explicitly into the objective function rather than expected to arise endogenously from performance maximisation alone.

These results have direct implications for the design of research funding systems. Current peer-review–based selection mechanisms combine subjective assessments of past performance with speculative evaluations of proposed future work, under conditions of limited expertise, time pressure, and intrinsic uncertainty. Several studies have highlighted problems with the system (Marsh et al., 2008; Bornmann, 2011). Even in contexts where research has already been completed and rigorously reviewed, predicting impact is difficult; forecasting the success of a proposed study is even more uncertain. This structural limitation contributes to high evaluation costs (Gross and Bergstrom, 2019), limited transparency, and widespread dissatisfaction among applicants (Vaesen and Katzav, 2017; Schweiger et al., 2024).

Against this background, our findings support alternative funding models that treat uncertainty not as a flaw to be eliminated, but as a structural feature of curiosity-driven research that must be managed through institutional design. Biased lottery schemes based on transparent, interpretable indicators can provide a scalable way to balance the benefits of concentrating resources on strong candidates with the need to preserve diversity, renewal, and openness to serendipitous discoveries. By combining evidence-informed prioritisation with a controlled degree of randomness, such mechanisms align funding decisions with the explore–exploit trade-off: they exploit signals that correlate with future performance while maintaining support pathways for unconventional lines of work that cannot be reliably forecasted ex ante. At the same time, they respond to well-known systemic tensions in contemporary funding regimes—such as path dependence (the Matthew effect), disparities across regions and demographic groups, and the opportunity costs associated with proposal writing and grant administration—by making policy goals explicit and by enabling empirical comparison of alternative allocation models rather than assuming a single optimal scheme (Gigerenzer et al., 2025). Our framework occupies an intermediate position between fully proposal-based peer review and the fully egalitarian distribution model studied in Vaesen and Katzav (2017).

5 Conclusions

We examined research funding allocation from a data-driven, decision-theoretic perspective. Using large-scale bibliometric data, we showed that scientific performance exhibits substantial variability and heavy-tailed behaviour, yet retains statistically meaningful predictability across consecutive periods. An aggregated, percentile-normalised proxy signal captures this structure robustly and provides a compact summary of expected future performance across multiple dimensions.

When funding allocation is optimised solely for future impact, both deterministic and stochastic mechanisms converge toward highly concentrated solutions that favour a small number of top-performing researchers. This convergence reflects the combination of partial predictability and heavy-tailed returns. It highlights a fundamental limitation of impact-only optimisation: exploration, diversity, and generational renewal do not emerge endogenously and must be incorporated explicitly as policy objectives.

To address this limitation, we introduced a stochastic allocation framework based on biased lotteries derived from a regularised expected-utility principle. This approach provides a principled way to balance exploitation of predictive signals with exposure to irreducible uncertainty, while offering transparent control over concentration and exploration.

Overall, our results indicate that biased lottery mechanisms are not a relaxation of rigour, but a pragmatic response to uncertainty, heavy-tailed outcomes, and the costs of proposal-based peer review. By complementing or partially replacing subjective evaluation with transparent, data-informed selection rules, funding systems can improve scalability, reduce administrative burden, and allocate a greater share of resources directly to research activity.

See pages - of supplementaryMaterial.pdf