Our methodology
How we calculate vaccine impact
Full model runs
Gavi WUENIC UNWPP
Modelling Groups
Coverage & Demography
VIMC Secretariat
Standardised Model Inputs
Disease burden estimates (for different coverage scenarios)
VIMC Secretariat
Gavi BMGF
Vaccine Impact estimates
Figure: Flow diagram of “full model runs” in VIMC.
The VIMC secretariat and modelling groups work together to supply a “full model update” every two or years, the next update will be in 2023/4. In this, each modelling group estimates the burden, measured through deaths, cases or DALYs, under a few vaccination scenarios. The vaccination scenarios and demographic data are prepared by the science and policy team in the secretariat. When the modellers have completed their estimates, they upload them so that the science and policy team can then calculate vaccine impact.
We calculate vaccine impact in a few ways or “views”. The views dictate how the impact is attributed, either to the calendar year where the difference in vaccine scenarios occurred, the year in which someone is born and the impact they experience over their lifetime, or the year in which the vaccination takes place. The method to calculate impact is slightly different between all these views, as is the interpretation, they are visualised in figure 2. One key similarity when calculating these impacts is that we compare the burden estimates under different vaccination scenarios. For example, we might compare the burden with and without vaccination in one calendar year and see what the difference in the number of deaths is. For birth cohorts, we compare the burden over their lifetime with and without vaccination to find the lifetime impact of that vaccine. Finally, to calculate impact by year of vaccination, we first calculate an impact ratio- this is the number of outcomes (like deaths) averted by one vaccine course. To calculate the impact ratio, we compare scenarios again to find the vaccine impact over the entire time period then divide this by the number of vaccine courses. Then we can multiply the impact ratio by the number of vaccine courses given in one year to see the impact of those activities. We stratify impact ratios by modelling group, vaccine, country and activity type and you can read more on the methods in Echeverria-Londono et al.
Figure 2: Visualisation of different impact views
Interim update
VIMC often needs to provide estimates more frequently than when the full model runs can take place- at least twice a year. This is usually as new vaccination data becomes available or there are new projections to analyse. In these cases, the science and policy team performs an “interim update” or extrapolation of what we think the impact would have been under the new vaccination scenario. In order to do this, we use the impact ratios calculated previously and apply them to new number of vaccine courses per year to get the new estimates of impact by year of vaccination. We tested this approach and found it works very well for projecting static models of disease dynamics, or projecting when the difference in coverage is small. This method struggles when we try to project the impact with very different coverage assumptions or for dynamic models of transmission – such as for measles where we might expect outbreaks to occur. As a result, we use this method carefully and acknowledge the limitations, to read more on the approach please see Echeverria-Londono et al.