Benchmarking
This section shows various benchmarks.
Runs of different durations
# benchmark for a typical daedalus run of 600 days
using Daedalus
using BenchmarkTools
infection_01 = Daedalus.DataLoader.get_pathogen("sars-cov-2 delta")
infection_01.r0 = 2.5
@benchmark daedalus("Australia", infection_01, time_end=600.0)BenchmarkTools.Trial: 132 samples with 1 evaluation per sample.
Range (min … max): 18.439 ms … 470.403 ms ┊ GC (min … max): 0.00% … 95.86%
Time (median): 23.583 ms ┊ GC (median): 0.00%
Time (mean ± σ): 38.041 ms ± 65.534 ms ┊ GC (mean ± σ): 38.95% ± 22.39%
█▅▂
████▄▇▆▁▇▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▄▄ ▄
18.4 ms Histogram: log(frequency) by time 457 ms <
Memory estimate: 41.81 MiB, allocs estimate: 106768.# benchmark for an RTM-exercise run of 100 days
using Daedalus
using BenchmarkTools
infection_02 = Daedalus.DataLoader.get_pathogen("sars-cov-2 delta")
infection_02.r0 = 2.5
@benchmark daedalus("Australia", infection_02, time_end=100.0)BenchmarkTools.Trial: 849 samples with 1 evaluation per sample.
Range (min … max): 3.958 ms … 452.513 ms ┊ GC (min … max): 0.00% … 99.03%
Time (median): 4.308 ms ┊ GC (median): 0.00%
Time (mean ± σ): 5.882 ms ± 15.762 ms ┊ GC (mean ± σ): 21.96% ± 16.52%
█▃
██▃▂▁▁▄▄▃▄▄▃▂▃▃▁▁▁▁▁▁▁▁▁▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▂▁▁▁▁▁▁▁▁▁▁▁▂▁▁▂▁▁▂ ▂
3.96 ms Histogram: frequency by time 20.4 ms <
Memory estimate: 8.62 MiB, allocs estimate: 19584.Effect of logging $R_t$
Logging $R_t$ in each timestep slows the simulation down by a lot.
using Daedalus
using BenchmarkTools
# turn off Rt logging and compare with benchmark above
infection_rt = Daedalus.DataLoader.get_pathogen("sars-cov-2 delta")
infection_rt.r0 = 2.5
@benchmark daedalus("Australia", infection_rt, log_rt=false, time_end=600.0)BenchmarkTools.Trial: 747 samples with 1 evaluation per sample.
Range (min … max): 4.097 ms … 455.221 ms ┊ GC (min … max): 0.00% … 98.98%
Time (median): 4.447 ms ┊ GC (median): 0.00%
Time (mean ± σ): 6.684 ms ± 23.199 ms ┊ GC (mean ± σ): 28.40% ± 15.74%
█ ▅▂ ▃▂
████▆██▆▄▄▁▁▁▁▁▁▄▄▁▁▁▄▄▄▁▁▁▁▁▄▁▁▁▁▁▁▁▁▁▁▄▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▄ ▇
4.1 ms Histogram: log(frequency) by time 38.3 ms <
Memory estimate: 8.23 MiB, allocs estimate: 18644.Reactive events
Reactive events currently require logging $R_t$ so are not benchmarked with logging turned off.
# benchmark for an RTM-exercise run with reactive NPI
using Daedalus
using BenchmarkTools
# Create a reactive NPI: reduce transmission when hospitalizations exceed 20,000
trigger_on = Daedalus.DaedalusStructs.ReactiveTrigger(20000.0, "H")
trigger_off = Daedalus.DaedalusStructs.ReactiveTrigger(1.0, "Rt")
effect = Daedalus.DaedalusStructs.ParamEffect(
:beta,
x -> x .* 0.3, # 70% reduction
x -> x ./ 0.3, # reset: divide by 0.3
trigger_on, trigger_off
)
npi = Daedalus.DaedalusStructs.Npi([effect])
infection_re = Daedalus.DataLoader.get_pathogen("sars-cov-2 delta")
infection_re.r0 = 5.0
@benchmark daedalus("Australia", infection_re, npi=npi, time_end=600.0)BenchmarkTools.Trial: 103 samples with 1 evaluation per sample.
Range (min … max): 23.082 ms … 541.529 ms ┊ GC (min … max): 0.00% … 95.40%
Time (median): 31.138 ms ┊ GC (median): 0.00%
Time (mean ± σ): 49.004 ms ± 82.178 ms ┊ GC (mean ± σ): 40.31% ± 23.25%
█▁
██▅▄▃▃▃▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▃ ▂
23.1 ms Histogram: frequency by time 501 ms <
Memory estimate: 56.36 MiB, allocs estimate: 127148.# shorter duration
infection_re2 = Daedalus.DataLoader.get_pathogen("sars-cov-2 delta")
infection_re2.r0 = 5.0
@benchmark daedalus("Australia", infection_re2, npi=npi, time_end=100.0)BenchmarkTools.Trial: 529 samples with 1 evaluation per sample.
Range (min … max): 5.594 ms … 437.935 ms ┊ GC (min … max): 0.00% … 98.60%
Time (median): 5.996 ms ┊ GC (median): 0.00%
Time (mean ± σ): 9.442 ms ± 26.853 ms ┊ GC (mean ± σ): 28.80% ± 16.73%
█ ▅▄
█▇▅███▆▆▅▄▁▆▆▅▁▁▁▄▅▁▄▁▁▁▅▁▁▁▄▁▁▁▁▁▁▁▁▁▁▁▁▅▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▅▄ ▆
5.59 ms Histogram: log(frequency) by time 46.5 ms <
Memory estimate: 12.20 MiB, allocs estimate: 28170.Timed events
# benchmark for a full run of 600 days
using Daedalus
using BenchmarkTools
# Create three-phase time-based NPI with different reductions
effects = [
Daedalus.DaedalusStructs.ParamEffect(
:beta, x -> x .* 0.7, x -> x ./ 0.7,
Daedalus.DaedalusStructs.TimeTrigger(60.0),
Daedalus.DaedalusStructs.TimeTrigger(120.0)
),
Daedalus.DaedalusStructs.ParamEffect(
:beta, x -> x .* 0.3, x -> x ./ 0.3,
Daedalus.DaedalusStructs.TimeTrigger(180.0),
Daedalus.DaedalusStructs.TimeTrigger(200.0)
),
Daedalus.DaedalusStructs.ParamEffect(
:beta, x -> x .* 0.5, x -> x ./ 0.5,
Daedalus.DaedalusStructs.TimeTrigger(300.0),
Daedalus.DaedalusStructs.TimeTrigger(365.0)
)
]
timed_npi = Daedalus.DaedalusStructs.Npi(effects)
infection_te = Daedalus.DataLoader.get_pathogen("sars-cov-2 delta")
infection_te.r0 = 3.0
@benchmark daedalus("Australia", infection_te, npi=timed_npi, time_end=600.0)BenchmarkTools.Trial: 136 samples with 1 evaluation per sample.
Range (min … max): 19.168 ms … 491.836 ms ┊ GC (min … max): 0.00% … 95.86%
Time (median): 23.464 ms ┊ GC (median): 0.00%
Time (mean ± σ): 37.996 ms ± 66.234 ms ┊ GC (mean ± σ): 38.25% ± 21.69%
█▆
██▆▇▆▄▆▇▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▄▄ ▄
19.2 ms Histogram: log(frequency) by time 463 ms <
Memory estimate: 42.71 MiB, allocs estimate: 110643.Timed events do not need $R_t$ logging and can be benchmarked without it.
infection_te2 = Daedalus.DataLoader.get_pathogen("sars-cov-2 delta")
infection_te2.r0 = 3.0
@benchmark daedalus("Australia", infection_te2, npi=timed_npi, time_end=600.0, log_rt=false)BenchmarkTools.Trial: 647 samples with 1 evaluation per sample.
Range (min … max): 4.895 ms … 433.873 ms ┊ GC (min … max): 0.00% … 98.73%
Time (median): 5.107 ms ┊ GC (median): 0.00%
Time (mean ± σ): 7.756 ms ± 24.173 ms ┊ GC (mean ± σ): 28.01% ± 15.87%
█ ▅ ▃▁
█▆█████▄▁▁▁▄▁▁▁▁▁▄▁▁▄▁▁▅▁▄▁▁▄▁▁▁▁▁▁▁▁▅▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▄▁▄ ▇
4.89 ms Histogram: log(frequency) by time 44.6 ms <
Memory estimate: 9.13 MiB, allocs estimate: 22566.