Off no longer leaves a free selectivity coefficient or a
scored prior. A selectivity linkage on a fleet whose Fleet_type is Off
now has its covariate coefficients held at their starting values, as that
fleet's base selectivity parameters already were, and a prior on one is
dropped with the same warning 5.64.5 gives an intercept prior. Before, the
coefficient was estimated with no data: free and flat with no prior, so the
Hessian was not positive definite, or set by its prior alone, adding a per-run
constant that made a drop-one-survey series' objectives incomparable. So a
linkage and its prior can stay in the spec while a fleet is switched off. A
row with no fleet (it reaches the live fleets) stays estimated. On an Off
member of a shared Selectivity_index block the prior moves, with a warning,
to the block's lead (its first fleet not Off), so switching off the lead
that carried the block's prior no longer refuses the fit; the coefficient
stays tied to the lead's where the lead has one. Fits with a selectivity
linkage on an Off fleet move by that prior's constant and lose the flat
parameter; no other fit changes. Not covered yet: a random-effect selectivity
linkage or a catchability linkage on an Off fleet, and an intercept that
carries only an init or bounds on a switched-off lead, which is still
refused.NEWS.md (CONTRIBUTING.md,
inst/RELEASE-CHECKLIST.md); 5.66.0 says so for linkage_spec(re_group =).?plot_index, the diagnostics vignette and the
5.64.3 entry now give the reason that holds: about half the observations
should fall below the median.NEWS.md lacked the blank line pandoc needs, so
the pkgdown changelog filed 5.64.1 under 5.64.2 (and five older versions
likewise).inst/RELEASE-CHECKLIST.md no longer says deep-checks can be dispatched
only on the default branch.linkage_spec() no longer has a re_group argument. It never did
anything, so a call that passes it now stops with unused argument (re_group = ...): delete it, and the fit is unchanged. A config saved with
it stops the same way in load_config(); delete its re_group: line. If you
set it expecting year-to-year deviations, your model had none. Write them in
the formula, one deviation per level of the variable after the bar:
~ temp + (1 | Year) (IID), ~ temp + rw(1 | Year) (random walk) or
~ temp + ar1(1 | Year) (AR1). Adding one changes the fit. The argument is
removed outright rather than deprecated, at the maintainer's direction; it is
a minor bump because it never changed a fit.materialize_linkage() reads as its steps. The 352-line function that
turns a linkage_spec() into linkage-table rows is now 66 lines over eight
helpers: the design matrix, the init / bounds / priors keys, the observe
checks, the strata, and the rows with their random-effect fields. The blocks
were moved, not rewritten; no output, error or package message changes (R's
own coercion warnings, from malformed input, now name the helper as their call).linkage.hpp defines
RCEATTLE_SEL_*, RCEATTLE_COMP_*, RCEATTLE_LINK_* and RCEATTLE_RE_*
after the tables in R/0-linkage_encode.R, and the accumulators, the
random-effect density, the offset calls and the prior lookup in
ceattle.cpp read those names instead of bare numbers.
test-linkage-encode.R checks each definition's value against R, and that
every selectivity and composition code has a consume site outside a //
comment. The linkage vignette now says strata$age_bin is a bin index
(age - minage + 1), as fit_mod() supplies it. No number moves.linkage.hpp's five rceattle_apply_*_linkages() functions read a
LinkageTable (process, parameter, strata, design column, link, design matrix
and coefficients), so nine of the eleven calls in ceattle.cpp fit on one
line (the two selectivity calls take three) and no function carries arguments
it ignores. No number moves.ceattle.cpp scores each group through rceattle_re_group_nll() (IID, random
walk or AR1) and gathers its deviations with rceattle_group_slots(), which
the linkage simulation in section 5.2 now shares. No number moves.ceattle.cpp section 2.3b gives the equation each link applies and defines
the row / slot / position / group terms the random-effect code uses;
linkage.hpp opens with a worked example;
vignette("environmental-linkages-and-priors") gains the equations under
"How the pieces fit together", with the stratum each applies to. Stale
declaration comments are corrected. Comments and documentation only; no
number moves.data_check() now requires a positive Time_varying_sel_sd on 2DAR1 and
3DAR1 fleets. Both forms score their deviation field at that sd under every
Time_varying_sel, but were exempt from the check, so a blank value on the
fleet leading a Selectivity_index built a model whose objective was NA
(and a 0 one whose objective was NaN) with no error naming the fleet or
column -- measured on GOA2018SS. Every fleet sharing the
Selectivity_index is asked, and so is a warm start, as for every other form.
Two of those did fit before and now stop: a mirror fleet with a blank sd under
random_sel = FALSE, whose sd is never read, and a warm start, which takes
the sd from inits. Enter any positive sd to refit either; neither reads it,
so the fit is unchanged. The bundled whamGrowthData ships 2DAR1 fleets with
sd 0 (and no N_sel_bins), so it never fitted as shipped and needs the
selectivity overrides its vignettes already make. It is a minor bump despite
the two setups that did fit, by the maintainer's decision, since any positive
sd restores them unchanged.unweighted_jnll_comp now drops the CAAL weight, and carries the initial
equilibrium catch. The copy that fills its unweighted rows overwrote the
CAAL row with the weighted value, so on the test fixture with
CAAL_weights = 0.5 it reported 2.957 where 5.913 was due, and it stopped
before the equilibrium-catch row, leaving that 0.
profile_components(weighted = FALSE), plot_profile(weighted = FALSE) and
the unweighted column of report_tables() read these rows. The objective
and every fit are unchanged. The documentation that called the index, catch
and penalty rows "structurally zero" there was also wrong: they always
equalled jnll_comp, and now say so.src/TMB/linkage.hpp and R/0-linkage_{encode,formula,table}.R go from
1,795 lines to 1,690 (comment lines 645 to 540); only comments change. The
linkage table's documentation now says re_index counts deviations across
both random-effect vectors, ?linkage_row documents its init_supplied
argument, and two internal helpers' documented arguments and return match the
code.src/TMB/ceattle.cpp (section 13) rewritten for
readability; no fit moves. About 2,020 lines become about 1,570, counting the
new header, and comment lines 666 become 309. Data likelihoods now come first
(13.1-13.6: index, catch, initial equilibrium catch, compositions, CAAL,
stomach contents), then priors and penalties (13.7-13.14); the random-effect
sd and rho priors and the random-effect density moved up from the REPORT
section. A new src/TMB/likelihood.hpp holds comp_row_nll() (shared by
compositions and CAAL), linkage_log_prior() (shared by the three linkage
prior loops) and fold_comp_tails(). Each simulation draw still sits beside
its density, in the same order.
Verified bit-identical to 5.64.5 (identical(), not a tolerance) at a
perturbed parameter vector for 2,196 of the 2,200 models the test suite
builds: the joint objective, the joint gradient over fixed and random effects,
every reported quantity, and a seeded simulate(). The other 4 differ only in
fT and consumption_at_age, which also differ between two runs of 5.64.5
because make_test_data() draws env_data unseeded.A pinned-intercept selectivity prior on an Off fleet is now dropped with a
warning instead of refusing the fit. (quiet_data_check = TRUE, as on a
refit, silences the warning.) Turning a survey off is an ordinary
sensitivity run and one prior spec is reused across the series, so refusing
the whole configuration blocked drop-one-survey runs outright. The prior is
removed rather than merely allowed: such a prior re-targets to a base
selectivity parameter the Off fleet does not estimate, and would still be
scored, adding a constant that differs per run and would make a drop-one
series' objectives incomparable. Reported by an AFSC GOA pollock user who had
replaced the refusal with a warning locally; that workaround keeps the
constant this drops, so such a user's objectives will shift.
A fit that previously errored now runs, provided the Off fleet keeps a
wired Selectivity form; no fit that worked before is affected. A workbook
that also blanks Selectivity still errors, on the unwired form rather than
on the prior. Three cases are deliberately left as they were: a slope row,
whose coefficient is estimated rather than re-targeted; apical, still
refused on an Off fleet because that parameter is freed rather than scored,
making it a flat direction and a non-positive-definite Hessian; and a fleet
sharing a Selectivity_index, where the shared block can be estimated even on
an Off member, so the prior is live and the existing refusal names the lead
fleet to move it to.
src/TMB/selectivity.hpp rewritten for readability; no fit moves. 738 lines
to 520 and comment lines 237 to 89. Each curve now writes one array, sel,
at x(bin), and the logistic forms read their slots through slope(k) and
inflection(k) rather than repeating the deviate and linkage-offset sum about
ten times. NonParametric, NonParametricIntegrable and its random walk share
one block, and the cases run in numeric order. Verified bit-identical (the
objective, the joint gradient over fixed and random effects, and the
selectivity, F, biomass and SSB reports) at 282 parameter vectors spanning
every form, age- and length-based, both sexes and each Time_varying_sel
mode the model accepts.The index and catch fit figures now draw the MEDIAN of the fitted
observation distribution, not its mean. The lognormal likelihood fits
log(obs) to a mean of log(hat) - bias_adjust_obs * sd^2 / 2, so
index_hat and catch_hat are that distribution's mean while the likelihood
is centred on its median, so about half the observations should fall below
the line. plot_index() and plot_catch() plotted the mean, raising the line
by exp(b * sd^2 / 2): 2% at an sd of 0.2, 13% at 0.5, 38% at 0.8. More of
the series then fell below it than above, and the fit read worse than it
was. Both now draw
hat * exp(-bias_adjust_obs * sd^2 / 2), using the sd each row was actually
fitted with, on the hindcast rows only. A projection row's prediction is the
removal taken, not an observation drawn with error, so it keeps the reported
value. plot_catch(mse = TRUE) is unchanged in full, on hindcast and
projection rows alike: run_mse() advances the operating model's endyr to
the last assessment year, so no year test separates the two there, and the
realized MSE catch is catch_hat, leaving nothing to convert.
bias_adjust_obs defaults to TRUE, so index and
catch fit figures from earlier versions draw the mean and need redrawing.
Natural-scale index fleets (MVN, MVNORM, Normal, TruncatedNormal) are
unchanged: their likelihood carries no offset, and their sd is absolute.
Nothing about the fit itself moves -- the offset was always in the likelihood;
only the figure changes.
plot_index_residual(residual_type = "pearson") now returns the Pearson
residual it claims. It computed log(obs) - log(hat) on a log-scale fleet
and obs - hat on a natural-scale one, neither standardized by the sd nor
offset to the likelihood's centre, so it disagreed with
residuals(fit, type = "pearson") on the same fit. It now matches that
method, which was already correct.
An MVN / MVNORM index fleet now standardizes by its covariance matrix,
not by the scalar sd its likelihood never reads. Those families score
0.5 r' Sigma^-1 r against index_cov and never read the scalar sd, which
stays at whatever the workbook carried -- so a fleet supplying index_cov in
natural units squared while leaving Log_sd at an old lognormal CV got a
Pearson residual wrong by the ratio of the two. residuals(type = "pearson")
and plot_index_residual() now divide by sqrt(diag(index_cov)), the
MARGINAL sd, and return NA on such a row where the matrix cannot be
resolved against the fitted years. plot_index()'s observation band for such
a fleet took the same wrong sd and is corrected with it: on one EBS pollock
configuration the drawn 95% half-width was +/-0.13 against a correct +/-1904,
so the figure asserted a survey biomass known to the tonne. Pearson
residuals, SDNR and the observation band change for covariance fleets; no
other family and no fit is affected -- report_tables() and
standard_output() republish that residual, so their fits$residual and
fits$sdnr move too. The marginal sd ignores the off-diagonal correlation
the likelihood does use (up to 0.54 on that same matrix), so these are
marginal residuals rather than fully whitened ones: a run of same-sign
residuals is expected under the model, and summing their squares understates
the fleet's likelihood contribution. osa_residuals() whitens.
data_check()'s warning for Estimate_index_sd = "Analytical" on a
covariance fleet is corrected with it. It told the user that
residuals(type = "pearson") and plot_index()'s observation interval
divide by the reported index_sd, which is no longer true of either, and
offered a remedy for a problem they no longer have. It now says that the
reported value is not the fleet's sd, names sqrt(diag(index_cov)) as the
one that is, and warns against reading fit$quantities$index_sd directly.
The trigger and the severity are unchanged. The y axis is therefore on a
different scale than before (standardized, labelled "Index Pearson
residual"), so these figures also need redrawing.
sim_mod() no longer invents an observation for a survey year the model
does not fit. The index likelihood scores a hindcast row only where
Observation > 0, so an NA there is unfitted — but both of sim_mod()'s
branches filled it anyway, by two different routes:
simulate = TRUE drew it. The lognormal, Normal and TruncatedNormal
draws filled every row, so a self_test() refit fitted survey years the
real data never had and recovery looked better than it was. On a 12-year
test model with year 5 set to NA, that row came back as 102.95 against a
prediction of 100.11.simulate = FALSE wrote the PREDICTION into it, which reaches
self_test(simulate = FALSE) and run_mse(regenerate_past = TRUE). That
is the same defect through a different door: a refit then fits the model's
own expectation as though it were data.Both now leave such a row as it arrived. Projection rows (Year <= 0, or
past endyr) are still filled by both branches, since run_mse() splices
the negative-Year rows in as each assessment's new data. The R-side predicate
asks the template's own question (Observation > 0 inside the hindcast
window) rather than is.na() alone, so the two cannot drift apart on a
non-positive observation.
sim_mod() also no longer warns that it "simulated an unusable survey
index" for such a row, nor for one on a fleet that is switched off or uses a
covariance family, none of which was ever drawn. If you grep sim_mod()
warnings, that message no longer appears for these rows.
Fits do not move. Simulations move only for a model that HAS an unfitted
hindcast index row: such a row no longer consumes random numbers, so later
draws for a given seed change too. Measured: 0 of the 11 bundled data lists
with an index_data have one, the Pacific hake MSE workbook has none (29
rows, 0 NA, 0 non-positive, 14 projection rows), and a sweep of 375 sibling
workbooks across Rceattle-models, GOA-multispecies-assessment,
GOA-ATF-ESP and GOA_circlulation_study found none either — so no bundled,
golden, hake-MSE or sibling-assessment draw changes.
A steep descending limb no longer rounds to zero selectivity at the oldest
ages. DoubleLogistic and DescendingLogistic computed the descending limb
as one minus a logistic, which loses relative precision as slope x
(age - a50) grows (4e-8 at 20, 1e-3 at 30) and is exactly 0 past about 37. It
is now computed as 1 / (1 + exp(slope * (age - a50))), the same curve at full
relative precision. With a slope of 8 and a50 of 8, selectivity at
age 13 was 0 and is now 4.2e-18; at age 12 it was 6e-4 off in relative terms.
log_sel_at_age (selectivity_se = TRUE) reported those ages at the 1e-300
floor (-690.8) and now reports their true log (-40 to -56), so their
standard errors there now come from the curve rather than the floor.
Fits move materially only where the descending limb is that steep within the
modelled ages, and only on its DESCENDING side: at a negative
slope x (age - a50) the limb is near 1 and both forms are exact, so what
matters is how far the argument reaches above zero. The golden references
barely do. Measured at the reference parameters, the largest positive
slope x (age - a50) is 0.83 on goa_ss and 1.16 on goa_ms across the three
fleets that use these forms (GOA2018SS fleets 1, 7 DescendingLogistic and 8
DoubleLogistic, all pollock, nages 10), where the old form's relative
error is at most one unit in the last place (2.2e-16). Hence goa_ss's
objective is unchanged to the last digit (single jnll_comp cells move by a
few units in the last place) and goa_ms moves by 1 unit in the last place
(1.8e-12). Bering Sea references use neither form.
jitter() refuses an unused argument instead of ignoring it. 5.62.0 gave
the function ... so a numeric first argument could reach base::jitter();
on the model path nothing read it, so a misspelled argument was dropped.
jitter(mod, njiter = 10) ran 50 jitters rather than the 10 asked for, and
reported convergence over that run. It
now errors and names the arguments the model path takes. The numeric path
still forwards ... to base, and jitter(x = 1:5) now says which argument
is unused rather than reporting the object's class.
This closes the model path only. The positional spelling is unchanged and
still drops silently: jitter(1:5, 2) binds 2 to njitter, returns at the
numeric fast path and hands base::jitter() an empty ..., so a base
caller's intended factor is lost. Naming base's arguments is the documented
route (?jitter), because the first two formals here are object and
njitter and cannot be reordered without breaking every existing call.
Refusing ... is a narrowing, so it was swept: of the 6 live jitter() call
sites in the sibling repositories (plus one commented out), 0 pass an
argument the model path does not take, so nothing shipping stops.
check_estimability named the wrong parameters. fit$convergence's
estimability record indexed BadParams$Param with WhichBad, which TMBhelper
sets to the positions of the near-zero Hessian eigenvalues -- used
elsewhere to subset eigenvector columns -- not to parameter positions. Since
eigen() sorts descending, the names were always the last k parameters
whatever they were. On the GOA cod bridge it reported five sel_dn6 entries,
all five of which TMBhelper itself marked "OK", while the six flagged
beta_linkage coefficients went unnamed. Neither number described
parameters: the count was of near-zero eigenvalues (5) printed under a
sentence that calls them parameters (6 were flagged), and the coordinate
summary was built from the same eigenvalue positions. It now reads
Param_check for all three.
Where near-zero eigenvalues exist but no single parameter loads on them, the
record stays a FAIL -- a singular direction is unidentified either way -- and
reports the count without naming parameters.
.check_estimability()'s own fallback used a different coding for the same
column, so the consumer handled one of the two shapes its producer emits,
and the one it handled was unreachable. The no-TMBhelper fallback wrote
Param_check as 1L/2L where TMBhelper writes "OK"/"Bad"; the
consumer tested == 2L, which is never true of TMBhelper's output. It now
emits "OK"/"Bad" so one shape leaves the function, and the consumer still
reads the retired coding so a saved identified does not go quiet.
Two knock-on effects, both on the no-TMBhelper path only. fit_mod() held a
fourth reader of this column that relabelled 1/2 to "OK"/"BAD"; those
two lines could never fire on TMBhelper's output and are now dead on both
producers, so they are gone. fit$identified$param_list therefore reads
"OK"/"Bad" rather than "OK"/"BAD" where the fallback ran.
vignette("developer-guide") points users at mod$identified, so the
spelling is visible; no number moves.
fit$.conv_hindcast's par, lower and upper were in a different order
from its gradient and index. They were built by looping
names(mapFactor) with a first-occurrence keep rule, while fit_mod() builds
the bounds it hands nlminb in map-level order and then reorders
blockwise into obj$par order. Two consequences: build_params() lists the
linkage coefficients after log_F and the template declares them before it,
so the blocks disagreed; and first-occurrence order swaps a block whose map
indices are shared out of order. Sharing alone is not enough -- on the GOA
cod bridge, whose Srv/Srv_ae1 pair shares Selectivity_index 4, no block
differs between the two rules, because those indices run ascending. The block
half is measured; the level half is latent, with no model on hand that
triggers it. The bounds check was correct throughout
-- the three vectors were mutually consistent, so parameters_outside_bounds
never misfired. Inside the package the block half degraded safely:
.conv_index_ok() tests the block sequence as well as the length, so a
mismatched order made .conv_index_for() return NULL and the bounds table
lost its coordinates rather than printing wrong ones. The level half passes
that guard and could attach the wrong coordinate. What actually misread was a
consumer-side positional join to fit$.conv_hindcast$par, which has no
such guard: on the GOA cod bridge log_growth_pars sat at 61:63 in
gradient and 195:197 in par, reading von Bertalanffy K as 0.0057 against
its true 0.2080. All three
now use fit_mod()'s own rule, and a block whose bounds are missing drops the
vectors rather than reporting a bound against its neighbour.
(Reported by a parallel session reviewing the SS3 bridge.)
retrospective() told a correct fit to refit itself when selectivity
deviations were random effects. Its map-drift probe rebuilt the comparison
map without random_sel, which build_map() hard-defaults to FALSE and,
unlike random_q, does not read off the data_list. So a
random_sel = TRUE fit, whose stored map frees sel_dev_log_sd, was compared
against a rebuild that maps it out. Measured on GOA2018SS with a
DoubleLogistic/RandomWalkAscending fleet: 49 blocks compared,
sel_dev_log_sd the single difference -- the same 49 that 5.54.1 reports as
"0 differ", so that figure was taken on a fit without random_sel. The probe
now passes the stored value. No number moves: the warning is advisory and each
peel's map is unchanged. One sibling script fits with random_sel = TRUE and
does not call retrospective(), so nothing shipping saw it.
The suitability-floor warning counted each bound cell once per hindcast
year, and could fire for a predator that never reached the floor.
suit_other holds one value per predator-sex-age, written to every year
(predation.hpp), so a 61-year hindcast reported 610 cells where 10 were
bound, and the lowest always resolved to styr. The count is now per
predator-sex-age and the year is gone from the message. The scan also skips a
predator with suitMode > 0: the floor is in
calculate_msvpa_suitability(), so a small suit_other under a parametric
suitMode is a fitted vulnerability rather than a bound, and naming it sent
the reader to diet_data for a code path that did not execute. An unreadable
suitMode warns rather than hides.
The M1_re refusal named the wrong set of supported models. It said
random effects were available for M1_model 0, 1 and 2, but 5.54.0 opened
M1_model = 3 to the by-year families. A user with an age-specific M who hit
the refusal was told to drop to M1_model = 1, discarding that level, when
M1_re = "iid_year" or "ar1_year" keeps it. Message only; the refused set
is unchanged.
The Time_varying_q refusal could not fire on a workbook that wrote the
integer codes. It tested Catchability against "AR1" and
"Environmental" before switch_check() canonicalises the column, so the
schema's own 5 and 6 fell through to the "Off" fill and died later on
missing value where TRUE/FALSE needed -- the failure the refusal was added
to replace. It now canonicalises through q_map first. Nothing shipping can
reach it: all 183 fleet_control sheets across the sibling repositories carry
a Time_varying_q column, and the refusal fires only when that column is
absent. One of those sheets does set the integer form
(BSAI yellowfin sole/Data/yfs_single_species_2022.xlsx, Estimate_q = 5),
so the canonicalised read is the one that matters there.
Two M1_base messages reported an array index as an age (rule 11). The
short-row warning now names ages on the species' own minage, and the
non-finite refusal says "age index", which is what it prints. minage = 0 is
live -- it is the SS3 converter's default -- so the old text named an age
those species do not have.
?build_selectivity, ?build_catchability and ?linkage_spec said
coefficients are per fleet. They are, by default -- but fleets sharing a
Selectivity_index or Catchability_index share one parameter block and so
take ONE coefficient between them, which is what 5.57.0 made true and is what
makes them mirror. All three @param entries now say so. The shared-block
rule was stated only in build_selectivity()'s priors paragraph, which a
reader asking about coefficients does not reach.
Two false column glosses in vignette("data-without-excel"), and a missing
one. fleet_control$Month = 0 was annotated "0 = mid-year". It is "not
specified", and that column reaches the model as flt_month, which is read
only for composition and weight-at-age under estimated growth -- not for a
survey index. The month the index predictor uses is index_data$Month
(index_n), which the vignette set to 0 with no annotation at all; it is now
annotated, because the template applies exp(-(month/12) * Z), so 0 is the
start of the year before any mortality and a mid-year survey needs 6. At
Z = 0.2 that is 9.5% in predicted numbers and at Z = 2.0 63.2%; it scales
with Z, so it moves trend rather than level and q cannot absorb it.
Bin_first_selected was annotated "youngest fully-selected bin"; selectivity
is 0 below that bin, which is the other end of the ascending limb.
The linkage grammar's time blocks now reach the vignettes. A new "Time
blocks" section of vignette("environmental-linkages-and-priors") gives the
~ cut(Year, breaks = ) form with a worked build_selectivity() example, the
treatment-contrast and ~ 0 + parameterizations, the design-column name that
init/bounds/priors must key, and why there is no block() helper
(rule 12 -- it would be a second grammar for what the formula already says).
The two soft-deprecation lists in
vignette("model-options-and-functionality") named (1 | Year),
rw(1 | Year) and ar1(1 | Year) but omitted a replacement for mode 3,
Block, leaving a time-blocked fleet with no migration target; both now link
to the new section. The grammar was not undocumented before -- it is a parenthetical in
?build_selectivity and a fuller aside in the developer-guide article -- but that article is
.Rbuildignored, so an installed package carried only the parenthetical.
Two traps the section now states. env_data short of endyr is the
dangerous case, not short of projyr: cut() keeps every level, so a block
with no rows gets an all-zero design column and an unidentified coefficient,
and R/0-build_linkage.R has no rank check. And the right-hand side is not
sealed to env_data -- model.matrix() falls back to the formula's
environment, so ~ temp * fleet with a fleet object in scope is accepted
and yields fleetB aligned positionally to model years rather than erroring.
Two template inputs were documented as ages; they are 0-based bins.
ceattle.cpp called bin_first_selected a "first selected age" and
sel_norm_bin1/2 an "age of max selectivity ... (NA = unused)".
rearrange_data() subtracts 1 and maps NA to -999 and a negative request
to -99 before the template sees either, and on a length-based fleet the
offset is 1 rather than minage, so these are bins and NA never arrives.
selectivity.hpp already said so; the two files disagreed. Converting one
back to an age with - minage gives a wrong selectivity curve, hence a wrong
q and index fit, with no error (rules 10, 11). The two sibling comments in
rearrange_data() itself are corrected with them.
Ptarget and Plimit were called a percentage; they are proportions.
?build_hcr's own @details says the 40-10 shape needs Ptarget = 0.40,
the defaults are 0.4 and 0.0, and the template computes
ref_SB0 * Ptarget(sp) -- but the @param entries and four DATA_VECTOR
comments said "percentage", where @param Ftarget two lines above already
had the right pattern ("For SPR F40%, enter 0.40"). Nothing validates the
range, so a user reading it literally and passing 40 gets a threshold of
40 x SB0: SSB is always below it, the ramp always engages, and every
Ptarget == 0.4 branch in mse_summary() silently falls through. Wording
only; no default or computation changes.
The apical-offset refusal's remedy was stated backwards in the vignette.
vignette("environmental-linkages-and-priors") told a reader refused on a
fleet that shares a Selectivity_index to "put the offset on the lead
fleet". The offset is applied per fleet (apical_offset(flt, sex, yr)), so
the group does not inherit one placed on the lead -- which the refusal in
build_selectivity() says, and which the same vignette says 44 lines
earlier. Following it gave an offset that moved the lead's curve alone while
the group reported one coefficient. It now gives the refusal's own remedy:
name every fleet in the group, or pass by = NULL. Introduced by a
documentation-condensing commit in this release range, so no released
version carried it.
Two guards that could not catch what they were written for.
test-docs-anchors.R allowed switch (sel_type) unconditionally, because the
developer guide quotes it as the name that is NOT the switch -- so a line
naming it as the dispatch escaped while the correct mention survived
elsewhere, which is the partial case that shipped for three releases. The
allowance is now conditional on every quoting line also carrying the
correction; verified by injecting that line and watching the guard fail.
test-plot-save-paths.R had a file-level skip_on_cran() over two blocks
that read only its own literals and getNamespaceExports(), so the whole
file reported zero result rows in both blocking jobs; the skip moved into the
one block that builds a fit, and 11 assertions now run under
NOT_CRAN=false.
test-predation-suitability-floor.R's skip_on_cran() moved from file level
into the five blocks that fit a model, so the four pure-R fixture blocks now
run under plain R CMD check. They are the only coverage the suitMode gate,
the per-cell count, the sex axis and the minage conversion have.
?plot_ssb_depletion now says what the denominator is, on the page this
release promotes to the canonical name. It is that year's DynamicSB0 under
DynamicHCR = TRUE; otherwise static SB0, except under HCR = 0 with
msmMode > 0, where it is female SSB in the last projection year. An
msmMode = 0 fit and an msmMode = 1, HCR = 0 fit on one axis are two
different ratios, which vignette("single-vs-multispecies") draws without
saying so. report_tables() already documents the multispecies denominator;
this page did not, and quantity_dictionary() still stops at ssb / SB0.
The src/TMB/ comments are shortened: comment words 23,766 to 22,949
(-817, -3.4%) and comment lines 2,717 to 2,607 (-110) across all twelve
translation units. This is a second pass over the surface 5.60.2 condensed;
the dead and duplicated material was already gone, so this one is prose
compression, not deletion. No code changed -- every translation unit
preprocesses byte-identically to the 5.63.1 base, each check run with a
control that detects a one-token mutation.
Nine wrong statements found and corrected while shortening. flt_type
was documented (0 = no, 1 = yes) for a switch with three values
(fleet_map: Off 0, Fishery 1, Survey 2); this file branches on == 1 for
fisheries at 13 sites and on > 0 for fishery-or-survey at 13 more, so a
reader who trusted the boolean reading would silently drop every survey.
pop_wt_index, ssb_wt_index, pop_age_transition_index and
flt_age_transition_index all claimed "Dim 3" where the code selects dim 1
-- flt_wt_index's own comment already had it right. est_index_q was
described as whether analytical q is used, over an eight-code q_map.
selectivity.hpp's case 9 attributed the form to RTMB "rpm", which names the
smooth shape-penalty mode; the form is ADMB AMAK "pm", as the same file's own
Doxygen said twelve lines earlier. "Code 12 is unused" now reads "Codes 10, 12
and 14", matching the registry. Two @param blocks called nages and
nlengths "max ages"/"max lengths" per species, which hard rule 11 exists to
prevent; they are bin counts. diet_ctl's gloss carried diet_obs's
stomach-weight parenthetical, and predation.hpp's "working sums per predator
age" were wrong on three of four dimensions.
Repeated wording unified. Five dimension arguments were documented 29
times in eight phrasings across the headers; they now use one verified form
each. comp_osa.hpp's four CDF helpers each re-derived the same keep-gating
contract, now stated once in its @file block; linkage.hpp repeated one
link_code/linkage_X/beta triplet five times, now stated once -- and the
single statement is wider than what it replaced, since the per-function copies
omitted link code 3, which is live for catchability.
Line width, stated because it is a visible change and not what was asked
for. Comment lines over 90 columns fall from 164 to 25 (the survivors are
three literature citations, two lookup legends, and one Richards formula kept
whole), but lines over 80 rise from 681 to 998: reflowing to a 90-column
allowance pulled the long tail in and pushed the middle out. Package median
width moves 76 to 77. comp_osa.hpp, comp_sim.hpp and spr.hpp have almost
no word reduction (21 words across the three) -- their gain was the hoist
above, and their line savings are mostly reflow.
Protected content was audited rather than assumed: the nine deliberately-kept
commented-out declarations and LEGACY markers are byte-identical and the six
documents citing them still read true; 29 year-citations with no delta; 93
numbered section headers unchanged with all 35 external citations resolving;
and @param names identical across all 34 documented functions.
R CMD check is back to one WARNING fewer, on all five runners. The deprecated aliases
plot_depletionSSB() (5.60.0) and plot_indexresidual() (5.62.0) take ... and nothing else,
and carried @inheritParams. That had nothing to inherit, because neither
plot_ssb_depletion() nor plot_index_residual() documents ... itself, so roxygen generated
both .Rd files with no \arguments section at all and checking Rd \usage sections reported
WARNING from 5.60.0 onward. Both now document ... explicitly. No change to either
function's behaviour, and the canonical names were never affected.Empirical suitability could put predation mortality at 1.3e+11 /yr, finite and unwarned. A
prey age whose average numbers approach zero while its diet proportion stays positive sends
stom_div_bio to infinity, so that one cell takes almost the whole normalized suitability
share; "other food" is 1 - sum(prey suitability), so it collapses, available food collapses
with it, and M2 = ration * N / avail_food diverges. The 1.3e+11 is the two-species test
fixture with its suitability window narrowed (other_food = 1e+05, avail_food 3.6e-06,
suit_other 3.6e-11); the bundled BS2017MS reaches M2 = 2.0e+04 /yr with suit_other
= 4.5e-10 at its own default cold start. No NaN and no error in either.
src/TMB/predation.hpp now holds the other-food term at a fixed fraction (1e-6) of the
modelled-prey term, so suit_other >= 1e-6/(1 + 1e-6) and the partition
sum(suitability) + suit_other = 1 stays exact. The floor is a unitless ratio, so it
needs no per-stock scaling.
Where it binds, measured on both multispecies reference models. At a converged fit it
is nowhere near: suit_other bottoms out at 2.9e-3 on BS2017MS and 5.3e-3 on goa_ms, the
tighter of the two 2900x above the floor. At a cold start it binds on both --
BS2017MS goes 4.5e-10 to 1e-6 and its max M2 2.0e+04 to 119; goa_ms likewise sits at the
floor, max M2 88.9. So /golden-check is bit-identical and structurally unable to catch
this: it evaluates at the stored MLE, while the optimizer starts where the floor binds.
Whose fits move. /golden-check's four reference models are bit-identical. The Pacific
hake MSE (MSE_yr2024.R, rule 15's predation check) moves: its three single-species fits
are bit-identical and its four multispecies fits move by 3.4e-09 to 4.3e-08 nats
(ms_run_DM_CSL +3.42e-09, run_ms_CSL_Mest_prior_DM_CSL +6.16e-09, ..._BH -4.27e-08,
..._stable +6.16e-09). That is the size of max2() not being bit-identical to the identity
in IEEE arithmetic, and comparable to those fits' own drift from the 5.33.0 published
references (up to 4.27e-08). Whether the floor binds in that model is unmeasured -- a
warning count cannot answer it, because the diagnostic reads the converged fit.
fit_mod() now says when that floor bound something. With the floor on, BS2017MS's cold
start still reaches M2 = 119 /yr, which is not a usable predation mortality. The warning names
the predator, the age and the cell count. (5.64.0 drops the year, which suit_other does not
have, and adds the sex on a two-sex predator.) It reads the converged fit, so it
reports a model that settled on the floor and cannot see the floor binding during
optimization -- which is the case that costs a gradient. Silent under msmMode = 0, where
suit_other is never written.
safebounds in deep-checks.yaml now runs on Windows as well as Linux. The 0xC0000005
access violation that job exists to attribute has only ever appeared on windows-latest, so
running the bounds-checked build on ubuntu alone could not reach it. fail-fast: false so a
lost runner on one OS does not cancel the other, and TESTTHAT_PARALLEL: false so a crash is
attributable to the file that caused it rather than to whichever parallel worker was holding
the corrupted heap. At this tag that arm measured nothing: verify-safebounds.R passed the
flag to its rebuild through system2(env=), which on Windows serves only a command taking
assignments on its command line, so the rebuild compiled nothing and the harness's positive
control refused the run. Fixed after the tag, in tools/verify/verify-safebounds.R; the Linux
arm was unaffected..par_description() removed and .is_timeout() given its callers. Both had no caller in
R/ and were kept alive only by their own tests. Nothing duplicated .par_description(), so it
is gone; .is_timeout()'s regex was a literal in three places, so it now takes a condition
or the messages a caller already holds and .report_errors() uses it twice -- three copies of
the pattern collapse into one constant.plot_indexresidual() is deprecated in favour of plot_index_residual(), the snake_case
spelling the rest of the package uses. The old name forwards and warns, so nothing stops
working; it draws the identical figure, which is pinned. Swept across the sibling repositories:
18 call sites, 0 of which break -- every one calls plot_indexresidual()
and now warns; the canonical name had no call sites there. A nineteenth hit is
prose listing the function name in a Quarto document, which cannot warn.jitter() no longer breaks base::jitter() for the spellings base callers use. Attaching
Rceattle masks base::jitter(), and because that function takes a fitted model under object,
jitter(1:5) previously errored with "Object is not of class 'Rceattle'". A numeric first
argument is now handed straight to base::jitter(), so jitter(1:5) and
jitter(1:5, factor = 2) return exactly what base returns (pinned by identical()).
Making jitter() an S3 generic was tried and does not work, which is worth recording: the
mask is caused by exporting the name, not by the function being non-generic, so S3 would not
remove it -- and UseMethod() errors before any method runs when the dispatch argument is
absent, so a generic on x could not also accept the model under object or Rceattle. The
live jitter(Rceattle = mod, ...) calls in Rceattle-models would have broken.
Two spellings still do not reach base, and are documented in ?jitter rather than fixed:
jitter(x = 1:5), because the first formal is object, and a positional second argument,
because the second formal is njitter. Name base's factor / amount, or call
base::jitter() directly.
The diet diagnostics refuse a fit that never predicted a diet, where they used to report a
complete misfit. A predicted diet needs estimated suitability and a predation model:
predict_stomach_content() reads diet_prop_hat, which only
calculate_msvpa_predation() writes, under msmMode > 0. So diet_hat keeps its setZero()
value both under suitMode = 0 with any msmMode and under msmMode = 0 with any suitMode,
and residuals(source = "diet") was handing those zeros back as fitted values -- every
observation became its own residual. Measured on the shipped BS2017MS (2025 diet rows):
suitMode = 0, 0, 0 with msmMode = 1, and suitMode = 4, 4, 4 with msmMode = 0, both gave
Fitted 0 .. 0 and Residual == Observed on 2025 of 2025 rows; suitMode = 4, 4, 4 with
msmMode = 1 gives 1050 non-zero and is unchanged. The existing is.null(diet_hat) test could
not see it, because the array is present and merely all zero.
residuals(source = "diet") now errors, naming both switches and the fit's own values.
plot_diet_comp(), plot_diet_comp1() and plot_diet_comp2() all read
residuals(source = "diet"), so all three inherit the refusal. plot_comp() only ever asks
for source = "comp" and is untouched. osa_residuals(source = "diet") already refused these
fits, by a different route (build_osa_data() skips a predator with suitMode <= 0), so the
two paths now agree on the refusal. They still differ on row coverage for a MIXED
suitMode: build_osa_data()'s skip is per predator, so on suitMode = c(0, 4, 4) it silently
omits predator 1's rows, while residuals(source = "diet") returns them -- correctly, since
under msmMode > 0 an empirical predator's diet_hat is a non-zero MSVPA reconstruction rather
than a zero. Six live hake call sites run that vector.
One sibling script is affected: Rceattle-models/Pacific hake/Dev/ATF_SBF_hake_test.R calls
plot_diet_comp2() on a msmMode = 1, suitMode = 0 fit and now errors -- its own comment
("diet estimate hake canni is good, in this model we do not have ATF predation, hence = 0")
is a reader taking the all-zero figure for a result. The other 14 of the 15 plot_diet_comp*
call sites under Pacific hake/ run
suitMode = c(0, 4, 4) and keep working.
A data_list carrying no suitMode is not refused: fit_mod() always writes it, so
absence means a hand-built list or a fit predating the field, and refusing there would break a
saved fit rather than catch a misreading.
The src/TMB/ comment surface is condensed and corrected: 3,130 comment
lines to 2,707 (-423, 13.5%) across all twelve translation units. No code
changed. Three independent checks agree, each with a positive control proving
it detects a one-token mutation: a string-literal-aware comment stripper,
clang -E -P output per translation unit, and a purpose-written C++ tokenizer
(56,498 tokens per side, identical SHA-256). /golden-check is not engaged,
because nothing the compiler sees moved.
Corrections a reader would have acted on. selectivity.hpp said every
selectivity form takes annual deviates and linkage offsets: the coefficient
forms (2, 5, 6, 7, 9, 13) take no offset -- build_selectivity() refuses one,
because mean-centring cancels it -- and DoubleNormalSS3 (15) has no deviate
array at all. growth.hpp's Richards within-year branch carried the von
Bertalanffy closed form, describing a different growth curve from the one the
line computes. Five DATA_IMATRIX column lists in ceattle.cpp named
Fishery_name / Survey_code, columns those matrices do not have. init_dev
was called a deviation throughout, but under initMode = 0 it is the log
initial numbers-at-age and no density scores it. recruitment.hpp described
srr_fun 1, 3 and 5 as carrying environmental effects; they are retired
aliases of 0, 2 and 4 that build_srr() refuses.
Two things the code does that nothing said. construct_Q's Omega is a
precision, not a covariance, and its rho_y / rho_a parameters act on the
opposite axes from their names -- the sole caller passes them in the
compensating order, so no fit is wrong, and both halves are now written down.
growth_plus_length = 4 advances a Richards species along the von Bertalanffy
increment.
Units and conventions supplied where they were missing: numbers-at-age in
thousands times weight-at-age in kg is metric tons, other_food is mt, and a
larger available-food denominator gives less predation mortality. nages is
a count of age bins rather than of age classes (rule 11), and selectivity.hpp
now records that its x variable is the 1-based bin ordinal with minage never
added.
What was deliberately left alone. Two LEGACY (scheduled removal: v4.5.0)
markers and the commented-out avgnMode / est_diet declarations stay: the
5.49.5 entry and inst/dev/REMOVED-kinzey-and-dev-cpp.md record them as
pending a decision rather than a sweep, and six live cross-references in
inst/dev/TRAPS.md, two tests and R/7-plot_ceattle.R read them. The numbered
section index and all 125 section numbers are untouched, since other files cite
them.
@param entries across five help pages had grown into mini-vignettes; each now gives
the argument's meaning, allowed values and default, with the rest under Details: param and joint in
profile.Rceattle(), weighted and relative in profile_components(), start in
self_test(), M1_re in build_M1() and integrate in linkage_spec(). The longest ran
18 lines. Nothing was rewritten -- the prose moved verbatim, so every claim it made is still
on the page, in Details rather than in the Arguments table. Allowed-value lists stayed in
@param where that is what they are: build_hcr()'s HCR is ten lines because it has eight
rules, and linkage_spec()'s strata lists its four permitted names.R/ that had none -- 0-build_linkage.R,
0-rceattle_class.R and 0-linkage_table.R. Source comments only; no help page changed.plot_depletionSSB() is deprecated in favour of plot_ssb_depletion(), which is the
snake_case spelling the rest of the package uses. The two have always drawn the same figure --
one was a bare alias of the other -- but neither carried a .Deprecated() call, so nothing
told a reader which name was canonical. The roxygen said plot_ssb_depletion() was the
deprecated one; that is now reversed.
What still works: every existing call. plot_depletionSSB() forwards to
plot_ssb_depletion() and warns, exactly as plot_logindex() forwards to plot_index().
Nothing breaks; scripts gain a deprecation warning.
Whose scripts warn: 18 call sites across the sibling assessment
repositories, 17 of them live (the other is commented out), including
Rceattle-models/Pacific hake/MSE_yr2024.R, which is rule 15's MSE and predation check. The
new canonical name had 0 call sites there, so this deprecates the name consumers actually
use -- a deliberate choice to make the package's own style rule hold. Update at leisure, or not
at all. Counted over all five repositories SIBLING-REPOS.md registers, Climate_MSE
included; a sweep of four misses three of its calls.
Inside the package: the S3 dispatcher (plot(fit, what = "ssb_depletion")), five vignettes
and four examples/ scripts called the old name and would have made the package
deprecation-warn at itself; all ten now call the canonical one.
plot_diet_comp1() was reviewed alongside this and deliberately left alone: its roxygen
records it as a named entry point provided so scripts can call it beside plot_diet_comp2(),
and plot_diet_comp() is already snake_case, so the style argument does not apply.
The developer working notes have moved out of the package. inst/dev/ shipped in the
tarball, so the triaged cleanup backlog, the contributor notes, the per-subsystem TODO-*.md
records and the equilibrium-catch specification went to every user — 193 KB of half-diagnosed
defect detail for a model whose output sets federal catch limits, which is not something to
publish before it is settled. They now live in a separate maintainer repository.
What still ships, and is unchanged: inst/dev/TRAPS.md (the verified traps with their
measured numbers), SIBLING-REPOS.md, SIMPLIFY-LOG.md, REMOVED-kinzey-and-dev-cpp.md and
the two runnable selectivity examples.
Where the open items went: four of the moved notes were cited from comments in shipped R/
and src/TMB/ code. Each is now a GitHub issue, and the comment cites the issue rather than a
file path — #234 the identity-link
recruitment floor, #235 the
NonParametric shape penalties, #236
Sel_norm_scope on Hake, and
#237 the MSE operating-model horizon.
Nothing executable changed: the edits are comments, test provenance headers, agent instructions and the developer-guide article. No function, argument, default or reported quantity moved.
A retrospective peel whose standard-error rebuild fails now carries no sdrep, rather than
the forecast refit's zero standard errors. Each peel is fitted twice, and the second refit
holds every hindcast parameter fixed, so reported from it every quantity in the peel's own
hindcast has a standard error of exactly 0 -- which is why getsd = TRUE rebuilds the peel at
those same parameters with the hindcast free and reports from there. When that rebuild threw, the
handler fell back to the refit's own sdrep: the very object the rebuild exists to replace. The
peel then presented a zero uncertainty band as a measured one, which reads as a certain quantity
rather than an unmeasured one. It now holds no sdrep at all -- the state getsd = FALSE
already leaves.
Both ways the rebuild can fail are handled, not just the throw. TMB::sdreport() does not
error on a Hessian it cannot factor; it returns an object with pdHess = FALSE, which was being
assigned as though it were usable. The returned object is now judged too -- the analogue of what
.refit_converged() judges on a fit -- and the warning says which condition fired, since an
unfactorable Hessian and a report with no finite standard error are different problems.
The peel is kept, not dropped, and print() now reports how many peels could not report
standard errors. The reason is also on the peel as attr(peel, "sd_report_error"). The failure is
deliberately not recorded on .conv_hindcast$sd_present: .refit_converged() drops a peel on
sd_requested && !sd_present, so recording it there would turn a missing band into a dropped
peel and change the number of peels Mohn's rho averages over.
The warning saying so is no longer discarded. It was raised inside run_one_peel(), which
runs in a .parallel_lapply() worker at the default cores, where a warning() is lost. It is
carried out on the peel and re-raised once after dispatch, naming each affected peel and its
terminal year. A peel that cannot be reported is still not a peel that failed: it is kept, and it
is counted separately from the dropped-peel tally.
Whose fits move: none. The rebuild never re-estimates anything, so no point estimate, no
objective and no Mohn's rho changes, and the regression test asserts both against a
getsd = FALSE run. What changes is retrospective()$Rceattle_list[[i]]$sdrep on a peel whose
rebuild threw or came back unusable: previously the forecast refit's sdreport in the first
case and the unusable rebuilt one in the second, now absent in both. vcov() on such a peel returns
NULL, where before it returned a covariance over the peeled-year log_F alone (throw) or the
rebuilt cov.fixed (unusable). The per-peel warnings also become one warning per call, so a handler
matching "^Peel 3:" no longer fires; the phrase "could not report hindcast standard errors" is
unchanged.
Figures change where a rebuild fails. A band that was drawn at zero width is now declined with a
warning(), so plot_biomass() / plot_ssb() / plot_recruitment() with add_ci = TRUE over a
peel list leave that peel's ribbon rows NA instead of collapsing the ribbon onto the line;
ggplot2 then reports the removed rows at draw, so expect two warnings rather than one. Measured with
a before/after ggplot_build() diff of plot_ssb(add_ci = TRUE) on a one-peel BS2017SS
retrospective with the report build forced to fail -- it succeeds on an unmodified run, so the
"before" state cannot be reproduced without forcing it. The ribbon layer holds 231 rows either way
(3 species x 39 unpeeled years + 3 x 38 peel years), of which 114 were finite and exactly zero
width before and 0 after, leaving the unpeeled model's 117 real rows. Those 114 were the
fabricated band.
No bundled configuration is known to hit the failure -- measured on BS2017SS, one peel,
single-species, random_rec = FALSE. Across the sibling repos 30 live retrospective() calls pass
no getsd, so each inherits its input fit's setting, which is TRUE wherever that fit holds an
sdrep (fit_control()'s default); the random-effect peels there are where a failure is most
likely.
Dead Kinzey & Punt (2009) code removed from the template. Eight
commented-out block spans, not the three an earlier note listed, and two of its
line ranges had drifted: the PARAMETER_* declarations, the
H_1 = exp(logH_1.array()) derivations, the avgnMode switch, the Kinzey diet
likelihood, two disabled REPORT blocks, the functional-response implementation
in predation.hpp, and the 42-line change log at the foot of ceattle.cpp.
Three section headers left heading nothing went too -- 3.7 and 4.14 in
ceattle.cpp, 8.2 in predation.hpp -- and a fourth was relabelled rather
than removed: 14.10. Kinzey predation functions had come to head a live
REPORT(mort_sum), the initial-state mortality accumulator, nothing to do
with predation. src/TMB/
falls 380 lines, predation.hpp from 738 to 517 with its FIXME/TODO count from
5 to 0, and ceattle.cpp from 5,986 to 5,827. The word "Kinzey" no longer
appears in the template.
All of it sat inside /* */, so the preprocessor already discarded it: stripping
comments and normalising whitespace leaves ceattle.cpp, predation.hpp and
selectivity.hpp byte-identical to 5.58.0, and clang -E -P output matches per
file. data_check() refuses msmMode %in% 3:9 in any case.
inst/dev/REMOVED-kinzey-and-dev-cpp.md records what was taken and now gives
the commit to recover it from, since there is no longer anything to uncomment.
?build_srr's Bmsy_lim described the wrong scale and the wrong
direction. It said the limit "carries the same scaling" as the Ricker beta,
which is estimated divided by 1e6, and that the penalty applies when beta is
estimated above it. The template divides Bmsy_lim by 1e6 before comparing, so
the scaling cancels and the value is metric tons; and posfun() fires when
1/beta exceeds the limit, which is beta below it. A limit supplied on the
parameter's scale is 1e6 too large, so the penalty on the Ricker SSB-MSY proxy
never fires.
man/build_map_selectivity.Rd hid four shipping selectivity forms. Its
hand-copied Selectivity table stopped at code 8, omitting NonParametricPM
(9), LogisticPM (11), NonParametricIntegrable (13) and DoubleNormalSS3
(15), all four in sel_map; its Time_varying_sel list was likewise stale, and
it attributed IID deviates to sel_sd_prior, the alias Sel_sd_prior
miscased rather than the canonical Time_varying_sel_sd. The page is published, so an
assessment author was told four forms do not exist. Both tables are replaced by
a pointer to "3. Selectivity (per fleet)" in
vignette("model-options-and-functionality") and to rceattle_switch_maps,
which removes the staleness rather than re-syncing it. The DoubleNormal
array-slot mapping and the N_sel_bins form list, neither of which the vignette
carries, are kept. The block also had two @description tags, which roxygen
concatenates.
?plot_maturity documented three arguments the function does not honour.
line_col is never read and the line width is fixed at 1, so both are now
documented as retained-and-ignored, the phrasing already used by
plot_stock_recruit() and plot_selectivity_vs_maturity(). species labels
the facet strips and does not subset, so plot_maturity(fit, species = 1) drew
every species and labelled the rest NA.
Harvest control rule 1 was named "constant catch" in the MSE pipeline.
hcr_map has CMSY = 1 -- maximize joint catch across species -- and there is
no constant-catch rule. One of the two copies is the stated reason for blanking
projection catch before each assessment and now names CMSY; the copy in
mse_summary() points at the registry and ?build_hcr rather than restating
the codes.
?fit_control's comp_offset said it does not apply to diet compositions.
The template adds it to both obs_diet_prop and pred_diet_prop, and its own
declaration comment now says comp/caal/diet rather than comp/caal.
other_food was declared in kilograms and is metric tons. It is summed
into the available-food denominator beside suit * prey_N * prey_wt --
numbers-at-age in thousands times weight-at-age in kg, which is mt by the
same convention the rest of the package uses -- so a value supplied in kg enters
1000x too large, overstating how much non-modelled prey a predator has
available. pred_mort_scalar divides by that denominator, so it deflates
predation mortality on the modelled species. The
schema doc, the template's DATA_VECTOR comment and R/data.R's @format
now all say mt, and inst/extdata/meta_data_names.xlsx is regenerated (103
rows) because the schema's doc strings ship into it verbatim. Note that
under msmMode = 2 the prey term in that sum is squared, so the denominator
is not a biomass in any unit there; the label describes msmMode = 1, which
is what every bundled dataset and live assessment runs.
Smaller corrections, each checked against the code it describes:
check_composition_data() and check_caal_data() were documented as refusing
zero-sum rows, a check that is commented out, and carried a bare throws line
that roxygen folded into the preceding @param; a build_map() comment gave the
index_data input as a CV, where it is Log_sd -- a log-scale sd under the
lognormal family and an absolute natural-scale sd under Index_distribution 3
and 4; selectivity.hpp described nages/nlengths as maximum bins rather
than counts (rule 11); a ceattle.cpp section numbered 5.12b between 5.1 and
5.3 where 5.2 was unused, renumbered and its twelve citations across five
files swept, since five of them sat outside the template and would otherwise
dangle; a cross-reference to section 13.4, which does not exist, and one to
14.6 where the random-effect density actually lives in 14.6b; a stale
ceattle.cpp line citation for the diet renormalization; @param maxage_sp,
unused by get_growth_matrix_r()'s body and kept only because a test helper
passes it; a "dropped next release" promise on two deprecated spellings still
present; a model_average() comment describing a conversion the code refuses,
and another labelling a sdrep$par.random test "All parameters random
effects"; and a stale 4.4 version in the template banner. avgnMode's
three codes are settable but only mode 0 has an effect, which ?fit_mod already
said and the template's declaration comment now says too.
Comment volume. R/ falls 112 lines (doc share 37.8% to 37.7%),
src/TMB/ 380, and man/ 533. 83 #-#-#-# decoration lines carrying no text
were removed; all but four -- two pairs -- bracketed a section header that holds
the RStudio fold itself, and none had the trailing dashes that would make it a section. Thirteen
comments narrating past defects were cut to what the code does now. 24 internal
helpers gained @noRd, so they no longer generate man/dot-*.Rd (193 .Rd
files to 169); none was listed in _pkgdown.yml. inst/dev/CLEANUP_BACKLOG.md's
marker census is re-derived: 49 TODO/FIXME markers on the 5.59.0 base and
35 here, counting a line as a marker only if it marks work -- three point at
an inst/dev/TODO-*.md note instead. The file's old recipe undercounted by four,
dropping genuine ceattle.cpp markers whose text cites one of those notes.
A second survey covered what the first sweep never opened -- the 274-file
test suite, the vignettes, inst/dev/, tools/ and .claude/, plus a second
pass over R/ and src/TMB/. It reported about 1,900 condensable lines and
sixty defects; this release takes the defects.
Two slash commands told an agent something false. /verify listed
verify-mse-schedule.R among the "before/after digests -- run twice, compare",
and that harness has no commandArgs() and no compare path, so an agent runs
it twice, reads two identical PASS lines and reports a change as verified --
the failure mode the same file's own text names. The real digest set is three,
not four, and verify-data-check-conditions.R was missing from the command
altogether; six harnesses take a COUNT as their first argument, not one.
/new-column described Bin_first_selected as an absolute age offset by
minage, where the schema says the opposite in capitals and
rearrange_data() applies a plain - 1 to it while applying sel_bin_offset
to its neighbours (rule 10).
Wrong switch codes, against hcr_map and the estimateMode map.
build_hcr_map() called HCR 1 "Constant catch" ten lines above the code that
turns log_Ftarget on for CMSY -- the last such claim in the repo.
test-dynamics-brps.R, which exercises the whole reference-point surface,
carried 22 wrong labels: estimateMode = 0 called "Run projection only" 15
times when 0 fits the hindcast AND the projection, HCR 4 called Tier 3 (it is
ConstantFSPR), HCR 3 called Constant F (it is ConstantFSSB), and
DynamicHCR = FALSE described as dynamic. Its five correct HCR = 5 Tier 3
labels stand.
?build_M1 and the refusal a user sees scoped the two-sex rule wrongly.
Both said M1_model = 2 needs two sexes "except for M1_re 1 and 4". All
three random-effect arms gate it on two sexes, so with one sex no arm frees a
deviation and the refusal fires under every family; the measured justification
(22 zero-gradient rows in the inner Hessian on GOA2018SS) sits in the 1/4
arm, which is what made the narrower reading look right.
vignette("introduction") printed mohns[1,2] as a Mohn's rho in a figure
title. Column 2 is Forecast year, so the first vignette an assessment author
reads always showed "mohns = 0"; it now reads the rho by name.
vignette("data-without-excel") listed Selectivity only to code 7, hiding
DoubleNormal, NonParametricPM, LogisticPM, NonParametricIntegrable and
DoubleNormalSS3 -- the list now matches sel_map exactly -- documented
PowerEquation as working where the schema marks it unimplemented, and omitted
AnalyticalArith.
?plot_data documented nine base-graphics arguments as live controls.
Measured 0 body references for all nine, against a signature spanning lines
52-70 of the 5.59.0 file: the plot_maturity defect three times over, on a
published page. They are now documented as retained-and-ignored, the page no
longer promises a _data_plot2.png that .save_ggplot() never writes, and
@return states what the point area means -- catch in metric tons, 1/SE for
indices, sample size for compositions.
Two claims this sweep itself got wrong are corrected here. ceattle.cpp
carried a FOURTH orphaned Kinzey header, misspelled "Kinzery", so the grep
behind the earlier "the word Kinzey no longer appears" validated an absence
over surviving content; and line 88 still held a table-of-contents entry for
the change log that was deleted. Separately, a global S -> S_at_age rename
had mangled the stock-recruit prose on the six SB0 / B0 / DynamicSB0
declarations an ABC derives from.
Smaller, each checked against the code it describes: three stale ceattle.cpp
citations in tests, drifted 410, ~480 and ~1,750 lines onto unrelated
blocks; three dangling tests-Growth/ and tests-Linkage/ paths in a flat
suite; two renamed test files; fit_control(osa = TRUE), an argument removed in
c839f764; sel_curve_pen's column description, wrong twice; 8.2 -> 8.2b
and two SPR0 -> SPRFinit annotations; spr.hpp's 6.5 -> 6.4;
max_nsex and beta_rec_pars, neither of which exists; a "see 1.3c below"
with no 1.3c; TMBphase()'s silent default and five identical M1 parameter
tags of which four were wrong; env_data's "Catchability = 5 or 6" where
data_check() errors on 6; UobsAge, which build_data() recognises by name but no
template reads and no bundled dataset carries, so it round-trips to nothing; Cheng 2025 ->
2024; Wassermann 2024 -> 2025; "twelve checks" where thirteen are registered;
and test-plot-save-paths.R's claim that a file-level skip_on_cran() is the
test-plot-* convention, against 15 files of which 6 skip at file level and 6
carry no skip_on_cran() at all.
The 87 lines of deferred 5.49.0 findings are recorded in
CLEANUP_BACKLOG.md Tier 2, where open-and-unfixed work belongs rather than
in SESSION_HANDOFF.md, which is for work in flight. The handoff copy is left
in place for the open pull request on that file to remove.
No fit can move. Of 35 changed R/ files, two differ at parse level, by
three string literals in all: the other_food and env_data doc strings in
R/0-column_schema.R, which are data the workbook reader and the generated
xlsx consume, and the M1_re refusal text in R/3-build_map.R. No likelihood
reads any of them, and no test pins the old wording. Every changed src/TMB/
file is byte-identical to the 5.59.0 base once /* */ and // are stripped
and whitespace normalised, each check run with a negative control. The only other code change is one assertion in
test-schema-cpp-dispatch.R, which pinned that the dead switch (msmMode)
was present and now pins that no msmMode dispatch exists at all.
run_mse() no longer extends NByageFixed into the projection. Numbers-at-age for a
species with estDynamics > 0 are an input, not a forecast, so holding the terminal hindcast
year across the projection invented data and pinned that species' population flat for the whole
MSE. It now carries only the years the input supplies.rearrange_data() leaves those years at the array's initialised 0 and the template computes
N_at_age = pop_scalar * NByageFixed(...), so the species contributes no biomass, catch or
predation in them. Raised before the parallel dispatch, since a warning from a
.parallel_lapply() worker is discarded.(Species, Sex) group, then subset them with
which(om_proj_yrs %!in% NByageFixed$Year) -- positions within om_proj_yrs, not row positions
in the grouped frame. With one group the two coincide, which is why it went unnoticed. Measured
on GOA2018SS (4 groups, 32 projection years): 128 rows built, 32 kept, so species 2 (both
sexes) and species 3 received no projection rows at all.Whose MSEs move: estDynamics is a control-sheet column, so a sweep of .R files cannot
clear this. Rceattle-models/Pacific hake/04-mse.R is in the affected configuration: its workbook
sets estDynamics = c(0, 1, 1), the script projects to 2025, and NByageFixed covers species 2
only to 2019 and species 3 only to 2024. Those two species now contribute no biomass and no
predation in the uncovered years -- a missing row reads as zero, which is the intended meaning, not
a number carried forward -- and both prey on hake, so that MSE's projected predation mortality on
hake moves. (Neither has a fleet in that workbook, so no catch is lost.) MSE_yr2024.R is unaffected: its coverage runs to 2066, which also means the Pacific
hake MSE gate does not exercise this change. An MSE that set estDynamics was previously running
on invented numbers for every species but the first, and on flat-held numbers for that one.
An intercept init or bounds on a shared block now reaches the parameter, instead of being
diluted by the rest of the group. Fleets sharing a Selectivity_index / Catchability_index
estimate ONE parameter block, and TMB:::updateMap() starts a shared level at
tapply(parameter.entry, map.entry, mean) -- the mean over the whole level. The push wrote the
donor's cell only, so the other members' build defaults pulled the start off the request.
Measured on BS2017SS with fleets 4 and 5 sharing a Selectivity_index and
linkage_spec(~ 1, init = list(intercept = 0.5), by = ~ fleet, fleet = 4):
| | log_sel_slp start | on the natural scale |
|---|---|---|
| before | -0.096574 | a slope of 0.908 |
| after | -0.693147 | 0.500000, as asked |
.stop_if_shared_block() refuses a value set on a FOLLOWER and tells the caller to put it on the
donor, so the spelling it recommends was the one that diluted.
The same defect on bounds was worse, because fit_mod() reduces bounds to one entry per map
level by first occurrence (match(levels(droplevels(mf)), as.character(mf))): if the donor was
not first in array order, the surviving bound was an unwritten DEFAULT rather than the requested
one. Both pushes now write every member of the block, so whichever cell the reduction picks is
right.
The expansion does not filter on species -- a group may span species, and the mean does not care -- so for a member of a one-sex species in a two-sex caller's group the worst case is a write into a padding cell, which is mapped off and is neither read nor part of the level.
Whose fits move: any fit that sets an intercept init or bounds through a sel or q
linkage on a fleet that shares its block. Nothing in the bundled datasets or the sibling
assessment scripts does -- the GOA cod bridge's three intercept inits are all unstratified growth
rows, and its selectivity specs are intercept-free -- so no shipped configuration changes. A fit
that does set one was starting somewhere nobody asked for and will now start where it was told.
src/TMB/growth.hpp advances a slot from the same cohort one year younger, so it is only
valid once that predecessor is itself on the growth curve. The test for "not yet" compared the
slot's age to max(minage, 1), where the first slot above the anchor is at
max(minage, floor(growth_age_L1) + 1). Where those differ, every year after the first
recursed from a linear-ramp length. Measured on a make_test_data fit at minage = 1 with an
anchor of 2.5: the slot above the anchor came back 2.2959 cm against a correct 1.6965
(+35.3%), with a 0.5994 cm spread across years that constant growth parameters forbid. It
propagates into every fleet's weight-at-age, and it put the two growth routines in
contradiction -- the same cohort at the same instant read 15.8752 cm from the within-year
advance and 19.8139 cm from Jan-1, a gap that is now exactly 0. The test is now
current_age - 1 < age_L1_safe || age == 0, and age_L1_ceil is deleted.-1 -- an out-of-bounds load this build does not report. The old
gate covered this only incidentally, via max(minage, 1) == minage; the age == 0 clause is
what covers it now, and without that clause the new first test alone would read -1 wherever
the anchor sits below minage.Nothing shipping moves. Every Growth_Age_for_L1 in the 188 SS3 control files reachable here
is 0.5 or an integer (1 x113, 0 x44, 0.5 x30, 3 x1), and for all of them the new gate reroutes no
slot at any minage 0-4 or nages 2-20; the max(0.5, minage) default is likewise bit-identical
for minage 0-4. Two mechanisms make that so: at an integer anchor the ramp ends exactly at l1,
so the recursion already inherited a curve value, and at a sub-year anchor the first slot above it
is Jan-1 age 1, which max(minage, 1) names when minage <= 1.
Both arms are pinned by new blocks in test-growth-vonbert-year-invariance.R; the non-integer
block fails 10 assertions on the parent commit, which is also testthat's cap. /golden-check is
not evidence here -- BS2017SS and GOA2018SS carry no growth_model, so estimate_growth()
is never entered; it passed as a no-regression check on everything else.
src/TMB/growth.hpp tested age + 1.0 < age_L1 -- age is the 0-based
slot and age_L1 the von Bertalanffy anchor age -- so the test only equalled the cohort's age
at minage == 1. It is now age + minage(sp), which is what the sibling Jan-1 routine already
uses. Three wrong numbers follow from it, all at minage != 1:
minage = 0 with a fractional anchor the mean length was discontinuous in the season:
10.000000 cm at fracyr = 0.5 against 8.815181 at 0.5 + 1e-7, so a fish shrank 11.9% as the
year crossed July 1. At month 9 the age-0 length goes 11.9163 to 13.0293 -- 8.5% in length and
about 23.5% in weight.minage >= 3 with time-varying growth the arm recomputed from the closed form and
discarded the cohort recursion: 23.7240 against the correct 27.6468, 14% low.estimate_growth() wrote. That ordering regression was
introduced by the first version of this fix and caught in review -- measured at 54% low on a
one-age minage = 0 model, 63% on a four-age one.This is NOT a no-op, and the reason matters more than the claim. minage == 1 is
bit-identical, unconditionally -- but minage = 0 is live, since it is the SS3 converter's own
default and both the GOA and AI cod bridges pass it with parametric growth. What keeps those two
unchanged is configuration, not the minage == 1 premise: GOA cod's anchor is 0.5 and its largest
fleet month is 6, so fracyr equals the anchor exactly and the first branch's <= wins -- a
margin of one month, and an August survey would move its age-0 length, ALK and weight-at-age.
AI cod is safe for a different reason, an integral anchor of 1.0, and only a fractional anchor can
diverge at minage = 0. Nothing live combines minage >= 2 with parametric growth.
The arm fires exactly when age_L1 is non-integer, fracyr > age_L1 - floor(age_L1), and
floor(age_L1) lies in [minage, minage + nages - 1]; at most one age slot ever matches.
/golden-check cannot see this change at all. Every bundled dataset resolves to empirical
weight-at-age, and the routine is gated behind growth_model(sp) > 0, so none of the four
reference models enters it. A green golden run is not evidence here; test-growth-*.R is the
cover.
test-growth-minage2-anchor.R: 4 blocks, 8 assertions, measured in three code states --
8 pass as shipped, 2 failures against dev (the minage >= 2 and minage = 0 cases),
and 1 failure with the age test fixed but the plus-group pin left second (the ordering
regression). Each rebuild was forced, for the reason in the next entry.spawn_month > 0, since fracyr = 0 lets
the first branch take everything; and an anchor ABOVE minage, since at the default
max(0.5, minage) the diverging slot's Jan-1 length is exactly l1 and both arms return the
same number -- the branches differ and the values do not.inst/dev/TRAPS.md records that a header-only edit under src/TMB/ does not trigger a rebuild,
so it is verified against the previous binary unless the artifacts are deleted first.Comments, help pages and vignettes condensed, and the documentation defects
found along the way corrected. Documentation in R/ fell from 39.3% of the
file to 37.8% (16,141 to 15,165 lines, 947,961 to 881,463 characters), man/
by 580 lines, src/TMB/ by 35, and the vignettes by 161. Most of the
reduction is help-page prose a vignette already carried: ?osa_residuals held
four @sections, two of which restate vignette("model-diagnostics"), and
went from 395 roxygen lines to 121; ?fit_mod's initMode section restated the
seven-mode table in vignette("model-options-and-functionality"). Help pages
link to the vignette instead. The 16 affected man/*.Rd were regenerated with
devtools::document(); NAMESPACE and _pkgdown.yml are unchanged.
Nothing executable changed but two documentation strings, so no fit can
move and /golden-check has nothing to measure. The four-model golden
regression was run regardless and passed, in a full NOT_CRAN suite of 10,507
assertions. The two strings are quantity_dictionary()'s description of
steepness and one osa_residuals() message (both below).
Four comment sites in the TMB template gave biomass in the wrong unit.
Three section headers read BIOMASS-AT-AGE (kg) and ssb-AT-AGE (kg), where
quantity_dictionary() gives biomass, SSB and biomass-at-age as mt --
numbers at age in thousands times weight at age in kg. A fourth read
TOTAL YIELD (kg), which is wrong in dimension as well as scale: catch is mt
or thousands of fish, per the fleet's Observation_units. Comments only.
The surviving (kg) labels in that file -- mature weight-at-age, spawning
output per fish, selected body weight, and other_food, which the schema
declares in kg -- are correct.
Four analytical-q equations in vignette("model-parameterizations") omitted
the /12 that ceattle.cpp applies to the survey month, making the
exponent twelve times too large. The same vignette's snapshot-index equation
had it right, so the document contradicted itself.
Three jnll_comp row labels named the wrong row, marking Slot 9 above
three writes to JNLL_SRR_PRIOR, which is row 8. A second, stale slot
registry duplicating the JnllRow enum was deleted: it stopped at row 18, so
rows 19 to 21 were missing, and it gave catch in kg. .JNLL_ROW_AXIS in
R/9-profile.R remains the registry. Two further stale labels (Slot 15,
Slot 16, on the ration rows) are left as they are, inside a commented-out
block of inert Kinzey and Punt code.
vignette("environmental-linkages-and-priors") said a linkage prior is
applied with no lognormal bias correction. The template applies
p1 - bias_adjust_proc * sd^2/2 and that flag defaults to TRUE, so a stated
prior is a mean rather than a median -- which the same file said correctly
twice elsewhere.
Catchability = "PowerEquation" was documented as working. The schema
marks code 4 not yet implemented and a user can set it today; the caveat is
now stated, and code 7 ("AnalyticalArith") is documented.
Six comments and two vignette passages credited parLapply on a PSOCK
cluster for retrospective(), jitter(), self_test(), profile() and
run_mse(), which dispatch through .parallel_lapply() and fork wherever the
platform allows, falling back to PSOCK only on Windows. mse_summary() does
build a PSOCK cluster and its comment is unchanged.
Smaller corrections, each verified against the code: a nages row in
vignette("stock-synthesis-conversion") described as the oldest age when it
is a count of age bins, which coincide only at minage = 1;
CA * Weight^CB where bioenergetics.hpp uses CA * W^(1+CB) with W in
grams; estM1, combine_data_sets(), fit$linkages$K and data_list$wt,
none of which exist (the real names being est_M1, combine_data(),
build_growth()$linkages$K and data_list$weight); a second comment
asserting an osa argument fit_control() has never had; four
wrong ceattle.cpp section cross-references; a claim that fit_mod()
suppresses build_map()'s warnings when it de-duplicates and re-raises them;
Time_varying_q = "AR1" listed without the note that data_check() refuses
it; re_group called a placeholder when it keys the sigma group;
DoubleNormalSS3 missing from the supported-forms list; the apical refusal
list missing Fleet_type = "Off"; two MSE conservation metrics named in an
index but never computed; a stale TODO asking for a feature the process
argument already provides; and a table row whose unescaped | truncated its
own advice where it rendered.
The analytical-q citation had its authors reversed in 24 of 26 places. It
is Walters, C. and Ludwig, D. (1994) 'Calculation of Bayes posterior
probability distributions for key population parameters', Canadian Journal of
Fisheries and Aquatic Sciences, 51(3), pp. 713-722, doi:10.1139/f94-071 --
not "Ludwig and Walters", and one site named Martell, a different author. All
24 are corrected across R/, src/TMB/, vignettes/ and tests/, and
vignette("model-parameterizations") now carries the full reference.
The survey month is read from the index_data row, not the fleet. Five
equations in vignette("model-parameterizations") subscripted it
$Month_{f_i}$, where the template takes mo = index_n(index_ind, 0), so a
fleet whose survey timing has moved carries a different value per year. Now
$Month_{{f_i},y}$, with the source stated.
The ration equation gave no units, and its weight is in grams. The
template powers weight_hat * 1000 and divides the result by 1000, so
$\alpha^{\delta}_p$ is a coefficient for a gram-scale weight and $\delta$
comes back in kg per predator per year. Both are now stated; neither was.
Two selectivity configurations in that vignette could not be run as
printed, pairing Time_varying_sel = 1 and = 5 with
Time_varying_sel_sd = NA, where the schema reads that column for modes 1, 4
and 5. Both now show 0.05, the value GOA2018SS and GOApollock set on
their time-varying fleets.
One message changed. osa_residuals() pointed composition callers at the
"Choosing a method" section of its own help page, which this release moves to
the vignette; it now names "Choosing the one-step-ahead method" in
vignette("model-diagnostics"). No test pins the text.
steepness now says it is a placeholder. Under mean recruitment the
reported value is a hard-coded 0.99 rather than an estimate, which
quantity_dictionary() and ?build_srr had both stopped saying.
Left deliberately long. R/data.R's @format is a registry three guards
in test-schema-canonical.R pin against the column schema and the switch
maps. R/0-column_schema.R's field dictionary is what that file's own roxygen
points at. build_srr()'s starting-value guidance, including the alpha/beta
scaling and the NA/NaN gradient evaluation it prevents, appears in no
vignette. articles/developer-guide.Rmd is untouched: its file table, its
filenames and its C++ switch quotes are all pinned by test-docs-anchors.R.
Two behavioural defects found during the sweep are filed in
inst/dev/CLEANUP_BACKLOG.md rather than fixed, each needing verification of
its own: run_mse()'s NByageFixed projection filter indexes a grouped frame
by year position, so an operating model with more than one Species/Sex
group of fixed numbers silently keeps only the first group's rows; and
growth.hpp's mixed growth branch compares a slot index with an age, which
disagrees with its sibling branches at minage >= 2. Both are unreachable in
anything shipping -- every bundled dataset and live assessment uses minage
0 or 1, and no live model carries multi-group fixed numbers.
A selectivity or catchability linkage on fleets that share a block now estimates ONE
coefficient for the group, not one per fleet. beta_linkage is a flat PARAMETER_VECTOR with
no fleet dimension, so adjust_map_shared_params() -- which reconciles 15 by-fleet slices onto
the group's donor -- could never reach it, and one design column on a shared group became one
free coefficient per member. Estimated separately the copies diverge, so fleets declared to
mirror each other fitted different selectivity, with no warning. This was the default path:
by defaults to ~ fleet for sel and q, and omitting fleet = does not avoid it, because
that argument is a filter whose NULL default means every fleet. The shared-block spelling is an
explicit by = NULL.
Measured on GOA2018SS fleets 9 and 10 (ATF_bottom_trawl and its length-comp twin, sharing
both indices), a ~ cut(Year, 2) block column on inf_asc, optimized:
| | coefficients | selectivity across the pair | objective | |---|---|---|---| | before | 2: -0.232615, -0.352109 | differs by 0.123 at its worst age | 12879.8748 | | after | 1: -0.240043 | identical | 12879.9970 |
The objective rises by 0.1222 nats, the only direction one fewer free parameter can go.
A saved fit still refits. The linkage table is unchanged, so inits$beta_linkage still has
one entry per row; only the map collapses them. TMB:::updateMap() means the level, so the
shared coefficient starts at -0.292362 and converges to -0.240044 -- the same point a cold fit
reaches. So there is a back-compat path and this is a behaviour change, not a breaking one. What
moves is the answer: expect the objective and one fleet's selectivity to change on refit.
Reported by the GOA cod bridge session, which measured it on a real model: 47 selectivity block
columns became 59 linkage rows, copies diverging up to 20.89 on the log scale, with a
non-invertible Hessian and a fit ~33 nats below SS3's optimum on fewer declared parameters.
With the fix, the 18 paired Srv/Srv_ae1 coefficients go to an exact match and exactly-flat
Hessian directions fall 9 to 5. (Its free-parameter counts, 344 to 328 against SS3's 330, are
specific to that session's uncommitted bridge configuration and are not package figures.)
Four configurations a mirrored block cannot honour are now refused.
One coefficient equalises a group's offsets only where every member has one. The base parameter is
shared, but the offset it scales accumulates per fleet (inf_offset(param, flt, sex, yr)), so
a row naming one member moves that member alone -- fitted, naming the lead alone leaves the pair
0.265 apart and the follower alone 0.248. "Put it on the lead fleet" is therefore right only for a
prior, which is re-targeted onto the shared base; every param code and the q offset write a
per-fleet slot, the apical height included. .stop_if_mirrored_block_linkage() refuses:
fleet = filter naming some but not all members whose block is
estimated, with the missing fleets named. A member that cannot carry a row at all (q Fixed,
Analytical, absent; a selectivity form not wired for linkages) is not owed one.2 * beta * X against the group's beta * X: one reported coefficient and two
curves, which no parameter count reveals.link, bounds or init -- the group would take one value by
table position. est_phase may differ: a held donor holds the whole group, which is how a
reference model that fixes some time blocks is reproduced.An intercept row is exempt from coverage and agreement: it re-targets the shared base and is
pinned to NA, so it cannot diverge, and .stop_if_shared_block() already governs it.
Nothing shipping is refused. Of 104 linkage_spec() call sites in the sibling repositories, 13
declare a random-effect structure and the GOA pollock 2025 scripts do pass spec lists, so both
shapes are live code -- but that workbook has no index group with two or more members whose block
is estimated (its one shared group pairs a survey with an Off fleet). The SS3-bridged GOA Pacific
cod bridge, which does share both indices across its *_ae1 fleets, was run against this branch by
the session that owns it: clean in three configurations, forward-pass objective 1912.2067 in
all three, and the baseline console output byte-identical to the same run before these rules. That
is a build and forward pass at estimateMode = 3, not a fit, and that model happens not to
exercise the Off-member case; GOA2018SS Selectivity_index 1 is the fixture that does.
normal(0.4, 0.1): jnll_comp["Linkage-table priors"] 13.232707, i.e. two terms, with and
without the tie. build_selectivity() has refused the selectivity equivalent since 5.42.0; the
accepted spelling is a prior on one member and a plain row for the rest.apical_offset(flt, sex, yr) is per fleet like every other slot, so it does not; the
message now points at naming the whole group or by = NULL.test-linkage-shared-block.R: 13 blocks, 56 assertions, measured in three code states so the
file is known to discriminate -- 56 pass as shipped, 15 failures + 1 error with the four
refusals removed, 27 failures + 1 error with neither the tie nor the refusals.test-linkage-intercept-base-param.R and test-linkage-double-prior-guards.R were failing on
the first version of this change and are green again. The intercept blocks now use an
intercept-only formula: ~ temp also emits a slope, and a slope on one member of a mirrored
block is what this release refuses, so the covariate was incidental to what they test.Selectivity = "DoubleNormal" fitted a flat curve from its default starting values, and now
fits a dome. DoubleNormal reads the two shared sel_inf slots as a PEAK and a right-tail
floor, where the logistic family reads them as two inflections, so the shared defaults were
wrong for that reading: the peak started at 0 -- below the first age -- and the floor at 10 on
the logit scale, i.e. plogis(10) = 0.99996. The starting curve was therefore flat at ~1 at
every age, which leaves the ascending width with no gradient, so the optimizer had no descent
direction and stayed on that ridge.
Measured on the GOApollock fishery, static selectivity, phased, inits = NULL:
| starts | selectivity-at-age | objective | AIC |
|---|---|---|---|
| peak 0, floor logit 10 (old) | constant 0.999996 | 3085.98 | 6611.96 |
| peak 5.5, floor logit 0 (new) | 0.0024 .. 0.9815 .. 0.4407 | 914.10 | 2268.21 |
| DoubleLogistic, same defaults, for scale | a dome | 918.15 | 2276.31 |
2171.88 nats, and the fixed form now edges out the double logistic by 4.05. 914.1043 is the
same optimum a hand-tuned inits reached, so the derived starts find what someone previously
had to set by hand.
The peak is derived, not chosen, and it is a BIN ORDINAL. An age-based curve is evaluated at
bin + 1 (selectivity.hpp), so the x-axis is 1 .. nages and the mid-range is
(nages + 1) / 2 -- 5.5 on a ten-age stock, and independent of minage. That is the same
expression the DoubleNormalSS3 block already uses for the same slot. (Reading it as an absolute
age gives the same 5.5 at minage = 1, which every bundled dataset has, and 7.5 on a
minage = 3 stock -- a milder version of the asymmetry this change removes.) A length-based
DoubleNormal keeps its length midpoint rather than being given a bin.
The peak carries the fix; the floor does not. With the peak at mid-range the fit reaches
914.1043 from a floor start of either 0 or 10; with the peak left at 0, a floor of 0 still fails
(2973.69, with the peak running to -859 -- sel_inf is unbounded). Logit 0 is kept because it is
the logistic's maximum-gradient point, not because LogisticPM uses 0 in the same slot: that
slot is a LOG there (age-1 log-selectivity) and a LOGIT here, which is the conflation
.sel_inf_is_natural() exists to prevent.
The fitted right-tail floor is not identified, and this change makes that the default outcome
for the form rather than introducing it. On the headline fit it lands near -17.8 with a standard
error in the thousands, and anywhere in [-21.6, -10.4] across multi-starts with the objective
unchanged to four decimal places; under newtonsteps = 3 it moves again. pdHess is TRUE and
convergence_diagnostics() returns OK, because the conditioning check reads the correlation
matrix. vignettes/articles/adding-a-selectivity-form.Rmd documents this property of the form.
Nothing shipping moves. No bundled dataset sets Selectivity = "DoubleNormal" (all 12 use
codes 0-4 and 6), so none of the four golden references reaches this and build_params() output
is bit-identical between dev and this change on every element of its 49-element list for
BS2017SS, BS2017MS, GOA2018SS and GOApollock. Four scripts under
Rceattle-models/GOA cod/Bridging/ do set the form, but inject their own sel_inf immediately
afterwards, and the live bridges have since moved to DoubleNormalSS3 -- so they are inert for
that reason rather than for absence. test-selectivity-double-normal.R never saw this because it
supplies its own starts.
Not a claim that the form is now the best choice. The escape from the flat ridge generalises
-- improvements of 2171.9, 1970.9 and 5970.5 nats on the GOA pollock, Atka and GOA cod fisheries
-- but on Atka the dataset's own NonParametric still fits better (755.6 against 989.9), and on
GOA cod the DoubleNormal collapses to a degenerate corner (sigma_desc to 0, floor to 1, i.e. a
plain logistic) and returns convergence FAIL. The change makes the form fittable from its
defaults; it does not make it safe to choose.
test-selectivity-double-normal-starts.R: 4 blocks, 15 assertions, 8 of which fail without the
fix. Three need no fit and so run on a pull request; only the fitted-curve block carries
skip_on_cran(). It asserts the starts, that every other fleet's starts are untouched (the
assertion that would catch a which() matching too widely, which is how a start change leaks
into a fit that never asked for it), that a length-based DoubleNormal keeps its length midpoint,
and that the fitted curve is a dome reaching 914.1043. The fit block asserts the shape of the
reported array before reading any value -- sel is not a reported quantity name (sel_at_age
is), and all() over a zero-length vector is TRUE, so an empty slice would otherwise satisfy
every check.Time_varying_sel = "RandomWalkAscending" is refused on every selectivity form but
DoubleLogistic, because it was silently fitting a static curve everywhere else. It walks the
ascending limb and holds the descending one fixed, and build_map() assigns its deviate indices
on DoubleLogistic alone. On the other parametric forms the combination fell through both arms of
that form's branch, so every deviate stayed mapped out and a static curve fitted where the
workbook asked for a walk, with no condition raised. The TEMPLATE does score the ascending
density for Logistic and DoubleNormal too (flt_sel_type 1, 3 and 8), so this is a
build_map() omission rather than a missing model -- which is why the fix is a refusal plus a
linkage route, not a claim that the model does not exist.Selectivity and Time_varying_sel
(parameters / free selectivity deviates): DoubleLogistic 316 / 96, DoubleNormal 220 / 0,
Logistic and DescendingLogistic 218 / 0 -- the same free parameters as "Off", and the
same fitted curve. The objective is not the same on Logistic and DoubleNormal: the template
scores the first differences of their ascending deviates, which sit at 0, so every hindcast year
after the first adds the constant log(sd) + log(4 sd) + log(2 pi) per sex. Of the four forms
measured only DescendingLogistic is bit-identical to "Off", and only under
random_sel = FALSE -- flt_sel_type 4 is absent from the ascending arm, and the descending arm
is gated on mode 4. (2DAR1 and 3DAR1 ignore Time_varying_sel altogether, which is why they
are exempt from the refusal rather than refused by it.)
LogisticPM, Hake, NonParametric and NonParametricPM also lack the arm but were already
refused per form.random_sel = TRUE: the deviation sd is freed for any form that is not
Fixed whenever Time_varying_sel is IID, AR1, RandomWalk or RandomWalkAscending, so
sel_dev_log_sd became a free parameter scaling deviations that were all mapped out -- the
same stray-hyperparameter shape as M1_re under an unimplemented M1_model (5.53.0).peak and sigma_asc for DoubleNormal,
inf_asc and slp_asc for the logistic family, which alias the same slots the template scores
-- gives a time-varying ascending limb with the descending one fixed. Time_varying_sel on a
parametric form is itself soft-deprecated in favour of those linkages, so implementing mode 5 for
a second form would add a route into a mechanism already on its way out. The error prints the
build_selectivity(linkages = ...) call for the fleet's own form.2DAR1 and 3DAR1 are exempt. They estimate their deviate field for every bin and year
regardless of Time_varying_sel, and build_map() says so with a warning that the column is
ignored for them, so the combination was never static there.fleet_control directly and only GOA2018SS and GOApollock set
RandomWalkAscending, both on the DoubleLogistic GOA pollock fishery; no script in the sibling
assessment repos sets it at all. Per inst/RELEASE-CHECKLIST.md, refusing a configuration that
never fitted the model it described is a minor bump.Time_varying_sel on a parametric form
is separately soft-deprecated in favour of selectivity linkages, so this closes a hole in a
mechanism that is already on its way out rather than extending it.test-selectivity-random-walk-ascending-forms.R pins the refusal, asserts the three refused
forms still accept RandomWalk and still free strictly more parameters than "Off" (the thing
the dropped combination failed to do), and derives from R/3-build_map.R that exactly one form
assigns RandomWalkAscending deviate indices -- so a new form added without that arm is caught
here rather than by a user. 3 blocks, 20 assertions; 6 of them fail without the refusal.
Registered in tools/ci/source-guards.R, whose set is now 18.AR1 is refused package-wide, so the c("IID", "AR1", "RandomWalk") membership tests in
build_map()'s selectivity branches name a value that can no longer arrive. Left alone here;
noted so the next reader does not take them as evidence AR1 is supported.plot_comp() set
theme_bw(base_size = 10) on its Pearson residual bubbles and theme_bw(base_size = 9) on the
composition fit panels, and plot_diet_comp2() set theme_bw() on its line and bar fits
(cases 1-3) but theme_classic() on its bubble panels (case 4). Each function now uses one
look, resolved on the arm the rest of the package already agreed with: plot_comp() on
base_size = 10, because its bubbles are pinned to plot.rceattle_osa()'s size scale -- the
same residuals appear in both figures -- and plot.rceattle_osa() draws them at 10; plot_diet_comp2() on
theme_classic(), because its case-4 panels are the same observed / estimated / Pearson bubble
triptych plot_diet_comp() draws, and that function is theme_classic() throughout.ggplot_build(): the diet panels' data layer is identical, with 6 theme elements changing
(panel.grid off, axis.line on, plus axis.text, axis.ticks, panel.border,
strip.background); the composition fit panels scale their fonts 9 to 10 and, because ggplot2
4.x derives geom defaults from the theme, the shaded area's outline linewidth goes 0.409 to
0.455 -- though comp_area_plot() leaves colour at NA, so that outline is never drawn. The
explicit linewidths on the fitted line and the zero line are unaffected.theme_classic() leaves panel.border blank, so the three faceted diet panels are now
delimited by their strip boxes -- black at linewidth 1 -- rather than by a box around each
panel. .rceattle_theme() would supply that border, but it is not available here: it sets
legend.title = element_blank(), which would delete the "Source", "95% CI", "Prop." and
"Abs(Resid)" legend names the diet plotters set on every panel.test-plot-theme-consistency.R asserts that no plotter in R/7-*.R draws one figure family at
two looks, pins which look each file's figures are drawn at, and records the three the package
uses -- .rceattle_theme() (theme_classic plus a panel border, no gridlines) for the
trajectory figures, theme_bw(base_size = 10) for the OSA and composition panels, and
theme_classic() for the diet family. It is not "residual diagnostics get gridlines":
plot_indexresidual() and plot_profile() draw residuals and profiles at .rceattle_theme(),
with none. Parsed from the call objects rather than the source lines, so a reformatted or
line-wrapped call cannot blind it, and a theme(text = element_text(size = ...)) override
counts as a base size -- that is the easiest way to reintroduce the defect while leaving the
theme_*() call alone. What it asserts is one look per set of theme CALLS, not per figure:
plot_comp() and plot_indexresidual(residual_type = "osa") both delegate to
plot.rceattle_osa(), so one function can still render two looks by argument. Unifying the
three onto one theme would take the gridlines off every OSA and composition panel; nothing here
blocks that, but it is now a conscious change rather than a drift.tools/ci/source-guards.R, so the guard job runs it against a real source
checkout with failures fatal. Three counts stated in that file are re-measured: the set is 17
files, not 16 (this change adds the 17th); all of them hold several test_that() blocks, 2
at the fewest and 14 at the most, where the file said 10 of 16; and 6 of the 17 are named in a
vignette, where it said 5 were named "anywhere else". The last two were wrong when written
rather than gone stale -- neither 10 nor 5 matches any reading of the file set at the commit
that introduced them.NULL file =
to any plotter -- the smoke test relies on the default -- and the string "png" appeared nowhere
under tests/ or tools/. So the code that writes user-facing files was exercised by nothing.test-plot-save-paths.R records what every plotter writes, keyed off the file prefix.
The suffixes are measured, not derived, and they are irregular: plot_recruitment writes
_R_trajectory, plot_catch _fishery_catch, plot_mortality _mortality_at_age,
plot_data _data_plot.BS2017SS fixture, and
all 29 are now covered -- 25 that write one figure by exact filename, and plot_comp (15
files), plot_diet_comp, plot_diet_comp1 and plot_diet_comp2 (9 each) by file count and
naming pattern, since those names embed spnames from the dataset. Only plot_form (a stub
with no file formal) and plot_profile (needs an Rceattle_profile) write nothing, and both
are named as exclusions with a stale-entry check..save_ggplot() (19 call sites),
plot_comp()'s own save_png() closure (1 definition, 3 uses) and the 5 inline
ggplot2::ggsave() calls in R/7-plot_comp.R, each with its own name and figure size. This is
the net a save-path unification needs first -- ggplot_build(), which CLAUDE.md names as the
net for a plotting change, never touches saving.file = to a plotter (169 in Rceattle-models, 34 in GOA-ATF-ESP, 12 in
GOA_circlulation_study). None globs the result, so a rename would not error anywhere -- it
would leave a differently-named figure.plot_index() writes the same filename on both scales. Its suffix is unconditional, so
plot_index(fit, file = "p") and plot_index(fit, file = "p", log = TRUE) both write
p_survey_indices.png -- two different figures, one file, whichever ran last. plot_logindex()
forwards with log = TRUE, and GOA-ATF-ESP/R/Run_2025_ceattle.R:274 calls it with file =.
Recorded in test-plot-save-paths.R and in the Tier 0 backlog; not yet fixed, because changing
a written filename is a user-visible change and the two scales need names chosen deliberately.plot_diet_comp() builds its name from
paste("Pred-", spnames[i]), so it writes
p_aggregated_diet_comps_year1_Pred- Arrowtooth flounder_prey_Cod.png -- a space after the
dash and the spaces inside each species name. plot_timeseries() already sanitises a species
name for its CSV with gsub("[^A-Za-z0-9]+", "_", ...); the diet plotters do not use it.
Also recorded rather than fixed, for the same reason.retrospective() now says when a fit's stored map no longer matches one built today. Each
peel's hindcast is fitted with object$map, so the peels reproduce the parameterisation the
model was originally fitted with -- which is what Mohn's rho is a comparison against, and is
deliberate. The cost is that a later fix to how a map is built never reaches a saved fit, and
that within one peel the two passes disagree: the hindcast uses the stored map while the
forecast-catch refit calls build_map() afresh, so a peel can be fitted and reported under
different parameter counts.warning() raised inside a
.parallel_lapply() worker is discarded, and the default cores is detectCores() - 6, so
the parallel path is the one anyone actually runs -- a per-peel check would have been invisible
exactly where it matters. Verified at cores = 1 and cores = 2.test-diagnostics-retro-map-divergence.R: 3 blocks, 8 assertions, failing on the parent. One
block pins that a current fit is silent, one that a perturbed stored map is named, and one that
the warning survives the parallel path.M1_model = 3 can now estimate by-year M deviations -- M1_re 2 (IID) or 5 (AR1 over
years). 5.53.0 refused it along with the rest of M1_model 3/4/5, because no arm wrote the
deviations; this is the one of those combinations that is well posed, and it is WHAM's
age-specific-mean M with ar1_y deviations.log_M1. Measured on
GOA2018SS: log_M1 keeps its 64 age-specific parameters, 126 deviations are freed
(42 hindcast years x 3 species, shared across sexes), 3 standard deviations, and the deviations
form a Laplace random block of 126 rather than penalised fixed effects. Every deviation carries
gradient -- none is exactly zero, the smallest is 5.8 -- and moving one moves the objective.
Identifiability here is that argument plus those gradients; a simulation-recovery study would
be stronger and is not done.num_re_sexes
fields, which is 1 unless M1_model == 2, so one field is what M1_model = 3 owes.M1_re 1/4 and 3/6 vary by
AGE, and an age-varying deviation and an age-specific level enter the likelihood only as their
sum -- so the mode puts the deviations at zero and absorbs everything into log_M1. The
identifiable version of that model already exists as M1_model = 1 with M1_re = 4.
M1_model 4 and 5 are in .M1_DEPRECATED_MODELS and the linkage grammar expresses the same
model as linkage_spec(~ temp + (1 | Year)).?build_M1, vignettes/model-options-and-functionality.Rmd and
vignettes/environmental-linkages-and-priors.Rmd all said 3, 4 and 5 have no deviation
block. Corrected in the same commit.test-mortality-m1-re-age-specific-year.R: 3 blocks, 21 assertions, 2 failing on the
parent. A third block pins that the by-age families and M1_model 4/5 stay refused, so
opening one of those later means deleting an assertion and saying why.test-mortality-m1-re-unsupported.R pinned the refused set and so failed on this
change, correctly -- that is what it is for. The set is now 16 of the 42 pairs, measured
identically on a ragged-nsex and an all-one-sex dataset: M1_model 3 with M1_re 1, 3, 4 or
6, plus M1_model 4 and 5 with every family.A species whose numbers-at-age are input scored an M random-effect density over a field it
does not estimate. build_map_fixed_natage() maps an estDynamics > 0 species' whole M1
random-effect block out -- log_M1_dev, M1_dev_log_sd and M1_rho -- after
build_map_m1() runs, so nothing of its M is estimated. But the template's density is gated on
M1_re(sp) alone, with no estDynamics test in any of the three JNLL_M_RE blocks, so it
still evaluated N(0, exp(0)) over constant zeros and contributed n * log(2*pi)/2.
Measured end to end on BS2017SS (nyrs_hind 39) with species 1's numbers-at-age fixed,
M1_model = 1, M1_re = 2:
| | species 1 | 2 | 3 | objective | |---|---|---|---|---| | before | 35.838603 | 35.838603 | 35.838603 | 1333557.488469 | | after | 0 | 35.838603 | 35.838603 | 1333521.649866 |
The difference is 35.838603 exactly -- 39 * log(2*pi)/2. Species 2 and 3 keep theirs
because their deviations really are estimated; that is the density at the starting values, not
a degenerate constant.
It was a constant, so zero gradient and no parameter bias -- but it made the objective
incomparable with an M1_re = 0 fit of the same model, and broke the invariant that the
"M random effects" jnll row is exactly 0 when nothing is estimated. An AIC or likelihood
comparison across M1_re settings on a model with a fixed-numbers species was reading that
offset as information.
fit_mod() now treats M1_re as 0 for such a species and warns, naming it -- the same style
as it already forces M1_use_prior to 0 when M1_model is 0. Nothing shipping is affected: no
bundled dataset ships a fixed-numbers species, and estDynamics > 0 requires NByageFixed
data that none of them carries.
test-mortality-m1-re-fixed-numbers.R: 2 blocks, 6 assertions, 3 failing on the parent. No
bundled dataset has a fixed-numbers species, so the fixture is built in the test from a normal
fit's N_at_age -- which is also the first fixed-numbers fixture in the suite.
Time_varying_sel and Time_varying_q now default to "Off". Neither had a schema
default, so an absent column was not filled -- and both are among the three fleet_control
names with exactly one longer sibling, the one shape where $ partial-matches instead of
returning NULL. switch_check()'s conversion read Time_varying_q with a bare $ and
assigned the result back with $<-, so an absent column became a present and wrong one: on
GOA2018SS it came out 0.05, 0.01, 0.05, Off, ... where the real modes are
RandomWalk, Off, RandomWalk, .... fit_mod() then refused with "Invalid Time_varying_q
specified for fleets", blaming the caller for a value they never set. Time_varying_sel failed
differently -- inside a dplyr::mutate, naming the expression rather than the column. An
absent column means no time variation, so it is now filled with "Off" and announced, and
such a model fits.Sel_norm_bin was being silently defeated by the same mechanism, on
the line that exists to apply it. .rce_apply_default() returns early when handed a non-NULL
value, and the call read the column with $ -- so when Sel_norm_bin was absent it received
Sel_norm_bin_upper and the default never applied. Measured: with Sel_norm_bin_upper set to
7 and Sel_norm_bin dropped, switch_check() returned 7, 7, 7, ... instead of "Off".
Sel_norm_bin is an absolute age, so that is a different selectivity normalisation on every
fleet, which moves q and hence the advice. All three fills now read with [[.rearrange_data() reached a TMB input through the same partial match, and that one was a
wrong number rather than a wrong diagnosis. It is exported and does not call
switch_check(), so no fill can protect it; its .required_fc guard did not list
Sel_norm_bin; and the bare $ read returned Sel_norm_bin_upper. Measured on GOA2018SS
with Sel_norm_bin = "Max" and the upper column "Off": dropping the short name made
rearrange_data() succeed silently with every fleet's sel_norm_bin1 flipped from
-99 (normalize by the maximum) to -999 (do not normalize) -- a different selectivity
scaling, hence a different q and a different SSB. Sel_norm_bin is now in .required_fc, so
an absent column is refused by name with the guard's existing "Run switch_check() first"
message, and the read uses [[ so the guard is not the only thing standing between a partial
match and the template.Time_varying_q doubles as the env_data column index for a fleet whose Catchability is
"Environmental" or "AR1", where "Off" is not a meaningful value -- so the column is
defaulted only when no fleet needs it as an index, and otherwise refused with a message naming
those fleets. Without that, the fill produced "Off" for such a fleet and the model died later
on missing value where TRUE/FALSE needed, which is less diagnosable than what it replaced.switch_check() path; the ~50
bare $ reads of these three names elsewhere in R/ run after the column exists and are
unchanged. CLAUDE.md says not to sweep those, and this is not that sweep.fleet_control sheet (across the four consumer repos), the three inst/extdata workbooks and
write_template()'s output all carry the columns, so the default never fires for them.
Resolution with a column present is unchanged -- RandomWalk, Off, RandomWalk, Off on
GOA2018SS. Golden unchanged.GOA2018SS minus Time_varying_sel the fit builds with 539
free parameters against the 621 of the complete workbook, because fleet 8's
RandomWalkAscending deviations are no longer requested.test-schema-partial-match.R now pins the invariant that matters: an absent silent-prefix
column gets its schema default, not its sibling's values. For each of the three it marks
the sibling with a distinctive value, drops the short name, and requires the result to be the
default and not the marker -- which is what the old assertion could not see, because it checked
presence only and ran on BS2017SS, whose siblings are all 0/NA and convert to a
plausible-looking all-"Off". 7 blocks, 28 assertions; 4 fail on the parent -- three for
the schema defaults, one for rearrange_data().M1_re > 0 with M1_model 3, 4 or 5 is refused. Through fit_mod() that is 18 of the
42 (M1_model, M1_re) pairs -- M1_model has six levels, 0 to 5 -- measured identically on
GOA2018SS (nsex c(1, 2, 1)) and BS2017MS (all one-sex). A stored fit built with one of
them stops refitting: retrospective()'s forecast refit, self_test(), jitter(),
sim_mod(), project_no_f() and run_mse()'s EM refits all rebuild the map through
build_map(), while retrospective()'s peels, run_mse()'s OM, reweight() and profile()
pass the stored map and do not. Rebuild with M1_re = 0, or with M1_model 0, 1 or 2 if the
time-varying M was the point.M1_model = 0 fit with M1_re > 0 now fits a DIFFERENT model, silently. It
previously produced constant M; it now estimates the deviations it asked for. On BS2017SS
with build_M1(M1_re = "iid_year") -- nothing else set, so the defaults -- the objective goes
from 1,537,143.80 to 995,573.78 and log_M1_dev from 0 free parameters to 117. Nothing warns,
because nothing is wrong: the old number came from the degenerate density below and was not a
meaningful likelihood. An operational fit of that shape gave constant-M advice and will now
give time-varying-M advice -- refit deliberately rather than by accident.build_map() directly on a single-sex species with M1_model = 2
also refuses, for all six M1_re. fit_mod() never reaches it: it downgrades M1_model 2 to
1 for a single-sex species first, which is why the count through a fit is 18 and not more.M1_re > 0 asked for time-varying M and silently gave constant M wherever no random-effect
arm existed for the M1_model. Every family in build_map_m1() nests its deviation writes
inside an M1_model test, so where none matched, log_M1_dev stayed entirely NA while
M1_dev_log_sd was freed anyway, outside that guard. M1_model = 4 raised no condition at all,
and M1_model = 0 is build_M1()'s default -- so the most likely call of all,
build_M1(M1_re = "iid_year") with nothing else set, was affected.
The free standard deviation made the objective unbounded below. M1_dev_log_sd has no entry
in build_bounds(), so it keeps the generic [-Inf, Inf], and with every deviation mapped out
the density contributes n * (log(2*pi)/2 + log sigma) for an element count n. The gradient
is therefore exactly n: measured on BS2017SS the objective falls by 1755 * k for
log sigma = -k and is already -Inf at -500. An AIC or likelihood comparison against the
constant-M model prefers the inert configuration by an arbitrary margin.
The cost is per family, not one number. n is the number of ages for M1_re 1/4, of
hindcast years for 2/5, and of age-year cells for 3/6. Measured on BS2017SS at the starting
sigma = 1, jnll row 16 ("M random effects"):
| M1_re | n | row 16 per species |
|---|---|---|
| 1 / 4 | nages | 11.03 / 11.03 / 19.30 |
| 2 / 5 | nyrs_hind | 35.84 each |
| 3 / 6 | nages * nyrs_hind | 430.06 / 430.06 / 752.61 |
Rceattle-models/EBS pollock/2024/06-time-varying-M.R records 56.06 nats, which is
61 * log(2*pi)/2 -- the year family on a 61-year hindcast. That script worked around the
defect by telling the reader to avoid the combination; the package now handles it. Its comment
is now out of date and is a /ecosystem-sweep item.
M1_model = 0 keeps its deviations rather than being refused, and this is the half of the
change that adds a capability. It holds log_M1 at the input schedule, so there is no free
fixed effect for the deviations to be absorbed into. The template was already reading them:
ceattle.cpp's assembly is M1_at_age = exp(log_M1 + log_M1_dev + ...) with no M1_model
gate, so only the map was withholding them. Measured at M1_model = 0, M1_re = 2: log_M1
free count 0, 126 free deviations on GOA2018SS and 117 on BS2017SS, non-zero gradient,
and setting one deviation to 0.5 multiplies that cell's M1_at_age by exactly exp(0.5). The
three family blocks now read M1_model %in% c(0, 1); M1_model 0 and 1 give identical
free-deviation counts for every M1_re.
A canonical M1_re or M1_model string now resolves to its integer code in
switch_check(). The arms compare against integers -- "iid_year" %in% c(2, 5) is FALSE --
so a string mapped every deviation out with no message, for every M1_model. build_M1()
canonicalises its own arguments, but a list from build_data(), combine_data_sets(), the
deprecated est_M1 alias or built by hand does not pass through it. (Not a workbook:
read_data() coerces every control row to numeric and the control schema has no M1_model or
M1_re row, so neither field survives a write_data() round trip -- an earlier draft of this
entry said otherwise.) Measured: M1_re = "iid_year" goes from 0 free deviations to 2688 on
GOA2018SS; "none" stays exactly empty, which matters because "none" > 0 is TRUE
lexicographically.
Both switches go through .map_switch(), the resolver every other per-species switch in
switch_check() uses, rather than build_M1()'s stricter .coerce_M1_arg(). The strict one
refused a factor, a numeric-looking "1" and an NA -- shapes that worked before and that
test-switches-map-switch-factor.R exists to support, since
read.csv(stringsAsFactors = TRUE) produces them.
A fixed-numbers species is not refused. build_map_fixed_natage() maps an
estDynamics > 0 species' log_M1_dev, M1_dev_log_sd and M1_rho out after
build_map_m1() runs, so the free sd cannot arise there and refusing would reject a correct
model -- a fixed-numbers predator is the common multispecies setup.
The four random-effect families now agree on the two-sex requirement. M1_re 1/4's
M1_model == 2 arm lacked the nsex_sp == 2 & guard that 2/5 and 3/6 carry, so a single-sex
species took deviations in its padding sex slot: free, Laplace-integrated, scored by no
density (which loops sex < num_re_sexes, 1 there) and read by no cell (M1_at_age loops
sex < nsex(sp)). 22 exactly-zero-gradient rows in the inner Hessian on GOA2018SS,
unreachable through fit_mod() because of its 2-to-1 downgrade.
The refusal reads whether an arm actually fired, not a restated list of supported pairs, so
opening an arm legalises its combination with no second registry. It is && rather than &
because the right-hand side subscripts log_M1_dev and build_map() is exported, and it names
the M1_model -- sex only where sex is the reason.
Which of the 18 to implement is a modelling question, not a default to invent. M1_model = 3
with M1_re 1/4 is not identifiable: the age-specific fixed effect and an age-varying
deviation enter the likelihood only as their sum, so the mode puts the deviations at zero and
absorbs everything into log_M1; that model already exists as M1_model = 1, M1_re = 4.
M1_model 4 and 5 are .M1_DEPRECATED_MODELS, and the linkage grammar expresses deviations
around an environmental prediction as linkage_spec(~ temp + (1 | Year)). M1_model = 3 with
M1_re 2/5 is well posed and left open deliberately.
test-mortality-m1-re-unsupported.R: 10 blocks, 86 assertions. No golden reference moves: all
four run M1_model = 0 and M1_re = 0, where no family block executes and jnll row 16 is
exactly 0 in every column.
Species sp's male growth parameters were species sp+1's female growth parameters.
build_map_growth() indexed log_growth_pars with a fixed stride of 4 per species, but a
two-sex species needs 8 slots: females took (sp-1)*4 + 1:3 and males (sp-1)*4 + 5:7, so
5,6,7 served sp1-males and sp2-females -- one TMB parameter for both, estimated once. K, L1
and L-infinity were fused across a species and sex boundary, and 3 (von Bertalanffy) or 4
(Richards) free parameters went silently missing. Measured, growth on for every species:
| nsex | distinct levels / expected | parameters lost |
|---|---|---|
| c(1, 2, 1) | 9 / 12 | 3 |
| c(2, 2) | 9 / 12 | 3 |
| c(2, 1) | 6 / 9 | 3 |
| c(2, 2, 2) | 12 / 18 | 6 |
| c(1, 2) | 9 / 9 | none -- a trailing two-sex species is safe |
Now a running counter, as build_map_m1() already uses, so the index advances by what each
species actually took. Fitting a two-species model with nsex = c(2, 1) through
fit_mod(estimateMode = "DebugBuild") now gives 135 free parameters against 132
before, from 9 distinct growth levels rather than 6. The recovered parameters are informed, not
merely free: every growth entry carries a gradient on that fixture (4.2e+06 to 1.5e+08 on
log_growth_pars, 2.1e+05 to 6.3e+05 on the SDs) and no entry in the whole vector is exactly
zero, which matters because a freed parameter the data cannot move gives a singular Hessian and
newtonsteps solves with it.
build_params() no longer dies on CAAL bin counts that differ between species. L1 and
L-infinity start at the smallest and largest length with CAAL data, taken by pivoting on the
per-species bin ordinal -- so a 3-bin and a 5-bin species produced Bin1, Bin3, Bin5, three
value columns for a two-column target, with NA for whichever species lacked that ordinal. It
failed with "number of items to replace is not a multiple of replacement length". A species
with a single CAAL length failed differently and earlier: slice(c(1, n())) duplicated its one
row, pivot_wider() warned and returned a list-column, and log() then refused it with
"non-numeric-alike variable(s) in data frame". Both are now a per-species min and max, which is
what the slice was approximating.
build_params() now refuses a CAAL length range it cannot use, naming the species, rather
than starting L1 or L-infinity at NA. It also refuses one distinct CAAL length, where L1
equals L-infinity: length_sd_at_age() interpolates the length-at-age SD as
sd0 + (sd1 - sd0) / (linf - l1) * (len - l1) and both growth_log_sd start at 0, so that is
0/0 for every age above age_L1 and the NaN reaches the age-length key and the likelihood.
data_check() already refuses one CAAL length unless nlengths is also 1, so a species with a
single length bin is the one shape that reached this. The message says to use
build_growth(fun = "empirical") for that species instead.
Neither of the two defects was reachable by anything shipping: no bundled dataset sets
growth_model (the schema default is 0), all four golden references run 0, every multispecies
sibling config uses build_growth(fun = "empirical"), and only whamGrowthData carries CAAL
data at all. The suite does fit multispecies parametric growth, in test-dynamics-brps.R
and test-composition-dm-diet-weight.R, but make_msm_test_data() sets every species
single-sex, and the collision needs a two-sex species that is not the last one. So what hid
this was single sex, not single species. It arms the moment parametric growth goes on a
multispecies model with a non-trailing two-sex species, which is the shape of the GOA
multispecies assessment (arrowtooth is two-sex at species 2 of 3).
No golden reference moves: all four run growth_model = 0, where the whole growth block is
mapped out. Verified -- six golden blocks, 21 assertions, unchanged.
The starting residual M was diluted toward 1.0 per year whenever M1_model >= 1 met a
species set with ragged nsex. log_M1 is dimensioned to the widest species, so a one-sex
species in a model whose max(nsex) is 2 leaves padding sex cells, and build_map_m1() indexed
them: map_list$log_M1[sp, , 1:nages_sp] with a bare comma under M1_model 1 and 4, and
[sp, 2, ] <- [sp, 1, ] under M1_model 3, put the padding in the same map level as the
real cells. TMB does not take a shared parameter's start from the first cell of its level --
TMB:::updateMap() is tapply(parameter.entry, map.entry, mean) -- so the start was the mean
over the level, and a padding cell sitting at log(1) = 0 pulled it toward an M of 1.0 per year.
Measured on GOA2018SS (nsex c(1, 2, 1), nages c(10, 21, 12)) at M1_model = 1:
pollock started at 0.637454 against an M1_base geometric mean of 0.406348 (+57%, and
exactly its square root) and cod at 0.710571 against 0.504911 (+41%). Both now start at
their own input. M1_model = 3 started at the square root of every age in the schedule
(1.178983, 0.830662, 0.692820, ... for 1.39, 0.69, 0.48, ...) and now reproduces it exactly.
M1_model 2, 4 and 5 are affected identically and fixed with it. Species 2 was always correct:
its two real sexes legitimately share one level under a sex-invariant model, so
sqrt(0.20 * 0.35) = 0.264575 is the intended value.
Free-parameter counts are unchanged -- 3, 4, 64, 3 and 4 for M1_model 1 to 5 on that
dataset. Only which cells belong to a level changed, and the padding is now mapped out under
every M1_model. test-mortality-m1-map-padding.R fits all five through fit_mod() and pins
both the level count and the zero padding cells; an earlier version of it covered only 1 to 3,
leaving the environmentally-driven 4 and 5 edited but unpinned.
Whether it changes a converged estimate depends on the optimizer settings, and the honest
answer is "only without phasing". The shared parameter's MLE is identified by the real sex --
the template reads sex < nsex(sp) -- but an unphased optimizer does not always reach it from a
start displaced this far. Fitting GOA2018SS at M1_model = 1 cold, phase = FALSE,
newtonsteps = 0:
| | before | after | change |
|---|---|---|---|
| objective | 12849.030601 | 12838.508102 | -10.52 nats |
| M1, species 1 | 0.38983395 | 0.28725155 | -26% |
| SSB at endyr 2018, species 1 | 362,066 | 330,402 mt | -8.75% |
| hindcast mean SSB, species 1 | 315,995 | 307,862 mt | -2.57% |
Both of those fits are WARN, not converged -- max|gradient| 2.2e-03 and 1.7e-03 against
convergence_diagnostics()'s OK tier of 1e-03 -- so this is a comparison of two unconverged
points, and the surface carries at least four local minima over the range: a start of 0.90
reaches max|gradient| 8.4e-04 (the only one in the OK tier) at an objective 4.15 nats worse
and species-1 SSB 37% lower. Which optimum an unphased fit reaches is therefore arbitrary, and
that is the defect's real consequence rather than any single pair of numbers.
Under phase = TRUE the difference disappears entirely on this dataset -- both sides land
bit-identically at 12838.508102 with SSB 330,415 mt. Phasing is not a general escape, and the
live case is why that matters. The GOA multispecies assessment runs three fits
(R/02_fit_models.R): the fixed-M single-species one is phase = TRUE and M1_model = 0, so
it is immune; the estimated-M single-species one is phase = TRUE with
M1_model = c("sex_age_invariant", "sex_specific", "sex_age_invariant"); and the headline
multispecies fit sets that same M1_model with phase = FALSE -- the configuration in which
GOA2018SS moved 10.5 nats. A warm start does not help either: the dilution happens when
MakeADFun averages the map level, so it applies to whatever log_M1 the inits hold. One
benefit that is real at any setting: phase = FALSE, newtonsteps = 3 crashes on the old
diluted optimum (solve.default, singular Hessian) and completes after.
M1_model = 3 is a counterexample worth stating: unphased, the fix makes the objective 0.746
nats worse and moves species 2's endyr SSB +3.21%, with both sides in the FAIL gradient
tier; under phase = TRUE both land at 12727.17 and species 2 moves 0.004%. The direction of an
unphased change is not predictable, which is the argument for removing the displacement rather
than for any particular improvement.
No golden reference moves: all four run M1_model = 0, where the whole array is mapped out,
so none of them covered any of this. Verified: all six golden blocks unchanged, and 22 files of
mortality, M1, linkage-M1 and switch tests green.
Under M1_model = 2 a single-sex species also claimed a second map index for its padding sex
and then reassigned it without incrementing the counter, so the padding took the counter's next
value -- which is the following species' first level. On nsex c(1, 2, 1) species 1's
padding and species 2's real female both landed on level 3. That cross-species collision is real
in build_map_m1(), and is unreachable through fit_mod() only because fit_mod() downgrades
M1_model 2 to 1 for a single-sex species before the map is built. An earlier draft of this
entry said a "later re-levelling" prevented it and that the branch's warning fires; neither is
true -- the branch and its warning are dead through fit_mod(), and factor() preserves all
five levels. The fix also removes a latent subscript out of bounds crash when M1_model 2 or
3 meets max(nsex) == 1.
This also fixes, at its root, the NaN objective that 5.50.0 addressed from the other end.
fit_mod(updateM1 = TRUE) with M1_model >= 1 on a ragged-nsex set returned NaN, because
the -Inf that path left in a padding cell belonged to a real map level and so entered the
parameter vector. With the padding mapped out it cannot, and the objective is finite even
with the old fill still in place. On one stated recipe -- inits from a mode-3 fixed-M fit,
estimateMode = 3, updateM1 = TRUE -- M1_model 1, 2 and 3 give NaN, NaN, NaN before and
843787.106345, 842894.271820, 919429.003512 after, with 53 non-finite cells still in the array
both times. The objective on this path depends on which fit you warm-start from, so the figures
are only meaningful against that recipe; the transition from NaN to finite is the claim. The
figures quoted under 5.50.0 (891822.517895, 890929.683370, 933405.044463) are the same recipe
on the old map with the fill fixed instead. The two
changes are complementary rather than redundant: 5.50.0 stops the array holding a non-finite
value at all, and adds the refusals; this one stops a padding cell being a parameter.
Who was exposed. The defect needs a multispecies model that mixes one-sex and two-sex
species, since a uniform nsex leaves no padding at all. Of the bundled datasets only
GOA2018SS qualifies (nsex c(1, 2, 1)), and single-species models are immune whatever their
nsex. The live GOA multispecies assessment is in the affected configuration: its 2025 workbook
has nsex c(1, 2, 1) and R/02_fit_models.R sets
M1_model = c("sex_age_invariant", "sex_specific", "sex_age_invariant"), so species 1 and 3 are
affected and species 2 is not. Of its three fits, the fixed-M single-species one runs
M1_model = 0 and is immune, the estimated-M single-species one is phase = TRUE, and the
headline multispecies fit is phase = FALSE -- the setting under which GOA2018SS moved 10.5
nats and 8.75% of species-1 SSB. Its own fit was not refitted; that was a deliberate decision,
taken when this entry still said phasing made the point moot, which for the multispecies fit it
does not. Whether its numbers move is unmeasured, and nothing above licenses an expectation
either way: on the same data M1_model = 2 moved by 5.7e-06 while M1_model = 1 moved by 10.5
nats, so it turns entirely on whether that surface has a nearby worse optimum.
The same bare-comma pattern remains in the log_M1_dev, M1_beta, M1_rho and M1_dev_log_sd
writes and is deliberately left alone: every one of those starts at 0, so averaging a padding cell
into the level is exactly inert, and changing them would alter random-effect map structure for no
numerical gain.
log_M1 no longer carries a non-finite starting value. The array is dimensioned to the
widest species -- [nspp, max(nsex), max(nages)] -- so a species with fewer sexes or ages
leaves padding cells, and the two fill paths disagreed about what went in them.
build_params() initialized at 1 but then read 1:max_age from the M1_base row, which is
legitimately blank past that species' last age; fit_mod(updateM1 = TRUE) initialized at 0, so
every padding sex cell became log(0) = -Inf. Measured on GOA2018SS (nsex c(1, 2, 1),
nages c(10, 21, 12)): 11 non-finite cells of 126 on the first path and 53 on the second,
of which 42 were -Inf. On BS2017SS (nages c(12, 12, 21)) both paths left 18. Both now
read only the sexes and ages the species actually has, and both then run the same two fill
helpers, so they produce the same array. Padding does not stay at the initial 1: it mirrors
its own species' M1 (see below).fit_mod(updateM1 = TRUE) with M1_model >= 1 on a species set with ragged nsex. Those
padding sex cells take a map index (build_map_m1() writes [sp, , 1:nages_sp]), so a -Inf
in them reached the AD tape and poisoned the objective. On GOA2018SS: M1_model 1, 2 and 3
all gave NaN and now give 891822.517895, 890929.683370 and 933405.044463. Everything else
is unchanged: all six golden blocks, and a cold GOA2018SS fit at estimateMode = 3
reproduces 919429.003511761 (msmMode = 0) and 942099.593383157 (msmMode = 1) to all fifteen
digits with max|gradient| unchanged, before and after. Golden blocks 2-6 are the strongest
check, because they cold-build BS2017SS at estimateMode = 0 -- whose padding held 18 NA
cells -- and still reach their pinned objectives through a full optimization; block 1 alone does
not exercise the fill, since it warm-starts from reference inits that already carry the
non-finite cells.M1_base row is now refused. A (species, sex) the model has but M1_base does
not name kept the array's initial 1, i.e. a residual M of 1.0 per year -- inside
build_bounds()'s [log(0.001), log(2)], so nothing downstream caught it and the fit simply
ran with it. Deleting a species' row from BS2017SS previously built a model with
M1_at_age = 1.0, silently.TMB:::updateMap() is tapply(..., mean)), so a 1 there pulls an estimated M1 toward
1.0 per year. One helper does this for build_params() and the updateM1 path, so the two
fills cannot drift -- that duplication is what produced the array(1) / array(0) divergence
in the first place. It runs after the linkage pass, so the padding mirrors whatever the real
cells hold once a linkage has supplied the level.log_M1 starting value is also refused, naming the offending
(species, sex, age) cells. The check runs after the linkage initial-value pass, not at the
fill: an M1 linkage carrying an init for its intercept writes the level over every real age,
so a workbook may legitimately leave M1_base blank and let the linkage supply it. Checking at
the fill would refuse that, and did in an earlier draft.M1_base cell at a real age now warns and carries the last
supplied age forward, where before the NA reached MakeADFun() -- data_check() has no
M1_base completeness check. One real example exists: AI cod - Dev/Data/2024_AI_cod.xlsx in
Rceattle-models gives nages 13 and fills only to Age10, so three real ages were NA.
Carrying forward is what the padding fill already does and the only value the workbook
implies; an earlier draft of this change refused instead, which would have stopped that
model fitting at all -- and, because fit_mod() builds a build_params() skeleton even on
the inits branch, would have taken retrospective(), self_test(), refit_like() and
run_mse() with it. The warning names the species, sex and ages, and says that under
M1_model = 3 those ages are estimated from that start. Of 375 sibling workbooks (217 with
an M1_base sheet) that is the only one affected, and no bundled dataset is. ?build_M1
states the requirement.fit_mod(updateM1 = TRUE) keeps the species / sex / age dimnames on log_M1, so
initial_params is as readable on that path as on any other. With the fill now identical on
both paths, the labels were the only remaining difference.M1_base past a species' last age, those padding cells are
unchanged, because the padding fill mirrors the species' own oldest real age and the
workbook already supplied that same value. On GOA2018SS that is nine cells for species 3,
whose row runs to age 21 against nages 12 at a constant 0.504911; they held
log(0.504911) before and hold it after. An earlier draft of this entry said they now carry
0, which described neither state. Nothing reads them in any case.Three adjacent M1 defects found during review are filed in inst/dev/CLEANUP_BACKLOG.md rather
than fixed here, because each needs its own verification: the starting M1 is diluted toward 1.0
per year under M1_model >= 1 on a ragged-nsex set (build_map_m1() shares padding cells into
a real map level and TMB:::updateMap() averages the level -- +57% on GOA2018SS pollock,
and identical on 5.49.6, so pre-existing); fit_mod(updateM1 = TRUE) bypasses the bounds check
entirely; and the linkage intercept-prior re-target reads log_M1 without a clamp.
source-guards.yaml's per-block check could not see a namespaced test_that( block.
source_reading_blocks() matched only the bare spelling, so 30 blocks across five of its
own sixteen files were invisible to the one check that exists to catch a silently skipped
guard. Four are written entirely as testthat::test_that( -- test-linkage-encode.R,
test-linkage-selectivity-apical.R, test-schema-jnll-rows.R,
test-schema-quantity-dictionary.R -- and the fifth, test-schema-cpp-dispatch.R, mixes the
two: its sixth block is namespaced and reads selectivity.hpp, so that file's cover goes 4
to 5. (An earlier draft of this entry said 29 blocks across four files, having missed the mixed
file -- which is the one the job header cites as the motivating case.) Proven before fixing:
adding a skip() to a block in test-schema-jnll-rows.R left the job reporting "all passed",
exit 0.blocks=), because it is not uniform. A block
is held to the strict rule -- asserted something, did not skip -- only when its own body matches
a discovery pattern. Three files read source through a top-level helper instead
(test-likelihood-caal-afsc.R, test-schema-quantity-dictionary.R, test-vignette-api.R), and
those blocks are now listed too, as (+n via helper), under a weaker rule: running and
asserting nothing is a failure, skipping is not. It has to be weaker, because the job runs at
NOT_CRAN=false by design and a skip_on_cran() fit block in one of those files skips
legitimately.skip() added to its last
running block passed the discovery check, passed the staleness guard, and the job printed "all
passed" -- because the no-per-block-cover list was reported and then left out of the stop()
condition. A file that asserts nothing has not exercised its source read, wherever the read
sits.INCIDENTAL_BLOCKS exemption is now refused. All three entries named block labels
that do not exist -- test-likelihood-caal-afsc.R's blocks had been renamed -- so the
exemptions matched nothing while the job still printed "3 exempt blocks". The labels are
corrected and every entry must now resolve to a real block in a real target, or the job stops
and names it.skip() to both blocks of
test-vignette-api.R exits 1 with "test-vignette-api.R (every block skipped)". The clean tree
exits 0. A file whose every block is allow-listed is exempt from the whole-file rule by
declaration -- that is test-likelihood-caal-afsc.R, whose three blocks all need a fit this
job deliberately does not build.This is the third instance of one pattern in as many days -- a guard measuring less than it
claims -- and the second and third inside the machinery built to catch it. The first was the sixteen guards
of 5.49.5; inst/dev/TRAPS.md has the class.
test-plot-smoke.R's last block tested 17 plotters by name, ran 15, and omitted 14 that
exist. Two names were not exported functions -- plot_catchresidual, which no version of
this package has had, and plot_sel_vs_mat for plot_selectivity_vs_maturity -- and an
if (!exists(fn)) next skipped both silently while an expect_gt(ran, 10L) floor had slack
to spare. The list is now derived from getNamespaceExports() rather than written out, so
a plotter cannot be omitted by being forgotten. 31 exported, 4 excluded with a stated reason,
27 exercised.plot_comp alone returns 15, plot_diet_comp 9 -- and inherits(p, "ggplot") was
FALSE for those, so the ggplot_build() gate silently skipped them. It covered 16 figures of
49. A broken plot_comp panel passed; it now fails.exists(). exists() defaults to
inherits = TRUE, so it resolved anything reachable from the namespace -- including base
functions and non-function objects -- and un-exporting a plotter left the block green. That is
a hard-rule-1 regression the block is named for catching. It now fails.plot_m_at_age and plot_m2_at_age_prop refuse on a minage != 1 model, naming the real
age ranges, because both default to age 1. That is precisely the defect this block exists to
catch and neither was ever in the list; the refusal is now asserted rather than the plotters
skipped. plot_form's Kinzey & Punt refusal is asserted the same way.R/ and src/TMB/; about twelve
narrated a past defect or a past version in the past tense, and each now states what the code
does, or states the counterfactual that explains why a guard is written the way it is. The substantive ones: .refit_converged()'s
roxygen spent fourteen lines on a superseded test against opt$Convergence_check;
build_map_catchability() and .fleets_with_index() each described the wrong Fleet_type
keying they replaced; fit_mod()'s skeleton filter narrated the MakeADFun error a stale
inits name used to cause; run_mse()'s seeding comment explained what the old positional
keying broke; and data_requirements_table.R opened on how data_check() used to hard-code its
conditions..rce_deprecate_warn()'s roxygen said the deprecation warning "turns on in 5.13.0 by flipping
this to TRUE" -- it is still FALSE by default at 5.49.5, 36 minor versions later, so a reader
concluded warnings fire when they do not. It now documents the default and the option
(Rceattle.warn_deprecated_args) that turns them on. And the reference-point recruitment comment
in ceattle.cpp described in the past tense a condition that is currently broad on purpose;
it now says why narrowing it would leave SB0 decaying, which is what HCR 5 and 6 read as the
depletion reference..refit_converged() now names the string it tests and records that under
getsd = FALSE Convergence_check is still set, but only ever to one of two gradient verdicts,
neither of which is that string -- which is why a run that ended at a maximum gradient of 1e13
is kept. That branch still reads the field, so the fact is load-bearing.plot_indexresidual() keeps one sentence of its sign-convention note: index residual
figures made before 5.9.0 are mirrored about zero relative to these. A reader comparing last
year's figure is the person at risk, and they are reading the help page, not NEWS.md. The
fuller account stays in the 5.9.0 entry.osa_residuals() said OSA
residuals work "with or without fit_control(osa = TRUE)" -- but fit_control() has no osa
argument and no ..., so that call errors; it was removed as a breaking change in 4.6.0.
They now say what is true: nothing has to be set at fit time, and any fit optimized at
estimateMode < 3 will do, which the guards above them enforce. One in run_mse() claimed a
divergence "does not displace the assessments after it", which overstates incomplete common
random numbers; it is scoped to the random-number stream, which is what the year-keyed seeding
actually fixes.R/0-rceattle_class.R,
R/0-tmb_helpers.R, R/6-fit_mod.R, R/7-plot_helpers.R, R/8-sim_mod.R,
R/9-self_test.R). Three of the six are not @noRd, so the rendered help really did start a
line with a comma -- visible in the fit_mod, dot-model_labels and
print.Rceattle_selftest diffs.Kept deliberately, because the rule's exceptions apply: comments explaining why an older input
path still executes are behaviour, not history -- a fit saved before 5.32.0 can still store a
retired srr_fun code, log_pop_scalar from before 5.35.0 was shaped by age, and
log_catch_sd / log_index_sd are still written as deprecated aliases by rename_output(). The
ceattle.cpp section 16 change log is a change log. Literature citations are the specification.
Comments on the accepted-and-ignored base-graphics plotting arguments must say they are ignored.
Regression-test provenance stays in its test header block, which is where two of the deleted facts
already live.
Three items went to inst/dev/CLEANUP_BACKLOG.md rather than being swept, because none is
safely history. A stranded Fixme: denominator is zero somewhere. Log of negative number. Check suitability. inside the ceattle.cpp change log, which change-log item 23 may have resolved but
nothing says so, and which would be a silently-wrong M2 rather than a crash if it is live -- filed
with the note that the marker is lowercase, so this file's own case-sensitive re-derive command
does not count it. And two separate LEGACY (scheduled removal: v4.5.0) markers, still live:
one on M1_mult.sum(), and one on the dynamic reference-point recruitment split, which feeds
SB0(sp, nyrs-1) and so wants a deliberate decision rather than a sweep. These are two different
paths, not one -- an earlier draft of the entry counted the grep hits and conflated them.
case
labels in ceattle.cpp, the column schema against R/data.R, the JnllRow enum against its
two R-side partners, the HCR-2 threshold against the literal source of run_mse() -- by reading
R/*.R and src/TMB/*.cpp off disk. Where they ran before, precisely: under R CMD check they
resolve ../../R and ../../src/TMB relative to the .Rcheck test directory, where neither
exists, so each skips and a skip is not a failure; test-coverage does run them against the
real source but passes stop_on_failure = FALSE with nothing reading the result;
deep-checks.yaml runs them properly and fatally, but nightly. So the gap was never that they
never ran on a PR -- it is that their result was discarded, or arrived a day late.source-guards.yaml runs them on a plain source checkout, pkgload::load_all() without the
TMB build, in about five seconds of R. It fails on three conditions, because two of the three are
how a guard dies quietly: a guard failing, a guard returning no result rows, and a
source-reading block that skipped or asserted nothing.skip_on_cran() heuristic,
then by a per-file passed > 0 floor -- and adversarial review demonstrated the identical hole
in both: most of these files hold several test_that() blocks, so one can be switched off while
its siblings keep the file's pass count high. Adding a skip_on_cran() is the likeliest
accidental edit in this repo (it is the idiom in 181 of 255 files), and doing it to the block
guarding the C++ dispatch map took 33 assertions dark with the job green.test-schema-cpp-dispatch.R's not-yet-implemented inventory -- the pins that non-parametric
growth is still stubbed and that the Kinzey & Punt predation block has no live msmMode branch
-- sat behind a skip_on_cran() that existed only for the fixture half of the block. The block
is split: the static reads run always, the fixture half still skips. That file's assertion count
goes from 114 to 117.length() == 16,
and a review showed that deleting a real guard while adding one decorative test that happens to
match a discovery pattern keeps the count at 16 and the job green; only 5 of the 16 are named
anywhere else, so 11 could have gone silently. The failure now names which guard left and which
arrived.NOT_CRAN=false is set as step-level env, per inst/dev/TRAPS.md, and the script asserts
the value it actually received -- setup-r-dependencies writes NOT_CRAN=true to
$GITHUB_ENV, and with it set this job produces eight confusing failures with nothing naming
the cause.deep-checks.yaml header had stale
or missing: 181 of the 255 test files carry skip_on_cran() (it said 140 of 192), and 94
of those carry it at file level, outside any test_that() -- which produces a zero-row result,
so such a file reports neither a pass nor a skip and disappears from the counts rather than
showing up as skipped.Note that adding the workflow makes the check appear on a pull request; making it required is a branch-protection setting, not something this repo holds as code.
test-schema-partial-match.R. $ partial-matches on a list and on a data frame
silently, and three schema columns have exactly one longer sibling, so a $ read of an absent
Time_varying_sel, Time_varying_q or Sel_norm_bin returns the sd or the upper bin
instead of NULL. The option that would warn, warnPartialMatchDollar, is off by default. The
new file turns it on locally -- not for the whole suite, where it fires inside readxl, dplyr and
TMB for names this package does not own -- filters warnings to names the schema or a data list
owns, and asserts data_check() makes no such read on eight bundled data sets. It carries a
positive control and floors so it cannot pass vacuously.$ reads are safe, because it is not obvious and it is fragile.
There are ~50 bare $ reads of those three names against 2 [[ ]] reads, and none is reachable
with the column absent: switch_check() and data_check() reach the column through the dplyr
pronoun .data$, which errors on a missing column rather than partial-matching, so the
pipeline dies loudly first; where it does not, switch_check() has already supplied the column.
The protection is therefore a property of which accessor comes first. Rewriting a .data$ read
as a bare $ would remove the loud failure and expose the silent one. CLAUDE.md now says so,
and the new test pins the loudness.R/, one row each. It named 21 of 67 as files;
the rest were described nowhere a reader would find them, which is most of what someone has to
navigate to make a change. The new "Every file in R/, and what it does" section covers all 67,
grouped by purpose rather than number prefix, since the prefix tracks pipeline order only for
the 1- to 6- chain and the 31 files under 0- are shared machinery.quantity_dictionary() (quantity, meaning, units, dimensions),
parameter_dictionary() and parameter_index(). Also records that the quantities older scripts
call NByage and biomassSSB are now N_at_age and ssb.inst/dev/, data-raw/, tools/verify/ and .claude/commands/ to the repository layout
table — all four are cited in the guide's prose and none was listed.adding-a-selectivity-form article and inst/dev/CONTRIBUTOR-EXPERIENCE.md all told a reader to
add a case to switch (sel_type) in selectivity.hpp. There is no such switch: it is
switch (sel_case), where sel_case is the Selectivity code unless form 13 sits under a
random walk, which rewrites it to -13. test-schema-cpp-dispatch.R has had this right all
along — it dispatches on sel_case with a comment saying so — so the code and its guard were
correct and only the documentation was wrong. Two of the three also cited a stale line number.switch_check() fills defaults through 23
hand-written calls, not a loop over the schema, so a column with a default needs a 24th;
meta = TRUE is the constructor default and R/data.R's @format is pinned to the schema by
test-schema-canonical.R; and meta_data_names.xlsx is committed, regenerated by
data-raw/regenerate_meta_xlsx.R. The schema's has_default is wired for fleet_control only.DoubleNormalSS3) to the TMB module table, which stopped at 13 and
so implied 15 was free. It is not; the next free code is 16.0-column_schema.R entry, which said its doc becomes the ?BS2017SS field text.
That text is hand-maintained in R/data.R and only asserted to agree. Corrected the
McAllister–Ianelli attribution: the weight calculation is in 6-rename_output.R, and
6-reweight.R iterates it. Corrected 0-parameter_index.R, which is the general
parameter-to-coordinate map rather than a selectivity-specific one.R/ with a
substantive description; every R/ filename the guide names exists; and every C++ switch (var)
the guide quotes exists in src/TMB/. The last is new and is what would have caught the
sel_type error — the earlier guards checked filenames, and the filename was right.
Deliberately not checked: whether a description is correct. Word overlap against the named
file was tried and rejected, because the large files mention nearly every concept and 17 of 67
accurate rows scored higher against some other file than their own.man/build_srr.Rd, which had drifted from its roxygen in two passages: a new
paragraph distinguishing the linkages key R_init (a unitless log-scale multiplier on the
initial age structure) from the reported quantity R_init (equilibrium recruitment at F = Finit,
in thousands of fish), and a rewritten sentence in the starting-values section.No change to any fit. This release only deletes code that was already dead, so every objective, parameter and reported quantity is unchanged.
src/TMB/Dev/ (wham_v0.cpp 2,715 lines, caal.hpp, osa.hpp, two CAAL simulation
scripts). It was never compiled: src/TMB/compile.R compiles the single translation unit
ceattle.cpp, neither Makevars descends into it, no #include names it, and
.Rbuildignore excluded it from the tarball. The now-dangling ^src/TMB/Dev$ line was removed
from .Rbuildignore.build_params(), build_map_predation(), build_bounds(), set_phases() and plot_form() —
171 lines of comments for six parameter blocks (logH_1, logH_1a, logH_1b, logH_2,
logH_3, H_4) that the TMB template does not declare and data_check() refuses
(msmMode 3-9). build_map_predation() goes from 101 lines to 35, of which 56 were one
commented block. plot_form() still exists and still errors, naming the modes that are
available; that error is behaviour and was kept. The starting values, bounds and the citation are
recorded in inst/dev/REMOVED-kinzey-and-dev-cpp.md, and the /* */ blocks in ceattle.cpp
are deliberately left for a C++-side commit..save_par(), an unexported helper with no call site anywhere, whose docstring
described a contract ("calling .save_par() at the top of each plot function") that no plot
function honoured. Seven other helpers that looked unreferenced were checked and kept: all are
live -- two reached only from tests (.is_timeout, .par_description), one from a data-raw/
script and a test (.rce_build_meta_data_df), and four passed as function values rather than
called (.okabe_ito_pal, .rce_sel_se, .depletion_reference_lines, .f_reference_lines).build_params() 3.3/3.4 to
3.2/3.3; build_map_predation() 4 to 2), and re-derived the CLEANUP_BACKLOG.md marker count,
which had drifted: 47 now, from 56 on 5.49.1.lp == 0 is also the last bin, and the lp == 0 branch is tested first,
so fill_age_length_key() read lengths_pop(sp, 1) on a grid whose only valid index is 0. The
model builds safebounds = FALSE, so that was a silent read of adjacent memory rather than an
error. A single bin now takes the whole length distribution, which is what it spans. Found by a
review that re-read the same function after the 5.49.0 fix landed -- the midpoint guard made the
file look done.Four statements in src/TMB/ceattle.cpp that were wrong about the code, so a reader could not use
them: sel_init is filled in section 5.12.1 rather than 6.5 and is read to build Z_init and
hence SPRFinit (the section number given for that was also wrong, so it now names the quantity
instead); Time_varying_q 1 and 2 are IID and AR1 per tv_q_map, not "penalized deviate" and
"random effect" -- whether the deviations are random effects is fit_mod(random_q = TRUE), and 2
was retired in 5.16.0 and is refused by data_check(); and the initial equilibrium catch is in
weight or numbers per the fleet's Observation_units, as the loop's own flt_units == 2 branch
and quantity_dictionary() both say, not kilograms.
Review findings on the 5.48.0 release (PR #184), fixed before the tag. Nothing here moves a
fitted number on a configuration that was already valid: the four golden references reproduce
their pinned objectives, the Pacific hake 04-mse.R chain reproduces all four of its stages to
every digit before and after these changes, and reading every .xlsx in the four consumer repos
through read_data() + data_check() gives the same verdict on all 375 as 5.48.0 does -- which
matters here because two of the changes below are new refusals and two loosen existing ones.
Two configurations that could not fit the model they named are now refused. Both are unreachable on a model that was converging, so a stored fit only stops refitting if it was already producing a non-finite objective or a prior on the wrong scale.
An initial equilibrium catch on a species with estDynamics > 0 is refused. The
equilibrium age structure N_eq is built only where the numbers-at-age are estimated
(src/TMB/ceattle.cpp, section 6, case 0 of the estDynamics switch). Under
estDynamics 1 or 2 the numbers come from NByageFixed, N_eq stays zero, and the
Baranov prediction is zero -- so the lognormal took log(0) and the objective was not
finite, which reads as a failed optimisation rather than as a configuration error.
build_map() also maps log_Finit out on such a species, so the row was fitted by
nothing in any case.
A lognormal, gamma or beta prior on a DoubleNormalSS3 slot other than dn_peak
is refused. Of that form's six parameters only dn_peak is stored on its natural scale
(a length, cm). top_logit, start_logit and end_logit hold logits and ascend_se /
descend_se hold logs, so a family with positive support was evaluated on the transformed
value: lognormal on a negative logit gives NaN, and on ascend_se it is a prior on
\eqn{\log\log(\text{width})} rather than on the width -- silently a different prior
than the one written. A normal prior on the stored value is well defined and is still
accepted. This extends to form 15 the refusal DoubleNormal's right_floor already
carried.
A length-based parametric selectivity on a finer population grid fits again.
data_check() refused any length-based fleet on a species whose population grid is finer
than its data grid when Time_varying_sel was on or N_sel_bins was positive. Neither is
a bin ordinal on a parametric curve: sel_inf_dev and log_sel_slp_dev are added to the
parameter, which is a length in cm, and N_sel_bins is read only by the bin-indexed
forms. The refusal now names the bin-indexed forms and the columns that really are data-bin
ordinals (Bin_first_selected, Sel_norm_bin, Sel_norm_bin_upper, Sel_pen_first_bin,
Sel_pen_last_bin, Sel_cap_bin). This was refusing the SS3-bridge configuration the
population grid was added to serve -- a size-selective fleet with time blocks.
A selectivity linkage on an integer-coded Selectivity column fits again.
.check_sel_linkage_support() compared the column to form NAMES as a string, so a workbook
storing Selectivity as 1 or 15 was refused with "whose form (15) is not yet wired for
linkages" -- a form that is wired. It now reads the column through the schema's own
resolver. fit_mod() canonicalises before this runs, so only the exported
build_selectivity() path was affected.
sim_mod(simulate = FALSE) now returns the expected initial equilibrium catch. The
simulate = TRUE branch redraws it (5.48.0), but the expected-value branch left it at its
observed value -- so self_test(simulate = FALSE) handed the estimation model the one real
observation beside expected values for everything else, and under
initMode = "FishedNonEquilibriumSelected" that row is the only thing informing Finit,
hence the initial age structure and the SSB scale.
data_check() and rearrange_data() resolve Ageing_error_index through one function.
Five copies of the "absent or blank means the row's own species" rule had drifted into two
different coercions: one went through as.character() and four did not, and as.integer()
on a factor-typed column returns the level code rather than the number typed in the
workbook. A factor-typed Ageing_error_index could therefore be validated against one
matrix and fitted with another. .rce_ageing_error_index() is now the single reading.
write_data() warns about the growth fields it cannot write. pop_lengths,
growth_sd_form, growth_plus_length, plus_group_decay, growth_sd_style and
growth_age_L1 reach data_list from build_growth() and have no workbook column, so they
round-tripped to nothing in silence -- the index_cov trap again. Losing pop_lengths is
the worst of the six: the grid falls back to the data bins, which is a different age-length
key, a different weight-at-length and a different length-selectivity grid, and a
configuration data_check() would have refused becomes one it accepts, because that
refusal is conditioned on pop_lengths being present. The warning names the fields, as the
model_config one already did.
A single-bin population length grid no longer reads outside the array. The midpoint of
the last bin takes the previous bin's width, so a one-bin grid read lengths_pop(sp, -1) in
growth.hpp and in the new selected-body-weight loop in ceattle.cpp. The model builds
safebounds = FALSE, so that was a silent read of adjacent memory, not an error. Both now
keep the edge itself, which is what selectivity.hpp already did.
NorthernRockfish2022 (31 combined-sex length rows, 50-age matrix). The existing
test runs on GOAatf, whose length-comp rows are all joint-sex and so exercise both
prediction loops together; no bundled fit with a non-identity matrix exercised a fleet that
is not joint-sex, where only the first loop runs. That branch is now pinned on its own.inst/dev/SPEC-equilibrium-catch.md section 5 described the initMode = 6 plus-group
divisor as an open defect; it was fixed in 5.48.0 alongside the equilibrium catch, and the
section now records that, with the line it lives on.vignettes/environmental-linkages-and-priors.Rmd
(which link warns on a natural-scale index family, SS3's power function raising vulnerable
biomass rather than the index, the q tensors' hindcast-only dimensions, which
convergence_diagnostics() check fires, the growth SD endpoints as linkage targets with a
base parameter, and BS2017SS being three species fitted in single-species mode),
vignettes/growth-estimation.Rmd, vignettes/model-diagnostics.Rmd and
vignettes/data-without-excel.Rmd (whose initMode list stopped at 5).vignettes/data-without-excel.Rmd attaches the package in its setup chunk. Its first
executable chunk calls write_template() and the narrative library(Rceattle) came
afterwards, so the vignette failed on its first line whenever the chunks were actually
executed -- which is why the weekly vignettes workflow had been red since at least
2026-09-07.CLAUDE.md and inst/dev/TRAPS.md carried the pre-5.48.0 jnll_comp row count (21) and
axis ranges in their short-form entries, and TRAPS.md claimed every (Intercept) linkage
coefficient is pinned, which R_init is not.A stock whose series begins after a regime shift starts at a recruitment other than
\eqn{R_0}, and Rceattle had nowhere to say so. The level had to go into init_dev,
which is penalised \eqn{N(-\sigma^2/2, \sigma_R)} per age, so the model paid a
recruitment-deviate penalty for sitting where the data say it sits.
That penalty biases the reference points. init_dev is
\eqn{\log N_{target} - \log R_{init} + \sum M}, so the optimiser can retire the penalty
by lowering \eqn{R_0} -- and does. On GOA Pacific cod it lowers it 0.37 log units, leaving
\eqn{R_0} 31% below the value the same data support, while terminal SSB moves only -1.35%. \eqn{SB_0}, \eqn{B_{40%}} and depletion
all scale with \eqn{R_0}, so a penalty on the initial state was moving the quantities
that set the catch limit.
R_init is a fourth recruitment linkage parameter, alongside R0, alpha and beta.
It is a log-scale multiplier on the initial age-structure, read at year 0 because the
initial state is one year, and it carries no deviate penalty. The spec has to travel
through build_srr(): fit_mod() overwrites data_list$srr_linkages from recFun, so
assigning one onto the data_list drops it in silence.
d$env_data <- data.frame(Year = d$styr:d$projyr, lvl = 1)
fit <- fit_mod(d, recFun = build_srr(linkages = list(
R_init = linkage_spec(formula = ~ 1, init = list(`(Intercept)` = 0.25))
)), initMode = "NonEquilibrium")
R_init is the only linkage parameter with no base parameter in rec_pars, so its
(Intercept) is the one that stays estimable rather than being pinned at NA. Its
init and bounds on that intercept are therefore the natural-scale multiplier on
\eqn{R_0} — init = 0.25 is a quarter, as every other intercept's init is
natural-scale — and are logged onto the coefficient in place of a base parameter. Only
year 0 is read, so one design column per species is all the initial state identifies.
Measured on GOA Pacific cod 2024, where Stock Synthesis carries this as an SR_regime
block on the year before the hindcast (\eqn{-1.3879}, a quarter of \eqn{R_0}, from the
bridging variant that sets that block's likelihood weight to 0; the production run
shrinks it and sits at \eqn{-0.678}, half of \eqn{R_0}): moving the level out of
init_dev drops the objective 49.91 nats, all of
it in the Initial abundance deviates row (54.975 to 5.062), with the index, catch,
length-composition and CAAL components unchanged to 1.9e-04 and the recruitment deviates
unchanged exactly. The stock-recruit curve is untouched -- \eqn{R_0} is the unfished
level and does not move with the initial state.
Refused rather than silently inert:
"log" (today only "identity" is reachable). The level is added
inside the exp(), so a natural-scale offset
would add metric tons to a log multiplier.init on the level that is not greater than 0, and a negative bound, since
both are multipliers on \eqn{R_0}. An unset init arrives as the table's default of 0
and reads as a multiplier of 1, i.e. no shift; a bound of 0 is allowed and logs to
-Inf, the same unbounded floor. Note that a lower bound above 1 needs an explicit
init inside it, or the default multiplier of 1 sits below the bound.initMode = "FreeParams", which estimates the initial numbers-at-age directly as
init_dev and never reads \eqn{R_{init}}.initMode = "OffsetEquilibrium", which already scales the same ages by
rec_dev[, 1], so the two are not separable.A prior on the level is read on its own coefficient, i.e. on the multiplier, so
lognormal(0, 0.5) gives the multiplier a prior MEAN of 1 -- no shift off \eqn{R_0} --
the same contract as the per-sex apical selectivity multiplier. Under the default
bias_adjust_proc = TRUE that puts its median at \eqn{e^{-\sigma^2/2}}; pass
bias_adjust_proc = FALSE to centre the median on 1 instead.
Under initMode = "FishedNonEquilibriumSelected" (6) the level scales every age of the
equilibrium age-structure, age 0 included, so the predicted initial equilibrium catch is
proportional to it.
A level fixed with est_phase = 0 survives a warm start: fit_mod() re-applies it over
any inits, with or without an explicit init (the table's default of 0 is a
well-defined multiplier of 1).
Shrink or bound the level on a real assessment. Stock Synthesis penalises its
equivalent — 0.5 * (log(R1/R1_exp) / (sigma_R / ave_age))^2, at a default likelihood
weight of 1 (SS_objfunc.tpl) — and no reference package surveyed (SS3, SAM, SPoRC,
OPAL, WHAM) leaves an initial-level scalar free. Rceattle leaves it free by design, which
reproduces the zero-weight variant rather than SS3's default; a lognormal(0, sigma_R / ave_age) prior on the (Intercept) expresses SS3's convention under
bias_adjust_proc = FALSE, which is the setting that centres the prior's median on a
multiplier of exactly 1, as SS3's penalty is.
This matters because the level is informed only by the first year's observations and is
weakly identified at the low end: once the initial cohorts are effectively annihilated,
the data cannot separate one very small level from another.
tools/verify/verify-sim-recovery-r-init.R measures, on its own fixture at a true
multiplier of 0.5, a few replicates in 60 running away to 5e-05 or below with a clean
gradient — no convergence filter finds them — plus a low bias of about 0.12 log units
over the rest. Four starting levels return the same estimate to four decimals at the same
objective (inst/dev/TRAPS.md), so these are not a local optimum a multi-start or
jitter() escapes: the data really do prefer the annihilated state. The counts are
machine-dependent — the harness has returned both 3 and 6 of 60 on identical arguments —
so re-measure rather than quoting them. A natural-scale lower bound above
0 keeps a fit out of that tail; a lower bound of exactly 0 logs to -Inf and does not.
A model with no R_init linkage is unchanged, including all four golden references.
A length composition built from an age composition used age_obs_hat, the composition
already smeared by the ageing-error matrix. The age-length key is P(length | true age)
and a measured length carries no otolith reading, so the smear has no part in it; Stock
Synthesis likewise applies its age_age matrix only to age and CAAL data, WHAM only to its
age and CAAL predictions, and SPoRC passes AgeingError = NA for a length composition.
Now built from the true-age composition, in both of the loops that predict one: a joint-sex
composition is assembled as females over the first nlengths bins and males over the next,
then normalized across both halves. Every length-comp row in the affected datasets is
joint-sex, so the second loop is the one they exercise.
This moves any fit with a non-identity age_error and length comps on the same species,
which is four bundled datasets and the external SS3 cod-bridge workbook (bundled GOAcod is
not among them — its matrix is the identity). The two GOA golden
references were re-pinned; BS2017SS/BS2017MS have an identity matrix throughout and are
unchanged, as is GOApollock, whose matrix is non-identity but which fits no length comps.
Refitting lowers every affected objective and moves the scale by a fraction of a percent:
GOAatf -0.9416 nats and terminal SSB +1953 mt (+0.223%), NorthernRockfish2022 -2.9022
nats and +14 mt (+0.019%), GOA2018SS -1.1445 nats and +1678 mt on species 2 (+0.161%).
It had no SIMULATE block, so sim_mod(), self_test() and run_mse() kept the real
observation while redrawing everything else. That observation is the only one
informing Finit, and therefore the initial age-structure and the SSB scale under
initMode = 6 — so a self-test conditioned every replicate on data its own operating
model had not generated, and reported the resulting Finit bias as if it were real. Drawn
from the same lognormal as its density, reported as equil_catch_obs_sim, and written back
into equil_catch_data.
Two problems on the population length grid:
data_list$pop_lengths skipped validation. ?build_growth documents it as the
inherit source, so a converter or an SS3 bridge can set the field directly and never
reach build_growth()'s validator. A decreasing grid returned a finite objective
with age-length-key probabilities down to −0.79, and a negative edge a negative
weight-at-length, neither flagged. Validated on the read path now.len_bins_m), but calculate_selectivity() took a
single width from bin 1, so on a non-uniform grid — which SS3 models routinely use, with
wider tail bins — a curve was evaluated up to 1.25 cm off the bin it labels and
disagreed with the probability it multiplies in the same expression. Now per-bin
midpoints, with one scalar width for DoubleNormalSS3's peak2 exactly as
SS_selex.tpl:153 uses binwidth2. Identical on a uniform grid, which is every bundled
dataset, so no existing model moves.Both were found by reviewing 5.46.0's new features against the Stock Synthesis source.
Ageing_error_index's fallback is now validated. Omitting the column, or leaving it
NA, uses the fleet's own species number — which is right while age_error is indexed by
species, and wrong as soon as it carries its own Ageing_error_index column, because the
fallback then points at whichever matrix sits at that position. data_check() validated
only a supplied index, so the fallback could hand a fleet another species' ageing error,
over another species' age range, silently: measured at 4,718 nats on GOA2018SS with
two ageing eras on species 1, and at 94,000 nats where the fallback index had no rows
at all and the smeared composition went flat. The effective index — the one
rearrange_data() uses — is now checked to exist and to belong to the fleet's own species.
A DoubleNormalSS3 end parameter below −1000 is refused. Stock Synthesis reads
P5 < -1000 as "selectivity nil through bin -1001 - P5" and P6 < -1000 as "constant
beyond bin -1000 - P6, and anchor the descending limb there" (SS_selex.tpl, pattern
24) — different curves, and a sub--1000 P6 also moves peak2. Rceattle implements
neither and honoured only -999, so a bridged SS3 control file using either encoding fit
a different selectivity with no message.
A pinned (Intercept) coefficient is 0 by construction: the process's own base
parameter carries the level and the row is mapped out of estimation. fit_mod() did not
re-impose that over supplied inits, and a slope-only spec produces a beta_linkage of
the same length as an intercept-only one, so the length guard passed and a stale
fitted slope was held on the mapped-out row for the whole fit. Refitting M1 = ~ 1 from
the inits of an M1 = ~ 0 + temp fit put natural mortality at 6.69x its intended
value, with no warning and nothing in convergence to show it.
The coefficient is now re-zeroed before the fit. This affects every process the linkage
grammar covers, not just recruitment. A genuine refit — retrospective(), jitter(),
self_test(), model_average(), run_mse() — already carries 0 there, so it is a
no-op for those, and leaves all four golden references bit-identical.
A fourth linkage link, for catchability only, completing the progression on the
natural-scale parameter: "identity" adds to it, "log" multiplies it,
and "exponential" raises it to a covariate-dependent power, q^exp(beta * x):
index_q_yr = exp((log_q + q_offset(yr)) * exp(sum beta * x(yr)) + q_dev(yr))
+ q_nat_offset(yr)
The name is SS3's own for it -- case 1: // exponential env link -- and is
deliberately not "power": SS3 has a separate q power function
(Q_setup option 3, pow(vbio, 1 + p), SS_expval.tpl:429-436), which is what
Catchability = "PowerEquation" and the dormant index_q_pow are reserved for.
An SS3 parameter line requests type 1 with an env-var of 1xx. Verified
against the pinned v3.30.22.1 source: parm_timevary is seeded with the base
parameter (SS_timevaryparm.tpl:50), type 1 multiplies it by
mfexp(beta * env) (:206-211) where type 2 adds (:215-220), the result is
Svy_log_q (SS_expval.tpl:408), and the 1xx decoding for Q_parm is
SS_readcontrol_330.tpl:3328. The deviation stays outside the multiply,
matching SS3's order (env at :200-245, devs at :252-330).
Where it applies, and where it does not. SS3's type 1 multiplies a parameter on whatever scale SS3 stores it on, so the restriction follows that scale:
SS_biofxn.tpl:1074, :265-275), where type 1 is parm * exp(beta * x) --
exactly what Rceattle's "log" link already computes. An SS3 NatM or
growth line carrying env-var 1xx bridges with "log". Applied to a
natural-scale parameter this form would raise it to a power, and on M -- where
log M is always negative, since M < 1 -- that inverts the sign of the
covariate effect.SR_LN(R0), and
SS_timevaryparm.tpl:53 seeds parm_timevary from SRparm), so this form is
the right one there. No accumulator consumes it yet, and the refusal says so
rather than offering "log", which is SS3's type 2.Svy_log_q only when the survey error type is lognormal or t; under a
natural-scale family (MVN, MVNORM, Normal, TruncatedNormal) it reads
the same slot arithmetically (SS_expval.tpl:413-419), where type 1 is again
"log". Rceattle holds q on the log scale whatever the index family, so the
model is still well defined there and those families warn rather than
refuse -- but it is the wrong bridge.log q,
so a formula with no intercept (~ 0 + x) or a fixed one (est_phase = 0)
makes map_linkage_adjuster() mask index_log_q, freezing the value being
multiplied -- and at Catchability_init = 1 that value is exactly 0, where
beta has an identically zero gradient and the fit converges on whatever
beta started at. Refused. A free base is left alone: where it starts is not
knowable at build time, because fit_mod(inits = ) and an intercept init
both override Catchability_init, so the fitted value is what
exponential_q_near_one reads instead.q_linkage_offset, which sits inside the multiply, so an
ar1(1 | Year) q linkage would have its deviations scaled by
exp(beta * x) while their density still scored them at a constant sigma.
Refused on both the same spec and the same fleet. An environmental effect
plus q deviations is still expressible with Time_varying_q, whose
index_q_dev the template keeps outside the multiply, as SS3 does its dev
blocks.Worth knowing even on q, and it is SS3's property rather than ours: because
beta multiplies a log, the effect scales with how far log q sits from
zero, and below q = 1 its sign inverts. Both are pinned by tests so nobody
reads them as a broken linkage.
Report beta with its base, never alone. To first order
log q_y = log q * (1 + beta * x_y), so the index informs the product
log q * beta. With an estimated base the fit may cross q = 1 and re-express
the same curve with the opposite sign of beta: simulating from
q = 0.3, beta = +0.4 and refitting returned q = 467.0, beta = -0.0775,
preserving log q * beta to 1.1% (-0.4761 against -0.4816) with the fitted q
series correlating 0.984 with the truth. Those figures are the
simulate-and-refit check in tests/testthat/test-linkage-exponential-link.R at
its own seed on one platform; the test asserts the invariant, not the values,
since an optimizer path need not be identical everywhere. Interpret fit$quantities$index_q; a beta quoted alone is not
comparable between models. A q prior pins the level if beta itself is wanted.
Because that is a silently-wrong-number shape rather than a documentation gap, a
new convergence_diagnostics() record flags it on the fit itself:
exponential_q_near_one (WARN) when a fleet's fitted catchability is within
0.1% of 1, where beta is unidentified. It reads the fitted base rather than the
starting value, because an intercept init, fit_mod(inits = ) and a shared
Catchability_index group each make Catchability_init something other than
where index_log_q began.
The new reported quantity is q_linkage_log_mult -- the linear predictor whose
exp() multiplies log q.
No fit changes for a model without a q linkage: the tensor stays at zero,
exp(0) is exactly 1, and x + 0.0 and x * 1.0 are exact, so the result is
bit-identical. For a model that does carry a log q linkage the sum
reassociates from (log_q + dev) + offset to (log_q + offset) + dev:
algebraically identical, but not bit-identical.
Motivation: the GOA Pacific cod SS3 bridge, whose LLSrv survey catchability
carries this link. Rceattle-models/SS3-bridge/GOA-estimation-parity.md predicted
+9.9645 nats of LLSrv index residual from its absence. Re-run on this version the
bridge closes 11.254 nats with the link (Index data 53.7116 -> 42.4573), all of it
in the Index data row, and no other jnll_comp row moves by more than 1e-6. The
earlier 9.9646 was measured on an ancestor of the initial-level and regime-penalty
commits, so the bridge's own baseline most likely moved rather than this release moving
it; that was not proven, and would need a 5.45.3 A/B.
A sixth initial-age-structure mode, in which the initial fishing mortality is weighted by the fishery's selectivity at age before it accumulates, so the first year decays with \eqn{\sum_{a' < a}(M1_{a'} + F_{init} s_{a'})}. This is Stock Synthesis's InitF convention, and \eqn{F_{init}} is the apical initial F because selectivity is normalized to a maximum of 1.
The existing fished modes are neither of these: 3 charges every age the same
\eqn{F_{init}}, and 4 applies it once rather than accumulating it. Under a
size-selective fishery neither is an equilibrium -- 3 kills barely-selected young
ages at the full initial F, and 4 does not decay the older ages with it at all.
The selectivity is the mean over the species' fishery fleets in the first hindcast
year. That is exact for a single fishery, which is what SS3's per-fleet InitF
reduces to here; Rceattle carries one \eqn{F_{init}} per species where SS3 carries one
per fleet, so a multi-fishery stock gets the mean shape.
Every other mode leaves the weight at 1 and is bit-for-bit unchanged, including
SPRFinit and therefore \eqn{R_{init}} under a stock-recruit curve.
On a species with more than one fishery, Finit is applied at the MEAN selectivity of
those fisheries and so is not the apical initial F of any one of them. data_check()
now warns, naming the species: the initial state sets the SSB scale.
Found while bridging the 2024 AI Pacific cod SS3 assessment; the derivation against
SS3's Early_InitAge is in inst/dev/SESSION_HANDOFF.md.
Four options that let an estimated-growth model reproduce Stock Synthesis 3.30's growth and spawning output exactly. Each defaults to the current behaviour, so existing models and the golden references are unchanged.
build_growth(pop_lengths =): population length bins. The age-length
key, weight-at-length and maturity-at-length are computed on these finer bins
(lower edges, cm) and then summed into the data length bins, as SS3 does with
its population bins. The linear growth ramp below growth_age_L1 starts from
the lowest population edge. Every data-bin edge must be a population edge.
Default: the data bins.build_growth(sd_form = "CV"): growth variability as CVs. The two
endpoints sd_L1 / sd_Linf become coefficients of variation and the SD of
length-at-age is CV x mean length at every age (SS3 CV_Growth_Pattern 0).
"SD" (default) keeps them as SDs in cm (pattern 2).build_growth(plus_group_length =): plus-group mean length. "M1"
(default) is the current M1-weighted mean; "none", "SS3.24" and "decay"
(with plus_group_decay) are SS3's Linf_decay = -998, -999 and a positive
decay rate. Under the three SS3 forms the plus group also grows within the
year, as SS3's does.L50_mat_len / slope_mat_len: maturity-at-length. Two new per-species
control columns (cm, per cm; logistic). When set, spawning output per fish is
maturity x weight integrated over the length distribution at spawning, SS3's
fecundity-at-age, instead of weight-at-age x the age-based maturity sheet.
Needs estimated growth. Round-trips through write_data() / read_data() and
appears in write_template().Spawning biomass, SB0/SBF, dynamic B0 and the SPR reference points now all read
one reported array, spawn_output ([nspp, nages, nyrs], kg per fish).
With all four set, the Aleutian Islands Pacific cod bridge (SS3 3.30.22.1, model
M24_1) reproduces SS3's length-at-age, Jan-1 age-length key and fecundity-at-age to
within Report.sso's printed precision, against gaps of 1.5% and 5-6% before. SSB is
within 1.5%, the rest being selectivity. Term-by-term numbers are in
inst/dev/SESSION_HANDOFF.md.
sd_plus_group: when SS3's Growth_Age_for_L2 is 999, SS3 pins the plus
group to CV_old, which is sd_plus_group = "WHAM" here, not "SS3". The
documentation now says so.
Selectivity = "DoubleNormalSS3" (15): SS3 size pattern 24. The six-
parameter double normal, each parameter on SS3's own scale (peak; logit top
width; log ascending and descending widths; logit initial and final
selectivity), in its own parameter array sel_dn6. An end left at SS3's
-999 is unscaled and its parameter fixed. The curve is not normalized, as in
SS3. All six parameters take selectivity linkages, under the SS3 manual's
names (dn_peak, top_logit, ascend_se, descend_se, start_logit,
end_logit), one name each. A link = "identity" block is
linkage_spec(~ cut(Year, breaks), link = "identity") and adds to the base
parameter, which is SS3's Blk_Fxn = 1 (temp = baseparm + parm,
SS_timevaryparm.tpl); it is not Blk_Fxn = 2, which assigns the block value
directly (temp = parm). Porting a Blk_Fxn = 2 model means entering each
block as its difference from the base, not the block value itself. SS3's annual
devs (dev_link 1) are a log-link (1 | Year) term with integrate = FALSE
and a fixed SD. Time_varying_sel must be "Off" for this form.start_logit at -999 and scales end_logit.Selectivity_index must agree on their ends too. They estimate
one sel_dn6 block, but the flag is read off each fleet's own starting value, so
the group would share a parameter and still be given different curves. Refused,
naming the fleets. Members on another form are not compared: nothing but
DoubleNormalSS3 reads sel_dn6.sel_dn6 is the only array DoubleNormalSS3 reads, and no other form reads
it, so a name used the wrong way round was estimated and changed nothing — a
time-invariant curve and a beta_linkage coefficient at zero gradient, with no
error. peak (which resolves to sel_inf) against dn_peak (sel_dn6) is the
pair to watch, since both the header and parameter_dictionary() call form 15's
P1 "peak". Now refused in both directions. The same gap on the older forms —
slp_asc on a non-parametric fleet, say — is left for its own change, since
those configurations are in released scripts; see
inst/dev/TODO-selectivity.md, Open 3.SS3 scores the catch the stock yielded under its initial fishing mortality in the
year before the hindcast, and creates an InitF parameter only when that
observation exists and is non-zero. Rceattle estimated log_Finit with nothing
to fit it to, so Finit sat on a ridge with init_dev -- 7.5x movement for 0.17
nats on AI Pacific cod. This supplies the missing half.
The observation is a catch_data row at styr - 1. No new columns:
Fleet_code, Year, Catch and Log_sd already say everything, and
write_data() / read_data() round-trip it unchanged.
clean_data() moves the row out of catch_data into a new data_list element,
equil_catch_data, so catch_data keeps one row per fitted catch. Every
predicted-catch quantity is indexed by that frame -- catch_hat, catch_sd,
plot_catch(), residuals(source = "catch"), sim_mod() and run_mse()'s
projection fill all pair with it row for row -- so a catch_data carrying an
extra row puts the wrong year's catch in each of them. The element is read back
by clean_data() itself, so cleaning twice is a no-op and a refit through
retrospective(), jitter(), self_test() or run_mse() keeps the
observation; it is combined by combine_data() and written back into the catch
sheet by write_data().
That year is shared with ordinary catch history, so the rule is gated rather
than marked. GOA2018SS carries 23 catch rows before styr, two of them on
1976, and a model has no way to say which a row is. A styr - 1 row is read as
an equilibrium catch only under initMode = "FishedNonEquilibriumSelected";
under any other mode it is history and is dropped exactly as before.
The restriction is the prediction's, not a policy: the equilibrium catch is
Baranov at Finit * s_a, and mode 6 is the only mode that builds the initial age
structure with that same mortality. Mode 3 charges every age a flat Finit and
mode 4 applies it once, so scoring this observation under either would fit
Finit to a catch the population was never subject to. On a stock with more than
one fishery there is a smaller version of the same gap, since Rceattle carries
one Finit per species: N_eq decays at the mean fishery selectivity while the
prediction uses the row's own fleet.
data_check() names the fleets whose rows it read whenever it reads any, so
nothing is reinterpreted silently. It refuses two rows for one fleet, a
non-positive catch, a non-fishery fleet, a Fleet_code that is not in
fleet_control, a Species that does not match that fleet, and a non-positive
Log_sd. A negative sentinel was considered and rejected: run_mse() reserves
negative Year for rows it splices back in as the next assessment's data, and
its window filters are on abs(Year).
rearrange_data() turns the rows it reads into equil_catch_ctl /
equil_catch_obs, and drops any styr - 1 row still sitting in catch_data
for a list that reached it uncleaned: such a row in the catch equation would
index F_flt_age(..., -1), an out-of-bounds read rather than an error.
data_check() reads the rows from either place, so it names the fleets whether
or not the list has been cleaned, and gives them the checks a catch row gets --
a Fleet_code that is in fleet_control, a Species that matches it, and a
positive Log_sd.
The prediction is Baranov on the deviation-free equilibrium age structure, using the fleet's own selectivity and weight in the first hindcast year:
C_eq = sum_a Finit * s_a * w_a * N_eq_a * (1 - exp(-Z_a)) / Z_a
Z_a = M1_a + Finit * s_a
jnll_comp gains a row, Initial equilibrium catch, and equil_catch_hat
joins the reported quantities; both are registered in the row-axis registry and
the quantity dictionary. Every bundled model and all four golden references are
unchanged.
If you read a jnll_comp row by position, read it by name instead. The new
row is appended, so the last row is no longer the linkage random effects; the
tests that did this now look the row up through .JNLL_ROW_AXIS.
age_error held one matrix per species and every read was age_error(sp, ...).
SS3 selects one per observation through its ageerr column, and a stock whose
ageing protocol changed part way through a series needs more than one: GOA
Pacific cod reads its pre-2007 survey otoliths about a year older than its
post-2007 ones, a bias of roughly two standard deviations at every age, and
Rceattle applied one of the two to the whole series.
The selection is per fleet, mirroring Age_transition_index, which already picks
among several age_trans_matrix matrices the same way. Two eras of one survey are
then two fleets sharing a Selectivity_index and a Catchability_index, so they
estimate one selectivity and one catchability between them.
Additive, and silent on anything that does not use it:
age_error gains an optional Ageing_error_index column. Absent, the index IS
the species, which is what the array held before; its first dimension is now the
matrix rather than the species.fleet_control gains an optional Ageing_error_index. Absent or NA on a row,
that row reads its own species' matrix.data_check() keys its age_error checks on Ageing_error_index, since that
is what the array is filled by. It refuses an index that names no matrix
(listing what was asked for and what exists), an index whose rows give more
than one Species — a matrix's rows are sized and offset by one species' age
range — and a fleet pointed at another species' matrix, which the species
default invites: with no index column on age_error, 2 means "species 2's
matrix", not "the second matrix". Coverage is reported per matrix, so a matrix
that stops short of the oldest true ages is named even where the species' other
matrices are complete; keyed on the species it was invisible, and the missing
rows stay 0 and are renormalized away.Ageing_error_index no longer inflates it.selectivity.hpp evaluated a length-based curve at the data bins while
rearrange_data() aggregated the age-length key to the same bins, so a model
whose composition bins are coarser than its population bins applied one
bin-average selectivity across a whole data bin. SS3 forms sel(L) * P(L | age)
at population resolution and bins the result.
Identical whenever the two grids coincide, which they do for any model supplying
no pop_lengths -- every bundled and golden model. growth_matrix and
sel_at_length move to the population grid, and the four places that combine them
follow: the selectivity-at-age conversion and the selectivity-weighted
weight-at-age sum every population bin, while pred_CAAL, which the length
compositions are built from, sums each data bin's own run of population bins.
pop_to_data_bin is monotone, so those runs are contiguous and are precomputed
once; with a single grid every run is one bin and both the work and the arithmetic
are what they were.
Everything that walks the curve walks the population grid with it: the
normalization reference and the division, the zeroing below Bin_first_selected,
and the projection copy that carries the terminal hindcast year forward. Bounded
by the data-bin count instead, a finer grid kept only the lowest lengths, where
an ascending curve is close to zero, so projected selectivity-at-age came back
identically zero -- and with it projected F, catch and the reference points.
The columns that name a bin stay data-bin ordinals, which is what the user
has and what data_check() bounds. They are not translated onto the population
grid: a data ordinal cannot address a finer grid without picking a reading, and
the readings differ. So on a species whose population grid is finer than its data
grid, data_check() refuses a length-based fleet that is indexed by bin --
a non-parametric or AR1 form, a time-varying deviation penalty, or any of
Bin_first_selected (above 1), N_sel_bins, Sel_norm_bin,
Sel_norm_bin_upper, Sel_pen_first_bin, Sel_pen_last_bin or Sel_cap_bin.
A parametric form that names no bin is a function of length and is unaffected,
which is the case the population grid exists to serve. A species with no separate
grid is unchanged.
DoubleNormalSS3 also gains the two things SS3 does that the first port did not:
the ascending limb is anchored at startbin, the first population bin reaching
the first composition data bin, and the bins below it take SS3's quadratic ramp.
Two things the refusal depended on, both fixed here:
Bin_first_selected first, and that column carries no schema default, so a
workbook without it gave a zero-length flag that excused every fleet —
including the non-parametric forms and the time-varying penalties, which the
column has nothing to do with. Time_varying_sel is read with [[ in the same
block, since $ returns Time_varying_sel_sd by partial match.max_bin, which dimensions non_par_sel, log_non_par_sel and the
selectivity-linkage offset tensors, is sized on the population bins. The
non-parametric forms build their curve over nlengths_pop, so on a finer grid
they wrote past the end of those arrays. With one grid nlengths_pop equals
nlengths, so no existing model changes.A fleet with length-based selectivity and estimated growth now weighs its catch and survey biomass by the mean weight of the fish it selects, sum_l P(l | a) s(l) w(l) / sum_l P(l | a) s(l), not by the age class's mean weight at the fleet's month. A length-selective fleet takes the larger fish of a young age class, so the age-class mean understated its catch weight; this is also Stock Synthesis's catch weight. Age-selective fleets and empirical-weight models are unchanged, including all four golden references.
What it moves, measured. The 2024 GOA Pacific cod approximation
(Rceattle-models/GOA cod/2024_pcod.R, 224 parameters, length-based selectivity
and estimated von Bertalanffy growth) is the one bundled-or-sibling model the
change reaches. Refitting it on 5.45.3 and on this version:
| | 5.45.3 | 5.46.0 | |---|---|---| | Objective | 6415.8172 | 6416.2875 (+0.470 nats) | | Terminal SSB (mt) | 1,004,925 | 935,106 (-6.9%) | | SSB series | | mean -7.1%, largest -12.9% | | Convergence | OK | WARN, max gradient 0.0013 |
SSB sets B40% and depletion, so read that -7% before refitting an assessment on this version. The WARN is a maximum absolute gradient of 0.0013 on R0, worth at most 0.00085 standard errors, not a failure to converge.
This is not evidence the change is an improvement, and the comparison that would
look like evidence is not one: that script is an approximation, sitting 63%
below the accepted 2024 SAFE spawning biomass on 5.45.3 and 66% below it here,
with SSB rising across a series over which SAFE declines. A 3-point move against
a 63% gap says nothing. The case for the change rests on Stock Synthesis parity
in GOA cod/Bridging/ and AI cod - Dev/Bridging/, where the bridges match SS3
to additive constants and SS3 weighs catch this way.
A shared catchability linkage row is now checked against every fleet.
NA is the shared-stratum sentinel and the template expands it to all fleets,
but .check_q_linkage_support() only expanded it when every row was NA, so
a table mixing a shared q spec with a per-fleet one was checked against the
named fleet alone. The shared row then reached fleets whose Catchability is
NA, Fixed, Analytical/AnalyticalArith or Environmental -- the forms
that check exists to refuse -- turning a fixed q time-varying, or leaving a
coefficient free with no gradient. Such a configuration now errors with
"catchability linkage on fleet(s) ... does not estimate q"; restrict the spec
with linkage_spec(fleet = ).
An aggregated composition figure no longer pools two observation
structures into one normalization. A fleet can carry more than one --
sexes combined in the early years of a series and disaggregated later -- and
the likelihood scores one density per structure. The aggregate pooled every
row for a fleet and comp type into one panel and renormalized across the
whole of it, so neither the combined series nor the sex-specific ones summed
to 1 and the two were not on comparable axes. plot_comp() now keys the
panel on the structure as well, but only for a fleet that mixes them: a
fleet with one structure, which is every bundled data set, keeps its previous
panel and title exactly. The effective-sample-size annotation on the same
figure already grouped on the key the likelihood scores a density over
(fleet, species, sex, year); the aggregate now follows it.
Reported by an external user running a two-sex model, who was otherwise going to add a mirrored fleet to work around it -- changing the model to fix a figure.
A joint-sex OSA series says it is half of one unit. A Sex = 3
composition is appended as a single decomposition unit over the stacked
female-then-male vector, with one cell dropped for the sum-to-one constraint
rather than one per sex, which the OSA figure then re-bases onto a common bin
axis so both halves can be read. Labelled - male / - female, that reads
as two separate distributions: the same external user asked why the residual
structure did not match the likelihood structure. It does -- the residuals
were never computed per sex -- so the labels are now - joint, male /
- joint, female, and a single-sex row, which genuinely is its own density,
stays unlabelled. No residual changes.
CONTRIBUTING.md now states a code style. R follows the tidyverse style
guide and C++ the Google C++ style guide, as tie-breakers where the
surrounding file does not settle the question; TMB idiom outranks both, and
neither is a reason to reformat untouched code. Three package-specific habits
go with them: read a list or fleet_control column with [[ ]] rather than
$, which partial-matches silently; write in the idiom of the file you are in
rather than converting base R and dplyr into each other inside the fitting
pipeline; and keep reformatting out of a change that can move a number.apical linkage adds to,
not just what it does. Both described the multiplier correctly and both
likened it to Stock Synthesis's male-offset option, without saying that the
form's own parameters are estimated separately for each sex and stay that way.
A two-sex Logistic fleet with an apical linkage estimates five
selectivity parameters -- a slope and an inflection per sex, plus one
multiplier on the sex named -- so the named sex's curve has three parameters
to the reference sex's two. An SS male offset is usually an offset from the
female shape, so there the shape is shared and one parameter carries the sex
difference; here both shapes stay free and the offset carries only the height.
That distinction decides what the parameterization is worth when the sexes'
compositions are thin, since each shape is informed by every row for that sex
while the height is informed only by the joint-sex rows (Sex = 3). Sharing a
shape across the sexes is not a switch: it needs a hand-edited map, which
the vignette now says rather than leaving the reader to infer it from the SS
comparison.test-golden-regression.R re-fit its four reference models and compared the
objective. That answered two questions at once: whether the likelihood had
changed, which is what a golden check is for, and whether a cold phased
optimization reaches the lower of two minima on this machine, which has a
platform-dependent answer. goa_ss has a second local minimum 52.9 units up, a
one-ULP change in a single log_F gradient element is enough for nlminb to
reach it, and goa_ms inherits the basin through its warm start -- so the two
GOA references moved together on the CI runners while reproducing exactly on
local macOS. deep-checks, the only job that runs this file, was red on main
for weeks over a summation order, which is the gate a release leans on.
The likelihood is now evaluated at the reference parameters, which removes
the optimizer from the gate: the parameters are committed as
tests/testthat/fixtures/golden-reference.rds (28 KB) and
fit_mod(estimateMode = 3) builds the object and evaluates without optimizing.
A change to the model moves the objective; a change to the optimizer's path
cannot. The check still asserts the reference is a stationary point, so a pin
cannot drift to a number that merely reproduces somewhere flat.
The fixture pins the parameters and the objective at the SAME point, which is
not free: newtonsteps moves opt$par and opt$objective, while
estimated_params and quantities stay at TMB's last.par.best. On goa_ss
those are 2.3e-08 apart in parameter space -- 9.1e-12 in objective, and a
gradient of 3.2e-06 rather than 1.3e-11. Pinning from one point (the polished
one) makes the reproduction exact and leaves the gradient gate its margin.
Measured at 5.45.1: all four reproduce their pinned objective to 7e-12 absolute,
which is the rounding of the 15-digit literal itself and not a model difference,
at gradients between 1.3e-11 and 5.0e-11 against a 1e-8 threshold. The
multispecies pair was the open question, since predation iterates inside
fit_mod(), and the iteration reaches the same state from the reference
parameters. The objective gate is 1e-10 relative, about 1e-06 nats. That is not
a hundredfold gain in sensitivity over the 1e-6 it replaces -- the old gate's
real discrimination floor on the GOA pair was the 52.9-nat basin gap, 4e-03
relative. What this buys is determinism. A 0.01 shift in the objective fails it,
verified by mutating jnll in the template and reverting.
The cold-start fits move to tools/verify/verify-golden-cold-start.R. They are
a harness rather than a test because their failure mode is the platform-dependent
one this change exists to remove: whether the higher minimum is as well polished
as the lower one is unmeasured, so a gradient assertion there could fail for
a reason that is not a regression. The harness asserts the direction that cannot
flake -- a cold fit must not land BELOW the reference, which is a likelihood
change and not a basin lottery -- and reports the rest.
What this gives up: an edit confined to the optimization path no longer fails the gate. A phase order, a starting value or a bound that is inactive at the reference can now move a cold fit into the other basin with the gate green. Run the cold-start harness when touching those.
Regenerate the fixture with tools/verify/regenerate-golden-reference.R, and
only when a model change is intended. The script refuses to write a reference
whose gradient is above 1e-4.
/golden-check and the deep-checks failure message describe the new meaning.
No package code changed: this is the test harness, a fixture and documentation.
README.md's pinning example names a version, and it went stale four times in
the 5.34.0-5.45.0 release -- at 5.41.0, 5.42.1, 5.43.0 and 5.45.0 -- each time
because a version landed on dev after the release paperwork was written. Three
were caught in review and one shipped to main, where it told an assessor to
install a tag that does not exist. test-docs-anchors.R now fails when the pin
and DESCRIPTION disagree.
fit_mod() passes build_bounds()'s range to nlminb, which respects it. The
Newton refinement that fit_control(newtonsteps) asks for afterwards does not:
in both .fit_tmb() paths it is a plain unconstrained step, so a parameter
nlminb parked on a bound can be pushed straight through it, and the value that
comes back is the one saved in the fit.
convergence_diagnostics() reported that as parameters_on_bounds at WARN,
whose message read "at a configured bound" -- because the test par <= lo + tol
is also true below lo. A value outside the range the model declared plausible
now gets its own record, parameters_outside_bounds, at FAIL, naming the
parameter and pointing at newtonsteps. A parameter genuinely sitting on a bound
still reports parameters_on_bounds at WARN and is no longer double-counted.
A tolerance band remains, scaled to each parameter's declared range, so a value
just inside lo - tol still reads as "at" the bound.
The par > -900 filter is gone, which is the substantive part: it hid any
parameter that had diverged DOWNWARDS -- a value at -1e6 against a lower bound
of -10 drew no record at all -- and that is the direction an unconstrained
Newton step most plausibly takes. The -999 sentinel it was there for (an
unfished fleet-year in log_F, an init_dev above the plus group) is now matched
by NAME and value instead. That skip is defensive rather than load-bearing: every
-999 slot is mapped out before the bounds vector is built, so none reaches this
check on a fit built from scratch, but a warm start can carry one into a slot
whose catch is now non-zero. An unnamed parameter vector keeps every element,
rather than emptying both records.
No fit changes. The Newton steps themselves are left alone in both paths --
clamping the one in .fit_tmb()'s fallback was considered and rejected, because
that path runs only when TMBhelper is absent, so neither the test suite nor the
golden references reach it, and an unverifiable change to how a fit is computed is
worse than a reported one. The diagnostic covers both paths.
newtonsteps defaults to 0, so no default fit is affected. Measured on a GOA
Pacific cod bridge at newtonsteps = 3: a growth parameter returned at 5.08e-05
against a lower bound of 1e-3, with a maximum gradient of 0.263 and a Hessian that
would not invert. At newtonsteps = 0, the same code and the same data, that
parameter sat at 1.00e-03, on the bound, with a maximum gradient of 0.00246 and a
clean sdreport -- so the three steps bought 0.0017 nats of objective and cost the
feasible region and the standard errors. The stale comment in R/0-convergence.R claiming
optimization is unbounded in fit_mod() is corrected.
Fleets sharing a Catchability_index share ONE index_log_q, and TMB starts a
shared parameter at the mean of its members' starting values. index_log_q is
log(Catchability_init), so the group starts at the geometric mean of the
members' inits and no fleet keeps the value in its own row. That has always been
the behaviour; vignette("model-options-and-functionality") has described it
correctly since 5.9.0, which corrected docs that said otherwise. Nothing said so
at run time, though -- unlike the deviation standard deviations, which
build_map() has warned about for some time.
build_map() now warns for the catchability itself, naming the group, the
differing inits and the geometric mean it will start from. Two things the
warning is careful about:
Catchability_init is blank, zero or negative seeds the whole
group at NA or -Inf rather than at any mean, and the group cannot fit.
data_check() requires the column positive only on fleets that carry index
rows, and exempts Analytical / AnalyticalArith because they solve q from
the data -- yet a fleet in either category still joins an estimated block. The
warning reports the non-finite start instead of naming a geometric mean the fit
will never reach.Catchability_init is read twice. Under Estimated-with-prior the lognormal q
prior stays centred on the lead fleet's value, so differing inits there move
the objective and not merely the starting point. The warning says so, and only
where a prior is actually scored. Relatedly, a non-lead fleet's
Catchability_prior_sd is never read either, which the vignette now states.Catchability_init's schema entry now says it must be positive on a fleet sharing
an estimated Catchability_index group too, not only on a fleet carrying index
rows, and Catchability_prior_sd's says only the lead's is read.
inst/extdata/meta_data_names.xlsx is regenerated from the schema accordingly.
data_check()'s existing reports are unchanged: the analytical case, a Fixed
lead leaving fleets on different inits, and a Catchability or Time_varying_q
differing within a group. The gap was the ordinary case -- forms agreeing, inits
not, a q estimated for the group.
No fit changes. This is a warning and a documentation fix.
Why it is worth a warning rather than a note in a vignette: a shared q at the
mean scales a survey's whole predicted index by a constant factor, and no
residual pattern distinguishes that from a real change in abundance. It was found
on an SS3 bridge for GOA Pacific cod, where a fleet created by a converter kept a
default init and dragged the survey's q to sqrt(1.4964) = 1.2233. Measured on a
forward pass -- estimateMode = 3, so the objective is evaluated AT the starting
values and the start IS the fitted value -- that was 18% low across all 16 index
observations and 6.23 nats, with the standard deviations right to 5e-07 and both
composition components agreeing to under 0.001. In a fit that estimates q the
start matters less, though it still chooses which optimum is found.
The internal helper .warn_shared_dev_sd() is now .warn_shared_block_start(),
since it no longer reports only standard deviations.
The package moved to afsc-assessments/Rceattle, and every link follows
it. DESCRIPTION's URL and BugReports, _pkgdown.yml's url and
navbar, the README's install commands and badges, ?print.Rceattle's install
hint, CONTRIBUTING.md, examples/Install_Rceattle.R and two vignettes all
named grantdadams. GitHub redirects the repository path, so the install
commands kept working -- but the documented website did not:
https://grantdadams.github.io/Rceattle/ returns 404 and
https://afsc-assessments.github.io/Rceattle/ returns 200, so URL,
_pkgdown.yml and the README all pointed users at a dead page. Pin
afsc-assessments/[email protected]: a redirect is not a permanent address, and
it breaks the moment anything is created at the old path.
An integer-coded Fleet_type no longer reads as estimated in
data_check(). data_check() is callable on a list straight from
read_data(), where the switch columns are still the integer codes the
workbook stores -- and every bundled data set that carries a fleet_control
stores them that way (GOA2018SS is 2, 2, 2, 2, 2, 2, 0, 1, ...), so the
integer is the shipped representation. 0 != "Off" coerces to
"0" != "Off", which is TRUE, so an Off fleet read as estimated: on
GOA2018SS with fleet 7 given its own Selectivity_index, it was named in
the "estimated Selectivity but no comp_data" error, and is not now. The
subset reads the column through .canon_switch(), as the same file already
did eleven lines further down, rather than through a fourth spelling of the
test -- .canon_switch() also resolves " 0 ", "0.0" and "00", which a
%in% list does not.
No fitted number moves. fit_mod(), build_map() and build_params()
each call switch_check() first, and on the canonical strings the old and
new readings are identical() -- asserted for five bundled data sets. Only
the un-canonicalized path changes.
The rest of the class is open, in inst/dev/CLEANUP_BACKLOG.md: the schema
types thirteen columns as switch with an allowed map, but nothing
enforces that at the boundary, so any comparison written against the
canonical spelling is wrong for a fleet_control that has not been through
switch_check(). Those sites are unreachable today because their callers
canonicalize first; the fix is to enforce the schema once on entry rather
than to convert them one at a time.
The distance-to-optimum report is withheld when the covariance cannot
carry it. The Newton step measures the distance to a minimum only if
cov.fixed inverts a positive-definite Hessian. On a saddle the step points
away from one, so "reaching the optimum would move the estimates by at most
0.0009 standard errors" read as reassurance printed directly beside a failing
pdHess. It is reported only when pdHess is TRUE. Severity is still read
on the gradient, so no fit changes status -- now asserted, where neither that
claim nor the laziness of the coordinate lookup had a test.
A negative variance no longer escapes as NaNs produced. Three sites took
sqrt(diag(cov)) unguarded, in a battery documented never to raise; they now
share .conv_se_from_cov(). The callers already dropped non-finite entries --
it was the sqrt() itself that warned.
A negative variance reports no standard error, rather than
sqrt(|variance|). summary() and report_tables()'s parameter table both
took sqrt(abs(diag(vcov(fit)))). vcov() is sdreport()'s fixed-effect
covariance with no pdHess gate, and an indefinite Hessian inverts without
being positive definite, so a variance can come back below zero (demonstrated on
an indefinite 2x2 Hessian; the defect is reachable only where pdHess is
FALSE): abs() then
printed a plausible standard error, with no warning, in the table a SAFE
chapter's executive summary is built from. Both now return NA for that
parameter and the real standard error for every other; the pdHess check in
convergence_diagnostics() says why it is missing, and print(summary()) says
which of the two reasons applies. A variance within rounding of zero
(-1e-18 on a flat ridge) reports 0, not NA: that is a direction the data
barely inform, not an indefinite Hessian.
hessian_conditioning's coordinates line up again. The column width was
computed inside .rce_par_summary(), which .check_hessian_eigen() calls
once per block, so each line sized to its own block name and the aligned
output vignette("model-diagnostics") illustrates was unreachable. The caller
now passes one width, and the vignette's example is regenerated from the
format print() emits rather than hand-written.
A blank Fleet_type is refused. The column has no schema default, so
nothing filled it, and a blank one is not "unset, take the default" -- it is
a fleet whose role in the likelihood nobody stated. switch_check() now
stops, naming the fleet and the values to choose from, and
rearrange_data() does the same: it is exported and reachable without
switch_check(), and a blank reaching it was handed to the template as
flt_type = NA.
rearrange_data() checks it before convert_switches(), which coerces the
column to integers: a blank and a value it cannot map both become NA there,
so a mistyped Fleet_type was reported as a blank cell. It now refuses, by
fleet and by value, any type outside the set validate_switches() allows --
Fleet_type could not be read for fleet(s) Bottom_trawl ('Fisherie'). That
closes three codes the old check handed to the template: 3 (the index takes
the survey branch and the catch is dropped, and because the predicted
composition is only assigned for types 1 and 2 while the composition
likelihood is gated on type > 0, the comps are fit against an all-zero
prediction and accrue a posfun penalty), -1 (contributes nothing, and
is eligible to lead its Selectivity_index group, whose penalty is gated on
flt_type > 0, so the group's shape and curvature penalty goes uncharged) and
2.7 (truncated to Survey). data_check() refuses all three, so no model
that fits through fit_mod() today is newly stopped. It is stricter than the
converter in one respect, deliberately: a padded or non-canonical spelling
(" 0 ", "00", "2.0") is refused here although convert_switches() could
resolve it, because the allowed set is the documented one. Write the canonical
name or the bare code.
NA was the damaging case, and the two halves of the package read it
differently. Fleet_type != "Off" is NA rather than FALSE, so
data_check()'s estimated-selectivity subset kept an all-NA row and died
inside its vapply on missing value where TRUE/FALSE needed, naming no
fleet and no column -- measured on Atka2022 with Fleet_type[1] <- NA --
while build_map_selectivity() treated NA as estimated
(.on[is.na(.on)] <- TRUE). An empty string or whitespace already produced
a correctly named error from validate_switches(); those are refused here
too, earlier and from one rule, rather than because they were broken in the
same way.
A factor-valued fleet_control column is read by its labels, not its level
indices. read.csv(stringsAsFactors = TRUE) factors every column, and
convert_switches() passed a factor straight to as.integer(), which returns
the level index rather than the code the label names. switch_check() ->
revert_switches() already resolved eight of the nine switch columns, so for
those the exposure is the exported rearrange_data() path, where
Fleet_type = factor(c("Off", "Survey")) reached the template as 1, 2 -- the
Off fleet fitted as a fishery, its catch entering the likelihood while
build_map(), which reads labels, pinned its parameters.
Time_varying_q is the ninth, and it is not on that list, so a factor there
reached the template by level index even through fit_mod(): a column of
factor("Off") became IID (1), estimating time-varying catchability
deviations nobody asked for. That is the one configuration in this release
whose fit moves; refit any model whose fleet_control was built through
read.csv(stringsAsFactors = TRUE). Neither switch_check() nor
validate_switches() refused it -- both compare labels, so it looked valid all
the way down.
Factor columns are now resolved once, in switch_check() and again in
convert_switches() for the rearrange_data() path. No bundled data set and
no workbook or script in the four consumer repositories supplies a factor
(checked: 375 workbooks, 15 bundled .rda, and every data.frame() /
read.csv() that builds a fleet_control), and the package requires R >= 4.1,
where the default is already FALSE.
Every spelling of a stated type -- canonical name, integer code, character
code, and Off -- is accepted exactly as before. No bundled data set and
none of the 183 consumer-repository workbooks carrying a Fleet_type column
has a blank one, so nothing that fits today stops fitting.
Still open, and tracked in inst/dev/CLEANUP_BACKLOG.md: an
integer-coded Fleet_type read before switch_check() canonicalizes it
reads as live at every bare != "Off" comparison, because 0 != "Off" is
TRUE. validate_switches() documents the trap and canonicalizes first;
fourteen other raw comparisons in R/ do not.
fit$convergence names what a flagged parameter estimates. The
max_gradient message, the hessian_conditioning loadings, and the
per-coordinate lines under hessian_conditioning, parameters_on_bounds and
estimability now follow each block name with the natural-scale quantity
from parameter_dictionary(): log_M1 (M1), rec_dev (recruitment deviations), log_F (F). The block name stays first, since it is what
map and the parameter list are keyed on.max_gradient says where the largest gradient sits. It gave the block
alone ('log_M1'); it now gives the species, fleet, sex, age or year from
parameter_index(), the way estimability, parameters_on_bounds and
hessian_conditioning already did. The coordinate is resolved only when the
check is not OK, so a clean fit still skips building the index.max_gradient reports the distance to the optimum in standard errors,
as $data$newton_step_se (the largest absolute value), $data$step_se (the
signed vector) and in the message. A quadratic approximation puts
the optimum a Newton step -cov.fixed %*% gradient away; dividing each
element by its standard error makes the size comparable across log, logit
and natural-scale parameters. It is reported only when the sdreport
describes the hindcast parameters (not under an estimating HCR), and the
severity is still read on the gradient, so no fit changes status.hessian_conditioning
count reads "(38 of 44 parameters; 67% of the direction)".Found reviewing the release PR that carries 5.34.0 through 5.42.0 (#158), over several adversarial passes. Corrections to 5.42.0's own guard, to the tests meant to hold it, and to documentation it left behind.
The negative-Sel_curve_pen refusal no longer skips a fleet whose penalty
the template does charge. The check skipped any fleet that followed another
fleet's Selectivity_index, on the grounds that a shared block is penalized
once, on its lead. But the template's lead (flt_sel_lead, built in
rearrange_data()) groups fleets by Selectivity_index and selectivity
form, while the rule the check borrowed -- the one the parameter map shares
on -- groups by the index alone. Two live fleets sharing an index with
different forms are therefore two groups to the template, each charged, and
the check called the second one a follower and passed over it: a negative
weight there reached ceattle.cpp and rewarded the deviation it names,
without bound, with data_check() reporting clean. The check now reads the
lead the same way the template does. No bundled data set and no workbook in
the consumer repositories has a group of mixed form (every bundled
fleet_control, including the three carried inside fitted example objects,
and 183 workbooks with a fleet_control sheet; the answer is the same read
raw or canonicalized through sel_map), so no existing model changes;
this closes the guard rather than moving a number.
The lead it builds also fails closed on a Selectivity the template cannot
read. .canon_switch() trims and rearrange_data()'s .pull_int() does
not, so " NonParametric" resolves here and reaches the template as NA --
a group of its own there, with the fleet leading and its weight charged.
Such a value now gets a key of its own here too, so the fleet leads and is
checked. switch_check() normalizes the spelling before either is reached,
so this is a guard against a hand-built fleet_control, not a path a
workbook takes.
test-docs-anchors.R now checks the schema column that shipped the stale
code. The guard added at 5.42.0, so that no schema description names a
selectivity code sel_map does not accept, matched only a slash-separated
run (2/9/13). Of the eight columns it names, four matched nothing at all --
including Selectivity itself, which enumerates every code one per line and
is the column form 14 was advertised in. It now also reads a comma-or-or
list, a per-line 13 = ... enumeration, and a code parenthesized after a
form name, quoted or not -- Sel_curve_pen1 writes "type 2/9/13" and
LogisticPM (11) in one sentence, so requiring the quotes missed 11 and
would have missed a stale code in that position. All eight columns now yield
codes. It is a shape-matcher, not a parser: a code written in a shape none of
the four patterns covers (14 - ..., forms 2 and 14) still slips, so it
narrows the gap rather than closing it.
A refusal test that passed for the wrong reason. The check that an
apical linkage is refused on an AR1 selectivity form matched the string
"AR1", and GOAatf ships Sel_curve_pen2 = 200 on the fleet the fixture
switches: on the AR1 forms that slot is a correlation, so data_check()
stopped first with its own out-of-range message, which also contains "AR1".
Deleting the refusal left the suite green. The fixture now zeroes that column
and the assertion matches the apical clause, so removing the refusal fails the
test.
.rce_sel_pen_lead()'s Off handling was untested. Replacing its off
argument with rep(FALSE, n) left test-selectivity-penalty-sd.R green,
because the sign check has its own independent Off skip. The file now pins
that an Off fleet never leads its group, that the next fleet leads instead,
and that the result is identical() to the flt_sel_lead `rearrange_data()
hands the template for the same table.
README.md's operational pinning example named 5.41.0, a version this line
never releases, so an assessor pinning a version for management advice was
sent to a reference install_github() can never resolve. It now names the
version this release tags. Like every version named there it resolves only
once that tag is pushed -- checklist section 3, and TODO-pre-transfer.md
B5 says to re-check the line whenever the version moves.
vignette("model-parameterizations") listed three cases the negative-weight
refusal does not catch; there are four. The fleet following another's
Selectivity_index was added to the check and to NEWS.md at 5.42.0 but not
to the reference table.
vignette("adding-a-selectivity-form") said code 14 was free and the next
form should take 15, in a paragraph whose own rule is that a retired code is
not free. 14 was advertised as a second integrable form in
meta_data_names.xlsx -- the template every workbook is built from -- before
being collapsed into 13, so a Selectivity column written in that window
holds it. It is now listed with 10 and 12 as taken. The same article named
three R sites that key on the selectivity form when there are at least eight,
attributed two of them to the wrong check, and described
.RCE_SEL_PEN_POSITIVE as driving the penalty-SD conversion; that conversion
is a separate registry, .sd_specs, which fails closed where
.RCE_SEL_PEN_POSITIVE fails open.
vignette("developer-guide") gave three switch-code facts the code does not
support: the selectivity.hpp dispatch list omitted the Ianelli
non-parametric form (2), which every bundled reference model uses, and
descending logistic (4); sel_map skips 10, 12 and 14, not 10 alone; and the
Sel_curve_pen1 / Sel_curve_pen2 slot map omitted form 13.
The 5.39.0 entry now says the stock-recruit bound is unconditional. Predation
is why it was added, and it sits under a predation heading, but
build_bounds() applies +/-30 to every model.
Found reviewing the 5.34.0-5.41.0 release (# Rceattle 5.41.0 and the versions
below it). The negative-weight defect is much older than that review: the
one-sided shape penalty has had no sign branch since it landed in December 2024,
and 4.10.0 added a second way in through Sel_shape_dir. 5.40.0 only extended it
to the integrable forms.
A negative Sel_curve_pen1/2/3 is refused wherever the template reads
it as a weight. A penalty slot multiplies a squared deviation, so a negative
weight rewards the deviation it names instead of penalizing it. Use a positive
weight, or the matching Sel_shape_sd / Sel_curvature_sd / Sel_devmag_sd
column, which is a standard deviation and cannot go negative. The one
exemption is "NonParametricPM" (9) under Sel_shape_mode = "Directional",
the only branch in the template that reads a sign -- there it switches the
penalty from the decreasing side to the increasing one rather than negating
it. That form's "Smooth" mode applies the weight two-sided and is refused
like the rest. "2DAR1" (6) and "3DAR1" (7) reuse these columns as AR1
correlations and are untouched.
Slot 3 is checked only on "NonParametricPM" (9) and "LogisticPM" (11):
"NonParametric" (2) and "NonParametricIntegrable" (13) also estimate
selectivity deviates, but score them with a Gaussian density on
Time_varying_sel_sd, so neither reads the slot. The rule throughout is to
refuse a weight only where the fleet's own wiring reaches it: the template
gates the penalty block on flt_type(flt) > 0 && flt_sel_lead(flt) == 1, so a
Fleet_type = "Off" fleet is skipped, and so is one that follows another
fleet's Selectivity_index -- the group is charged once, on its lead.
(Narrowed in 5.42.1: the template groups by Selectivity_index and
selectivity form, so only a follower sharing the lead's form is skipped. A
follower with a different form leads its own group, is charged, and is
refused. Read that entry, not this sentence, for the rule in force.)
Measured on BS2017SS fleet 1 (NonParametric, N_sel_bins = 8, with the
shipped Sel_curve_pen2 = 12.5 active): ramping the fleet's sel_coff
downward by a constant step per bin takes the JNLL_SEL_NONPARAM row to -29,
-122, -502 and -2032 at steps of 0.5, 1, 2 and 4, and the whole objective to
-6.6e6 at a step of 256 -- quadratic, and falling without bound, while the
composition likelihood flattens and cannot counteract it. The row is exactly
[Sel_curve_pen1 * (N_sel_bins - 1) + Sel_curve_pen2] * step^2 up to a
constant, so the curvature penalty does bind -- on the single second
difference where the ramp meets the coefficient repeated past N_sel_bins --
and the objective diverges only once |Sel_curve_pen1| exceeds
Sel_curve_pen2 / (N_sel_bins - 1), 1.79 here. A smaller negative weight is
merely anti-shrinking rather than divergent; it is refused too, because
rewarding a deviation is wrong at any magnitude. data_check() now stops,
naming the fleet, the slot and the standard-deviation column that sets the
same weight safely.
Sel_shape_dir = "Increasing" is refused on every form but
"NonParametricPM" (9) under Sel_shape_mode = "Directional". It used to
negate Sel_curve_pen1 on every non-parametric form. "NonParametric" (2)
and "NonParametricIntegrable" (13) have no increasing-direction penalty at
all -- the template hard-codes max(d, 0)^2, d being the log-selectivity
drop from one bin to the next -- so "Increasing" never did what it said
there; it fitted the reward above. There is no like-for-like migration: only 9
in "Directional" mode implements the direction, and it is not a drop-in (it
charges its average-selectivity term only when Sel_avgsel_pen > 0, defaults
its penalty range to Bin_first_selected rather than the first bin, reads
Sel_curve_pen3, and refuses Time_varying_sel = "IID"). If you did not mean
an increasing penalty, drop the column -- the default is "Decreasing".
Unlike the sign check above, this one is not waived on a Fleet_type = "Off"
fleet: the direction states what the form can express, and write_data()
persists the weight it sets.
Sel_devmag_sd is refused on "NonParametric" (2) and
"NonParametricIntegrable" (13). It writes Sel_curve_pen3, which neither
form reads: both estimate selectivity deviates, but score them with a Gaussian
density on Time_varying_sel_sd. The column was a silent no-op on both.
A negative sel_curve_pen carried in inits is refused too.
sel_curve_pen is a parameter, not data, so inits from a stored fit
override the Sel_curve_pen columns. Without this a fit saved before 5.42.0
would keep its negative weight through retrospective(), profile(),
run_mse() and self_test() on a workbook the user had already corrected --
measured at -3.81 on a decreasing ramp and -28.99 at three times that ramp,
with the column reading a valid +20. fit_mod() now applies the same rule to
the parameter actually in use and names inits as the source. This closes the
sign only: a stored inits$sel_curve_pen still supersedes an edited
Sel_curve_pen column at any magnitude, silently (a column retuned from 200
to 20 keeps 200 on a refit). That is tracked in inst/dev/CLEANUP_BACKLOG.md.
An apical selectivity linkage on a fleet with Fleet_type = "Off" is
refused. No data are fit to such a fleet, so log_sel_apical was a free
parameter in a flat direction: a singular Hessian and a failed getsd with
nothing naming the cause. It was the only selectivity parameter build_map()
freed without a Fleet_type gate.
A selectivity prior on a fleet with Fleet_type = "Off" is refused, for
the same reason: build_map_selectivity() maps that fleet's log_sel_slp and
sel_inf off, so the prior was evaluated against a fixed value and added a
constant to the objective while constraining nothing. The apical linkage was
given this gate above; the prior path had been left without one.
The negative-weight refusal no longer fires on a slot the fleet's
configuration makes inert. LogisticPM's slots 1 and 3 and
"NonParametricPM"'s slot 3 are charged on the time-varying deviates, so
under Time_varying_sel = "Off" the term is identically zero and the weight is
never read -- verified by objectives bit-identical at +w, 0 and -w on
BS2017SS. A negative value there is inert rather than wrong, and is now
allowed; it is still refused once the deviates are estimated. The slots charged
on the base curve (1 and 2 on the non-parametric forms) are checked whatever
Time_varying_sel says.
data_check() no longer requires Sel_curve_pen2 on a time-varying
LogisticPM fleet. The template says in as many words that
sel_curve_pen(flt,1) -- the 0-based C++ spelling of the same column -- is
unused in that branch, so the column was required and
then ignored -- and the schema, ?BS2017SS and
vignette("model-parameterizations") all say LogisticPM does not use it. The
two weights it does read, Sel_curve_pen1 (the random walk on realized
log-selectivity) and Sel_curve_pen3 (the walk on the age-1 deviates), are
still required, and the message now names what each one weights.
An apical selectivity linkage is now refused on the fleets that actually
share a block, and allowed on the ones that do not. Selectivity_index is a
group key, not a fleet code: the lead is the group's first fleet that is not
"Off", and a group of one shares nothing. The check compared the key to
Fleet_code, which both refused offsets that were perfectly identifiable
(GOAatf fleet 3 and GOA2018SS fleet 11, each the sole member of its group,
and GOA2018SS fleet 9, the lead of the group {9, 10}) and, where a
group's key equalled its second member's code, let the follower
through: build_map_selectivity() freed the cell and
adjust_map_shared_params() then mapped it off, pinning the offset at
exp(0) = 1 with the fit converging and nothing reported. Both now follow
.shared_block_lead(), the rule the map itself applies, and the error names
the lead fleet. The same comparison governed selectivity priors and is
fixed with it. This tightens as well as loosens: a fleet whose Fleet_code
happened to equal its group's key was always allowed before, so an apical
linkage or selectivity prior on one that a lower-numbered fleet leads (group
{2, 5} keyed 5, say) now stops. That block was the lead's, so the linkage
was freeing nothing.
osa_residuals() records the discrete = TRUE method override. A
Gaussian method cannot score a discrete observation, so those composition rows
fall back to oneStepGeneric. The fallback happened but was neither announced
nor written to the returned object's method attribute, so the attribute --
and print() -- named a method no composition row had used.
The non-finite-residual messages now name the "cdf" limitation where it
applies. Both the headline warning and .osa_retry_tail()'s message sent
the analyst to re-check convergence and sample sizes without mentioning the
measurement recorded in the same file: on a random-effects fit with a large
composition data set the Laplace inner problem fails on the depth of
conditioning, and the retry recovers nothing (1879 before, 1879 after). Both
now add that reading, and only where it can hold -- the headline warning shows
it under method = "cdf" on a fit with random effects, since a Gaussian
method returns every residual finite on the same fit. Convergence and the
sparsest compositions remain the first thing to check otherwise.
plot() on an rceattle_osa object warns when it drops residuals. It
filtered to the finite ones and warned only when every residual was
non-finite. Under method = "cdf" the failures are a contiguous tail, so the
survivors are a time-biased subset -- at the documented 1879-of-4538 loss the
figure drew a clean Q-Q panel, with an SDNR annotation, and said nothing.
convergence_diagnostics() reports a NOTE under getsd = FALSE. With
no sdreport the Hessian eigenvalue, pdHess, sdreport and estimability
checks all return nothing, and the battery reported "OK" --
report_tables()$model$converged then printed OK into a SAFE table.
Nothing distinguished "every check passed" from "the strongest checks never
ran". The fit is unchanged. Only estimateMode "Estimate" and "Hindcast"
run the battery, and each refitting diagnostic resolves its own getsd:
retrospective(), jitter(), self_test() and profile() follow whether
the source fit kept an sdreport; reweight() follows that fit's
fit_control$getsd, so a default fit still reads OK; run_mse() refits at
getsd = FALSE, so an MSE's estimation fits now read NOTE -- a status only,
changing no result.
The "NonParametricPM" (9) directional shape penalty now has a
limiting-case and specification check. Nothing verified that
Sel_shape_dir = "Increasing" penalizes an increasing curve on the one form
that implements it; the existing tests
covered the weight = 1/(2*sd^2) arithmetic and one "Decreasing" fit. On a
strictly increasing curve the decreasing direction now charges 0 and the
increasing direction 5.6, mirrored on a strictly decreasing curve, and all
three shapes tested match the ADMB/AMAK sel_like(1) one-sided SSQ recomputed
from sel_coff to 1e-10, driven both through Sel_curve_pen1 and through the
Sel_shape_dir column a user actually writes. This is why the direction is
refused elsewhere rather than implemented there: on 9 it is now checked.
The estDynamics list in vignette("data-without-excel") still offered code
3, retired in 5.35.0 and refused by switch_check() since. The same file
listed the codes correctly 344 lines earlier.
Sel_shape_dir is documented as NonParametricPM-only in the column schema
(which ships verbatim into meta_data_names.xlsx), in ?BS2017SS and in
vignette("model-parameterizations"). All three described the directional
sign as working on every non-parametric form.
osa_residuals(method = "cdf"). The one-step-ahead residual is defined
through the conditional CDF, qnorm(F(x)), and is standard normal by the
probability integral transform whatever shape the conditional has. Until now
the model could not supply that CDF, so only the Gaussian methods were
available; those approximate the conditional as normal and standardize the
observation against its conditional mode. The template now supplies it: the
continuous binomial 1 - I_p(x + 1, n - x) for a composition bin, which is
defined at the fractional counts composition data hold, and pnorm for the
aggregate index, catch and covariate series (src/TMB/comp_osa.hpp, gated by
keep.cdf_lower / keep.cdf_upper under a new osa_mode = 2). The fitted
objective is untouched: the gates are zero except inside a
[TMB::oneStepPredict()] call, and the terms are not even evaluated outside it.
The composition residuals it returns are the only ones that pass a
self-test. Residualizing simulated observations at the parameters that
generated them makes the answer exactly iid N(0, 1), so the methods can be
scored rather than argued about. On BS2017SS over 20 replicates of 4538
composition bins (tools/verify/verify-osa-cdf.R):
| method | mean | sd | lag-1 acf within a composition | KS rejects |
|---|---|---|---|---|
| oneStepGaussianOffMode | +0.103 | 0.918 | +0.060 | every replicate |
| cdf, discrete = FALSE | +0.610 | 1.262 | +0.434 | every replicate |
| cdf, discrete = TRUE | -0.007 | 0.995 | +0.001 | none |
The null standard error on the autocorrelation is 0.015. On the aggregate series, which are genuinely Gaussian, all three agree to 5e-5 and all three pass.
discrete now defaults per method: TRUE under "cdf", FALSE
otherwise, which is what every method that existed before this one already
did. The default changed from FALSE to NULL to express that; passing
TRUE or FALSE explicitly still does exactly what it did, and FALSE under
"cdf" now says in a message that those composition residuals are biased up.
It has to be
TRUE under "cdf": a composition bin holds a count, so its conditional CDF
is a step function and qnorm(F(x)) inherits the step, which is why the
middle row of the table above is the worst of the three rather than the best.
Randomizing over the step removes it: qnorm(F(x) - U f(x)), Dunn and Smyth
(1996), the construction Trijoulet et al. (2023) prescribe. The
randomization is drawn serially under seed after the per-observation loop
returns, so this method stays bit-reproducible with parallel = TRUE, and
attr(osa, "discrete") records what was used.
predicted is NA on every row under method = "cdf".
[TMB::oneStepPredict()] returns Fx, px and nll for this method and no
fitted value, and nothing is substituted: the Gaussian methods' predicted is
a conditional mode, and a marginal fitted value in its place would give the
column two meanings depending on method. The Dirichlet-multinomial rows
below run under a Gaussian method and are blanked with the rest, so the column
means one thing across the object. Fitted values remain in fit$quantities
and in residuals(fit, type = "pearson"). (sd is NA under the package
default too: only method = "oneStepGaussian" ever returns one.)
This is also what resolves the negative composition predicted values
reported as issue #108 point 1: 404 of 4538 rows on BS2017SS, minimum
-10.86, each with a positive-biased residual. Under "cdf" no expected
count is formed at all. The Gaussian methods still report and warn about them,
unchanged; WHAM does the same thing on its own example, so this is not an
Rceattle defect.
Index_distribution = "TruncatedNormal" is exact under "cdf", in the
main call. The template supplies the truncated CDF
[Phi(z) - Phi(-m)] / Phi(m) in closed form, so the family no longer needs
its own "oneStepGeneric" call over (0, Inf). That removes the numerical
integration, its cost, its failure to converge on some random-effects models
(which fell back to an approximate spline), and its side effect of moving the
other fleets' residuals.
Known limitation: compositions at scale under random effects. On a
random-effects model with a large composition data set, method = "cdf"
returns non-finite residuals in bulk and is very slow. On BS2017SS with
random_rec = TRUE (159 random effects, 4538 composition bins), 1879 of
4538 residuals are non-finite against 0 for "oneStepGaussianOffMode" on
the same fit. The failures are a contiguous tail and the same rows come back
clean when residualized on their own, so it is the depth of the conditioning,
not the observations; redoing the tail on a fresh call recovers nothing.
What binds is that depth, not the presence of random effects: the same
method residualizes 1680 composition bins on a 22-random-effect model
correctly. So try "cdf", and when the warning reports non-finite residuals
in bulk, fall back to a Gaussian method for that source, taking its
composition residuals as under-dispersed by about a factor of two and a half
(the numbers are in the bullet below). "cdf" is sound on fixed-effect
models, and on random-effects models for the aggregate and covariate series.
A Dirichlet-multinomial composition cannot use "cdf" and is residualized
with "oneStepGaussianOffMode" instead, announced in a message and recorded
as attr(osa, "method")["DirichletMultinomial"]. The conditional is a
beta-binomial, which has no elementary CDF; the reference implementation TMB
ships (contrib/OSA_multivariate_dists-main/distr.hpp) sums the pmf over
0..floor(x), which is a step function of a fractional count and costs O(x)
beta functions per bin. This is said rather than approximated quietly,
because a missing CDF term does not fail loudly. It makes both tails equal,
giving Fx = 0.5 and a residual of exactly 0 for every bin.
|residual| is censored at 8.04 under "cdf", in both directions, and
osa_residuals() warns when any residual sits there. The upper end is
forced: Fx is recovered from 1 / (1 + exp(nlcdf.lower - nlcdf.upper)) in
double precision, which saturates at the last double below one, so nothing
reading a CDF can report past 8.21 on that side. The lower end is not
forced: that expression takes a small F down to about 1e-308, a residual of
-37. It is censored to match anyway, because an asymmetric ceiling would show as a
long left tail against a wall on the right, which is what skewness in the
residuals looks like. The cost is real: osa_diagnostics() computes SDNR and
the tail statistics on the censored values, and it bites hardest on a short
series where one observation drives the statistic. Which method to reach for
then is measured rather than asserted
(tools/verify/verify-osa-cdf-accuracy.R, a 12-year survey with one
observation multiplied by 200): "oneStepGaussian" reports it uncensored at
38.98 (SDNR 12.89), "cdf" censors to 8.04 (SDNR 4.59), "oneStepGeneric"
compresses it to 3.33 (SDNR 2.21), and the package default
"oneStepGaussianOffMode" returns NaN there, making its SDNR unusable
rather than merely large. That last failure is magnitude-dependent: at a x20
outlier the default is finite and matches "oneStepGaussian", so reach
for "oneStepGaussian" specifically, not for "a Gaussian method", and on
the fleet in question rather than a whole composition source, since it costs
an nlminb per observation. The template shrinks the CDF away from 0 and 1 by
four machine epsilons rather than one, so the ceiling does not land on that
recovery's rounding tie, which produced 6 infinite residuals on BS2017SS.
"cdf" is exact only without random effects, and the recommendation splits
by data type there. With random effects [TMB::oneStepPredict()] integrates
the CDF over the latent states by Laplace, and that integrand is a Gaussian
times a sigmoid rather than a density. Against the exact Kalman innovations of
a linear-Gaussian state space model, fullGaussian and oneStepGaussian are
exact to machine precision while "cdf" errs by 7e-4 to 4e-2 as the latent
state becomes more informative. That is a result about a LINEAR-Gaussian
model and does not transfer wholesale: the Gaussian methods are exact when
the one-step-ahead predictive is Gaussian, which needs the model to be linear
in the random effects, and Rceattle's index and catch are exp() of cumulated
log recruitment deviations through the population dynamics. fullGaussian and
oneStepGaussian cannot differ for a Gaussian conditional, and on a
17-deviation fixture they differ by 0.091 on both index and catch, where
"cdf" differs from oneStepGaussian by 0.017 on index, so no method is
exact for index or catch under random effects and this release does not
claim one. "ecov" is the exception: its conditional genuinely is
linear-Gaussian, the two Gaussian methods agree there to 4e-14, and "cdf"
sits 0.139 away, about a quarter of the residual sd, so prefer a Gaussian
method for that source. It does not reverse for
compositions, whose conditional is discrete and skewed, exactly what the
Gaussian methods get wrong, and by much more. Simulating from a
22-random-effect model with the recruitment deviations redrawn (1680
residuals): oneStepGaussianOffMode gives mean +0.513, sd 0.404 and KS
rejection in 120 of 120 replicates; "cdf" with discrete = TRUE gives mean
+0.006, sd 1.002 and 6 of 120, the nominal 5%.
Both halves of that split are what the SAM authors do across two packages.
stockassessment::residuals.sam() takes the Gaussian default with
discrete = FALSE, their composition package compResidual::resMulti()
hardcodes method = "cdf", discrete = TRUE. Rceattle fits both kinds of data,
so it chooses per observation type inside one call
(vignette("model-diagnostics")).
Every [TMB::oneStepPredict()] call now conditions on the observations that
precede its group (conditional =), instead of discarding them. The
composition and aggregate rows must be split into separate calls under
"cdf" because they need different discrete settings, and without this the
split zeroed the aggregate data terms while the compositions were
residualized, which on a random-effects model moved the composition
residuals by up to 0.99 on a 21-random-effect fixture. It also improves the
pre-existing "TruncatedNormal" split. Fixed-effect models are unaffected,
and verify-refit-like.R is bit-identical.
osa_residuals() says when composition residuals are taken at the default
method. The default is biased on composition data by the scoring table in
?osa_residuals, and stays the default because "cdf" returns non-finite
residuals in bulk on a deeply nested random-effects model. Naming any method,
the default included, is taken as a choice and stays silent.?print.Rceattle work as written.
The version pin pointed at 4.3.0 and the tag convention was documented as
@vX.Y.Z; releases since 5.0.0 are tagged bare. The example links pointed at
blob/master, and there is no master branch.print() on a model_config lists the fields it imposes. fit_mod(config = )
overlays only the fields the config set, so this is what a config will change
on a data object, readable before fitting rather than from the warnings the
fit raises.A non-parametric selectivity form whose deviations integrate.
NonParametric charges its shape penalties on each year's realized curve, so
under random_sel = TRUE the density the Laplace approximation integrates is
tilted and the reported deviation SD is not the SD of the deviations;
fit_mod() refuses that combination, and the random-walk mode for its own
reason (inst/dev/TODO-selectivity.md). Selectivity = "NonParametricIntegrable" (code 13) keeps NonParametric and
NonParametricPM exactly as they are and adds the missing density: it is the
Ianelli base curve with the decreasing, curvature and average-selectivity
penalties charged once on the base coefficients, and the deviations scored by
dnorm(0, sel_dev_sd) on the estimated bins, so random_sel = TRUE estimates
the SD from a complete density.
Time_varying_sel picks the structure, as it does for NonParametric:
"Off" estimates no deviations, "IID" gives independent annual deviations,
and "RandomWalk" gives increments from the base curve with the start-year
increment fixed at 0. Under "Off" the form reproduces the NonParametric
objective to the last digit, at any Bin_first_selected. It reads no
Sel_cap_bin, and fit_mod()'s refusals name it as the alternative. The
per-year mean of a year's deviations is removed by the curve's centring, so
the data never see it: under random_sel = TRUE it integrates out exactly,
under random_sel = FALSE those directions are pure prior.
A non-parametric fleet's coefficients below Bin_first_selected are held at
0. They are mapped off, but the curve reads them: each year is centred by the
log mean over every bin, so a value there shifted the whole curve while no
density scored it. inits from a fit with a lower Bin_first_selected hold
such values: 0.9 in those cells moved Atka2022's fishery objective by 704
nats and year-1 selectivity by 0.21. This affected NonParametric (2) and the
new form 13. NonParametricPM (9) was never affected, because its
branch already zeroed those cells before centring, so no NonParametricPM fit
needs revisiting. A fit started from the build defaults, the golden fits
included, is unchanged.
What the estimated SD is worth. tools/verify/verify-sim-recovery-np-integrable.R
draws deviates at a known SD on Atka2022's fishery (multinomial age
compositions, input sample sizes 2 to 236), simulates the observations and
refits with random_sel = TRUE. The estimate is biased low: 0.24 to 0.26
against a true 0.35 across two runs of eight and ten replicates, every
replicate below the truth (reported SE on the log scale 0.12), 13% low at
0.70, and 9% low at 0.35 with the sample sizes multiplied by ten. Every scored cell has its density and no estimated cell
is unscored (checked cell by cell), so this is the Laplace marginal
likelihood's known downward bias for variance components on small
multinomial samples (Breslow and Lin 1995), not a scoring defect. Read the
reported SD as a lower bound on the process SD unless the compositions are
well sampled; MCMC through tmbstan on the fitted object removes the
approximation.
build_bounds() applies it to every model, single-species
included, and the refit stop below reaches any fit whose rec_pars sit
outside it, estimated or fixed.
A fit saved on the old unbounded ridge, with log alpha or log beta beyond
+/-30, no longer refits: build_bounds() stops because its starting values
are outside the bounds, which takes retrospective(), profile() and
run_mse() on that fit with it. The error names the block and says so.
Restart such a fit from values on the stock's scale
(build_srr(srr_alpha_init =, srr_beta_init =)); the curve it came from was
the degenerate one this release exists to report.msmMode > 0, so check_convergence() returned a NOTE
without looking at the curve, and the flat Pacific hake curve passed. It now
reports a WARN when the curve is flat over the observed SSB range
(Beverton-Holt predicted/asymptote above 0.9 at the lowest SSB), linear (below
0.1 at the highest; for Ricker, a density-dependence factor above 0.9), when a
Ricker peaks below the lowest observed SSB, when log alpha or log beta sits at
the overflow bound, or (with getsd = TRUE) when a log-scale standard error
exceeds 10; the record holds alpha, beta, their standard errors and the
density dependence at both ends of the SSB range per species. A curve held
at its inputs is a NOTE, not a warning. The hindcast standard errors it reads
are now kept in the fit's convergence snapshot, since under an estimating HCR
fit$sdrep is the projection's.tools/verify/verify-sim-recovery-srr-msm.R
imposes a Beverton-Holt curve with its bend inside the observed SSB range on
the two-species fixture (fished with a strong pulse, SSB spanning about 7x)
and refits it from the true values after redrawing the observations and the
recruitment process; a dispersed option starts a log unit away. Over 30
replicates from the truth, species 2 recovers cleanly (log alpha 2.51
against a true 2.55, empirical SD 0.19, mean reported SE 0.20; log beta
-5.64 against -5.66); species 1, whose recruitment is noisier, recovers in
the replicates that converge and runs to the ridge in those that do not.
From dispersed starts species 2 still recovers (2.58 against 2.54) and
species 1 reaches the ridge in half the replicates, which is the behaviour
the new check reports.sample_rec() projects the mean multiplier at both draw sites
(test-functions-sample-rec-agreement.R); one fitted Ianelli case under
predation; the degenerate-curve check on both ridges and both forms.
?build_srr says what the R0 slot starts at under a multispecies curve.log_F
per fleet and year shared by the sexes, so a sex difference in F can only
come from selectivity, and no form had a height parameter: the logistic
family and DoubleNormal peak at 1 for every sex, the non-parametric forms
re-centre each sex, Hake normalizes each sex by its own maximum. The new
selectivity linkage parameter apical (base block log_sel_apical,
[n_fleets, nsex], log scale) multiplies one sex's whole curve by
exp(log_sel_apical), applied after the form and before the shared
normalizer, so it works for every estimated form. Name the fleet and the sex
that holds it, as Stock Synthesis's male-offset option does:
build_selectivity(linkages = list(apical = linkage_spec(~ 1, by = ~ fleet + sex, fleet = 3, sex = "male", priors = list(intercept = lognormal(0, 0.5))))).
The multiplier equals the ratio of the sexes' peak heights only where their
shapes peak equally (the logistic family on an age axis); for a dome with
sex-specific shape it is the pointwise multiplier on that sex's curve. Only
the male:female contrast is identified (the common level is log_F), so
fit_mod() refuses an apical linkage that names no fleet, no sex or both
sexes, one on a one-sex species, a Fixed or AR1 fleet, a mirror fleet, one
with link = "identity", and one on a fleet whose
Sel_norm_scope = "WithinSex" normalization would divide the offset out.
The contrast is informed only by joint compositions (comp_data$Sex = 3), and
fit_mod() warns when the named fleet has none and the offset has no
prior, since nothing then informs the ratio. Naming fleet and sex is
enough: by defaults to ~ fleet + sex for this parameter.
The multiplier is compiled into the template only when a linkage names it,
so every existing model keeps its AD tape and the golden fits are unchanged;
inits and a stored map from an older fit are filled with the block fixed
at 0, so saved fits still refit. tools/verify/verify-sim-recovery-apical.R
is the simulation-recovery harness: on GOAatf's fishery with a true log
offset of 0.693, 120 replicates give a mean of 0.685 (empirical SD 0.28,
mean reported SE 0.35), so the estimate is unbiased and its standard error
conservative.Catchability = "AR1" has
been refused since 5.12.0 and the live form is a q linkage
(ar1(1 | Year) with observe), but two template blocks keyed on the
retired code stayed: one that overwrote index_q (discarding the linkage
offsets) and its AR1 density, plus the index_q_rho parameter only they
read. Both blocks and the parameter are gone; index_q_rho drops out of
parameter_dictionary(), set_phases() and the map. An older fit's inits
and stored map naming it are accepted (the block is dropped as retired),
and a stored map sizing log_pop_scalar by age (before 5.35.0) is
collapsed as inits already were, levels included, so a map that estimated
an age-specific scalar collapses to one per species rather than stopping in
TMB, so retrospective(), profile() and run_mse() on a saved fit still
run. A map name the model has no parameter for is dropped with a warning
(a retired block is dropped silently), so a misspelling no longer fixes
nothing; a map missing a parameter the model has stops with a message
naming it, where TMB used to fail on the template read; a map level no cell
holds is dropped rather than becoming a parameter without a start value;
and a map that estimated an age-specific log_pop_scalar warns that only
the first age is kept. A script that sets inits$index_q_rho keeps running,
with that assignment now inert. No reachable fit changes; the golden fits
are unchanged.CONTRIBUTING.md (setup, tests, branches, what a
pull request owes) replaces the branch table and commit convention in the
developer guide. A new article, Adding a selectivity form, traces
Selectivity = "DoubleNormal" through every file, fits it on GOApollock,
and shows what the drift guards report on a half-finished form;
test-docs-anchors.R checks every path, function and code it quotes. The
C++ template is now published as a Doxygen reference from the site's
Contributing menu (Doxyfile, built by pkgdown.yaml); the growth header's
equations render, and stale @param names in the growth, selectivity and
predation headers are corrected. The forms table no longer calls the double
normal six-parameter: it estimates four (peak, two widths, right-tail floor).
Two defects the trace found are recorded in inst/dev/CLEANUP_BACKLOG.md,
not fixed here: a DoubleNormal fleet with Time_varying_sel = "RandomWalkAscending" silently drops its deviates, and the form's default
starting values describe a flat curve the optimizer does not leave.fit_mod(config =) overlays only the fields the config set. It replaced
data_list$model_config wholesale, so a config from model_config() or
load_config() silently dropped every linkage attached with
build_data(model_config = ) (57 random effects to 0 on a GOAatf survey
rw(1 | Year) fit). Now only the fields the config set, defaults included,
replace the data's, with a warning where they differ (a build_*() field
compared as save_config() writes it). A config saved from a
fit sets every field, so it still reproduces that fit. A script that relied
on the wholesale reset must name every field it means to set.species =, sex = or fleet = filter on a linkage spec now warns
when it does nothing: when by does not include the term, and when it
matches none of the model's levels (the whole spec was dropped, so a male M
prior on a one-sex species vanished silently). Refits stay quiet.|resid| > 3 expected 13.5 flags in a 5,000-residual panel and 0.3 in a
100-residual one. The OSA panel now flags above qnorm(1 - 0.05 / (2 n))
(Bonferroni per panel), so under the model a panel expects 0.05 flags
whatever its size. The Pearson panel keeps 3: those residuals are not N(0, 1).fit_mod() and run_config() say what random_sel and random_q gate: the
Time_varying_* deviations only, with one estimated sd per
Selectivity_index group; linkage random effects integrate regardless.build_srr()'s starting values name the exp(9) mean recruitment start.build_osa_data()'s note on comp_offset names its three fill sites.growth_model > 0) the ages pooled in
the plus group were weighted by survival at a hard-coded M = 0.2 whatever the
species' M; they now use the base M1 at the oldest age (fixed or estimated,
without year-varying offsets). This is still survival at M, not at total
mortality: fishing and predation are excluded, so a heavily fished or preyed
plus group is still somewhat too long. With M1_model > 0 the plus group's
length and weight now depend on the estimated M1. Length, weight and spawning biomass of the
plus group move for every parametric-growth fit whose M is not 0.2: on the
2024 GOA Pacific cod model at its starting values (M = 0.49) the plus-group
mean length goes from 99.89 to 98.80 cm, its weight from 12.28 to 11.48 kg,
and first-year female spawning biomass down 0.66%; refitted, its objective
goes from 6416.12 to 6415.82 and terminal female spawning biomass from
48,360 to 48,219 t (-0.29%). Empirical weight-at-age fits, and so the golden
references, are unchanged.None of the refused configurations below fitted the model it described, so this release stays a minor version.
estDynamics = 3 ("FixedScaledByAge") is retired. It never estimated
an age-specific multiplier on the input numbers-at-age: build_map() freed
only the first age's scalar under predation and none in single-species mode,
so every such fit was the estDynamics = 2 model. Measured on the
fixed-numbers fixture, codes 2 and 3 both gave objective 222899009.1243875
with one free scalar. switch_check() now refuses the code and names 2;
rebuild a stored fit with estDynamics = 2, unchanged. log_pop_scalar and
the reported pop_scalar are one value per species: a script reading
quantities$pop_scalar[sp, 1] now reads pop_scalar[sp] (the GOA CEATTLE
figure scripts are updated), and fit_mod() collapses an older fit's matrix
in inits to its first column. The multiplier is estimated only under
predation (msmMode > 0); in single-species mode nothing informs it, so 2
fits as 1, which the schema, ?BS2017SS and data_check() now say.estDynamics > 0) is refused. Its recruitment is read from NByageFixed
and rec_pars is fixed, so the linkage fitted nothing: on the fixed-numbers
fixture an intercept prior added 486.98 nats to the objective (and so to
AIC) with no free parameter, and a covariate slope was a free parameter with
a zero gradient, which leaves the Hessian singular. A spec with no
species = expands to one row per species, so it is refused too; the message
names the estimated species to put in species = c(...).random_sel = TRUE is refused for Selectivity = "NonParametric" with
Time_varying_sel = "IID" at every Sel_curve_pen setting. Until now
Sel_curve_pen1 = 0 lifted the refusal, but the average-selectivity penalty
is always charged on each year's realized curve and does not scale with the
deviation sd, so the sd that fit reported was the sd of a tilted density
(about 5% of the precision low at sd 0.35, more as the sd grows). Fit with
random_sel = FALSE, the penalized AMAK formulation. (From 5.40.0 the
integrable form NonParametricIntegrable takes
random_sel = TRUE instead; this refusal does not apply to it.)R_hat of -89 and a NaN objective, stock-recruit penalty and dynamic B0
under Beverton-Holt, and a NaN log(alpha * SPR0) in R_hat and steepness
under Ricker. When the model has an identity-link recruitment linkage,
hindcast recruitment, R0, R_init, the penalty curve, R_hat (first year
included) and the Ricker log arguments are kept positive by posfun() (a
0.001 barrier: a badly negative curve returns a value well below 0.001, not
one fish) with the excursion charged to the "Zero n-at-age penalty" row, and
fit_mod() warns on the offset. Projected recruitment and the recruitment
behind SB0 and dynamic B0 are not yet floored and can still be negative under
such a linkage (inst/dev/TODO-srr-multispecies.md item 14); check those
series. Without such a linkage the template is exactly as before, so no
existing fit moves.check_convergence() reports a non-zero or non-finite "Zero n-at-age
penalty" row, naming the species and the root-sum-square excursion below
the floor (WARN under 1, FAIL above or when the row is not finite; thousands
of fish for numbers and recruitment, unitless for the Ricker intercept): a fit
whose numbers-at-age, Ricker intercept or recruitment sat on the 0.001 floor
is not the model as specified, and nothing else showed it..map_switch() passed a
factor through (read.csv(stringsAsFactors = TRUE)), and every downstream
comparison against an integer code was FALSE rather than an error, so
srr_est_mode = factor("LognormalPrior") fitted as code 1 with the prior
dropped. Factors and numeric-looking strings now resolve to the code they
name, and a numeric-looking string outside the map ("99") is an error, as
an unknown name is.Time_varying_sel_sd or Time_varying_q_sd and refitting from
inits was a silent no-op. The columns are read on a fresh build only;
fit_mod() now warns, as it does for Comp_weights, unless random_sel /
random_q estimates the sd.R,
and NA for its stock-recruit quantities. Its rec_pars are fixed, so
R, R0, R_init, avg_R, steepness and SPR0 held the
build_params() placeholder (R0 = exp(9), a flat 8103 that neither the
model nor the user set); in single-species mode the equilibrium SB0 and
B0 were built on it, and with DynamicHCR = FALSE the depletions divided
by it. R is now the first-age input numbers (NByageFixed times
pop_scalar, sexes summed) in the years NByageFixed covers and NA in
any other (a year with no row holds zeros), with no confidence band; the rest
are NA, and plot_stock_recruit() draws its points but no curve. sample_rec() and
retrospective() skip such a species (its rec_dev is mapped out). Under DynamicHCR = TRUE the depletions are the input numbers
relative to themselves and stay reported; under predation MSSB0 replaces
SB0.data_check() says when Sel_norm_scope is not read. On a two-sex
Hake or LogisticPM fleet the column changed nothing, without saying so
(measured identical to every digit on GOAatf fleet 3). Neither form goes
through the shared normalizer, for different reasons: Hake normalizes each
sex to its own maximum in its own block, while LogisticPM does not normalize
at all and reuses Sel_norm_bin as a penalty bin range. The notice names the
reason that applies, and the schema and the vignette say it too.retrospective()'s note on a penalty-form peel with no penalty years now
covers a peel that keeps a single year.Multispecies ConstantF projects at the input F. The multispecies
projection loop in fit_mod() reset every species' log_Ftarget to 0, so a
ConstantF run under predation projected each species at F = 1 whatever
Ftarget was, including a ConstantF estimation model refit inside
run_mse(). Projected catch, SSB and depletion change for those runs; with
HCRorder > 1, so do later species' multispecies SB0. A single Ftarget now
recycles to every species, as ?build_hcr documents (with more than one
species it stopped with a map-size error), and a missing or negative
Ftarget under ConstantF is an error. An Ftarget of 0 is stored as
log F = -999, the value build_params() gives a fleet with no catch, not as
-Inf; the projection was already unfished, only the stored parameter changes.
A single-species projection under a rule that estimates Ftarget starts it
at log F = 0 when inits hold that no-fishing value, as the multispecies
loop already did; before, inits from a ConstantF fit with Ftarget = 0
left the rule's Ftarget at 0, where its gradient is exactly 0.
Projected F_flt and F_flt_age are indexed by fleet, fisheries only.
The projection wrote F_flt by species index, so a fishery's row reported
another species' F, and survey rows took NA times F or a full share of F.
Both are reported quantities only, so no fit changes. Hindcast rows were
already correct.
A species with input numbers-at-age (estDynamics > 0) is projected at
F = 0 and takes no harvest control rule. build_hcr_map() already left
its log_Ftarget / log_Flimit unestimated and the reference-point
penalties already skipped it, but the projection still fished it at those
start values (F = 1 under most rules). Its projected catch is now 0. Its
numbers are input, but a fixed species with a fishery now has a higher
within-year mean abundance in the projection, which changes predation
mortality on and by it, and its own SSB changes when spawn_month > 0.
Ftarget, Flimit, SPRtarget, SPRlimit, SBF and DynamicSBF, all set
by that unestimated F, are NA in fit$quantities; report_tables() blanks
the five it reports with that reason, and plot_f() draws no Ftarget or
Flimit line for the species. mse_summary()
reports the species' P(Fy > Flimit) metrics, and the P(SSB < SSBlimit)
metrics that read SBF, as NA; a fixed species with a fishery reports
catch IAV and P(Closed) as NA, like an unfished one. Its depletion is still
reported.
In the four-species hake model this is arrowtooth, sablefish and California
sea lions; none has a fishery and spawn_month is 0, so its dynamics and
catches do not change, but its summary reports NA where it reported 0.
mse_summary() takes multispecies dynamic depletion from the operating
model's DynamicSB0, as it already did for single-species models.
om_terminal_dynamic_sb0 and om_terminal_depletion_dynamic were read from
the no-fishing refit (OM_no_F). Dynamic SB0 is the OM's own history with no
fishing: the stock-recruit curve, the realized recruitment deviations and the
predation suitability fitted in the hindcast. On the Pacific hake MSE
(MSE_yr2024.R, Beverton-Holt operating model, two simulations) hake's
terminal dynamic depletion goes from 0.99 to 0.78.
remove_F() sets F to 0 from the year after endyr by default, not from
the year after the latest suit_endyr. When the suitability window ended
before endyr it removed fishing inside the hindcast: on the hake MSE,
2020–2023 of a 2023 hindcast. A new styr argument gives the first year
fished at F = 0; under predation it must fall after the empirical-suitability
window (the suit_endyr of every predator with suitMode = 0 and non-zero
fitted suitability), since removing fishing inside it would change the
suitability the model was fit with. As before, the
projection is unfished whatever harvest control rule the model was fit under,
so styr can be no later than the year after endyr.
run_mse() now builds OM_no_F with no fishing
after the original operating model's terminal year, so it matches the OM
through that year. The OM no F: SSB Collapse and OM: SSB Collapse from F
metrics change for runs whose suitability window ended before endyr; an MSE
saved under 5.33.0 keeps its old OM_no_F, so rerun run_mse() to update
them.
None of the refused configurations below fitted the model it described, so this release stays a minor version. Stored fits with them no longer refit.
data_check() refuses an R0 linkage under a stock-recruit curve fitted
in the hindcast. Beverton-Holt and Ricker recruitment read only alpha, beta
and SSB, so in a single-species model R0 is derived from them and the
linkage did not change recruitment. In a multispecies model R0 sets only
the initial recruitment level (R_init), so an intercept-only linkage
(~ 1, a prior or fixed value) is accepted and a covariate or random effect
is refused. Put an environmental effect on productivity through an alpha
linkage, or use srr_fun = "mean" with the R0 linkage.srr_est_mode = "BetaPrior" on a Ricker curve does not
refit; build_srr() has refused the pair since 5.30.0. The prior is on
Beverton-Holt steepness and never applied to a Ricker curve. Rebuild with
srr_est_mode = "Estimated", the model it fitted.srr_est_mode = "Fixed" together with an alpha linkage fixed at its init
(est_phase = 0) is refused. Each fixed alpha at a different value. Keep
the linkage alone.build_srr()
call, with the penalty years inside styr to endyr:
build_srr(srr_hat_endyr = ) on its own resets the curve and its priors to
the defaults (mean recruitment, srr_pred_fun = 0), which drops the penalty.
retrospective() still runs when only srr_hat_endyr is past endyr,
truncating it to each peel.srr_est_mode = "LognormalPrior" and an alpha
linkage intercept prior no longer refits (see Bug fixes): one density
counted twice. Keep one.Catchability = "Estimated-with-prior" with a prior on that fleet's q
linkage intercept is refused: both are priors on the same q. Set
Catchability = "Estimated", or drop the intercept prior.M1_use_prior = TRUE with a prior on that species' M1 linkage intercept is
refused unless M2_use_prior = TRUE, which puts the M prior on total M.
Both are priors on the same M1. Set M1_use_prior = FALSE, or drop the
intercept prior.minage = 0 with a Beverton-Holt or Ricker curve is refused. The curve
read that year's spawning biomass before the model computed it, so it gave
zero recruitment in the hindcast (srr_fun) or in SB0, the reference points
and a proj_mean_rec = FALSE projection (srr_pred_fun, the Ianelli form
included). Use mean recruitment.Alpha priors, fixed values and starting values outside a linkage. Each
still works and warns once from build_srr(); refits are silent.
srr_est_mode = "Fixed": use linkages = list(alpha = linkage_spec(~ 1, est_phase = 0, init = list(`(Intercept)` = <alpha>))).srr_est_mode = "LognormalPrior" on a Ricker curve: use an alpha linkage
prior, which gives the same objective, with init at the prior mean for the
same start.srr_prior as alpha's starting value (supplied, not the default 4): use
srr_alpha_init or init on an alpha linkage.srr_prior stays for the Beverton-Holt steepness prior. Bmsy_lim is
unchanged.
Lognormal priors and the Ianelli stock-recruit penalty are mean-centred
under bias_adjust_proc. With fit_control(bias_adjust_proc = TRUE), the
default, each is centred at -sd^2/2 on the log scale, so the value given is
the prior mean of the natural-scale quantity. The Beverton-Holt steepness
prior and the recruitment deviations already worked this way. With FALSE
the value is the median, as before, but the recruitment deviations lose their
centring too, so a fit with both a lognormal prior and recruitment deviations
cannot be reproduced as it was by this flag. Reproduce an old prior with the
flag on by shifting its input: prior_lognormal(p1 + s^2/2, s),
Catchability_init * exp(s^2/2), a Ricker srr_prior * exp(s^2/2), or
M_prior * exp(s^2) for the old M centre (inst/dev/TRAPS.md). The
Ianelli penalty with centred deviations has no input to shift. A value
between 0 and 1 gives priors a centre that is neither the mean nor the
median. This applies to
prior_lognormal() in every linkage (intercepts, slopes and random-effect
SDs), the Ricker alpha prior (srr_est_mode = "LognormalPrior") and the
catchability prior (Catchability = "Estimated-with-prior"). The penalty's
curve term now treats the curve as the mean of recruitment, as in Ianelli's
EBS pollock model (pm.tpl, Dorn 2002); pm.tpl penalizes its recruitment
deviations without a sigma, which no setting reproduces.
bias_adjust_proc = FALSE gives the AMAK form, where the curve is the
median. Every fit that uses these features with the flag on changes. With
the flag off, only fits with an M prior (below) and penalty-form projections
(next item) change. Wide priors change most: for prior_lognormal(0, 2) the
mean falls from 7.39 to 1 and the median from 1 to 0.14.
In the penalty years the Ianelli form penalizes recruitment twice, as a
deviation from R0 and as a departure from the curve. With the flag on,
centring the penalty lowers the recruitment the two favour by
exp(-sigma_R^2/4): 8.6% at sigma_R = 0.6. Their mean was the geometric
mean of R0 and the curve; it and the median fall alike. Recruitment the
data inform moves less.
Measured changes:
prior_lognormal(0, 2) priors, and they use an M prior. Objectives rise
by 3.0 to 6.8, and hake terminal SSB changes by at most 0.86%. The survey
DM weight falls from 43-49 to 31-35 (MSE_yr2024.R) and from 32-35 to
23-25 (04-mse.R), and the Ianelli Beverton-Holt alpha, fitted then with
no alpha prior, rises from 18.8 to 41.0. Every fit still has a
positive-definite Hessian.bias_adjust_proc = FALSE and no M prior,
such as the GOA pollock 2025 assessment (not refit).For a single-species curve fitted in the hindcast, sample_rec(sample_rec = FALSE) and retrospective() now set the projected recruitment deviation to
the hindcast's mean deviation from the curve, log(mean(exp(rec_dev))),
which equals the multispecies form's log(mean(R / R_hat)). They used
log(mean R) - log(R0), which applied the curve's SSB effect twice: projected
recruitment was off by the hindcast-average R/R0 the curve implies, 8.6% low
for a Beverton-Holt curve at steepness 0.8 and a hindcast averaging 40% of
SB0 (27% at steepness 0.5), and often high for a Ricker curve. This changes
an operating model under run_mse(sample_rec = FALSE), retrospective
forecasts, and dynamic reference points in projection years (and projected F
under DynamicHCR = TRUE). In the model sample_rec() returns, projected
recruitment changes only with proj_mean_rec = FALSE. run_mse()'s
default, sample_rec = TRUE, is unchanged.
sample_rec(sample_rec = FALSE) and retrospective() now set the projected
recruitment deviation for the Ianelli penalty form (srr_fun = 0 with a
curve in srr_pred_fun) to log(mean(R / R_hat)) over the penalty years
(srr_hat_styr to srr_hat_endyr), the mean ratio of recruitment to the
curve. They used the mean log ratio over every hindcast year, which projects
the median recruitment around the curve. This applies whatever
bias_adjust_proc is. The deviation sets recruitment in a retrospective's
peeled years and, under run_mse(sample_rec = FALSE), in the operating
model's added years. In the model sample_rec() returns, the dynamic
reference points (DynamicB0, DynamicSB0, DynamicSBF) always change;
projected recruitment changes only with proj_mean_rec = FALSE; and
projected F changes only under HCRs 5, 6 and 7, through projected SSB with
proj_mean_rec = FALSE and through the dynamic reference points with
DynamicHCR = TRUE. sample_rec() stops if no penalty years fall in the
hindcast; a retrospective peel that ends before the penalty years uses its
own years instead, with a warning.
The M prior is centred on M_prior. It was centred at
log(M_prior) + sd^2/2 for either flag setting, so M_prior was neither its
mean nor its median. It is now the mean under bias_adjust_proc = TRUE and
the median under FALSE. Every fit with M1_use_prior or M2_use_prior
changes, including fits with the flag off.
Dynamic B0 uses the hindcast's realized recruitment for cohorts spawned
before the first model year (minage > 1). Under a stock-recruit curve,
including the Ianelli form, those years used the curve at the first year's
spawning biomass. Under mean recruitment with proj_mean_rec = FALSE they
used that year's R0, which differs only with a year-varying R0 linkage.
Neither is the recruitment the hindcast used. DynamicB0, DynamicSB0 and
DynamicSBF change. With DynamicHCR = TRUE so do the projections, the
HCR 3 target, and biomass_depletion and ssb_depletion. Equilibrium SB0
and SBF do not change. Models with minage = 1, including every bundled
dataset, are unchanged.
Under the Ianelli penalty form, dynamic B0 uses each year's realized
deviation from the curve. Before srr_mse_switchyr the hindcast is on mean
recruitment, so rec_dev there is measured from R0, and dynamic B0 multiplied
the curve by it. On the Pacific hake operating model the curve sat at about
1.5 times R0 in those years, so no-fishing recruitment ran about 1.5 times too
high in every year before the switch. It now takes log R - log R_hat, and
with fishing near zero dynamic B0 equals hindcast biomass, while
numbers-at-age stay above the model's 0.001 floor. Suitability is
unchanged: the dynamic runs use the suitability fitted in the hindcast.
DynamicB0, DynamicSB0 and DynamicSBF change for every penalty-form model
(srr_fun = 0 with a curve in srr_pred_fun), and so does mse_summary()'s
dynamic depletion for a single-species penalty-form operating model. With
DynamicHCR = TRUE so do the projections, the HCR 3 target, and
biomass_depletion and ssb_depletion. Models with a curve fitted in the
hindcast, or on mean recruitment, keep their hindcast dynamic B0.
A linkage intercept fixed at its init (est_phase = 0) holds that value
when fit_mod() is given inits. The init was applied only to freshly
built parameters, so a warm start kept the inits value and mapped it off:
a refit at alpha = 500 warm-started from a fit at 1170 stayed at 1170. It
also wins over srr_alpha_init. This applies to every process with an
intercept linkage (recruitment, M, q, selectivity, growth). Refits through
retrospective(), profile() and the other diagnostics keep their fitted,
or profiled, value, and skip the srr_prior, srr_alpha_init and
srr_beta_init starting values too; profile() over alpha or beta on a fit
with those was flat. A hand-written profile loop through
fit_mod(inits = , map = ) over a linkage-fixed parameter is now held at the
init: use profile(), or give the loop's linkage est_phase = 1. On a fit
with srr_alpha_init or srr_beta_init, jitter() and self_test() now
perturb alpha and beta (they were reset after jittering), and retrospective
peels start from the fitted alpha.
Stock-recruit penalty years must lie in the hindcast. data_check()
refuses srr_hat_styr < styr and srr_hat_endyr > endyr under the Ianelli
penalty. Past endyr the penalty applied to projected recruitment (BS2017SS,
endyr + 5: objective +1,215), and sample_rec() averaged over different
years than the template. A stored srr_hat_endyr is kept when endyr is
lowered; Breaking changes gives the rebuild.
A Ricker srr_est_mode = "LognormalPrior" together with an alpha linkage
intercept prior is refused. They are the same density, which the objective
counted twice.
The warning that a supplied map fixes both stock-recruit parameters no
longer fires on a refit's own map: not for a debug or projection map, nor
for a species whose alpha or beta linkage estimates a covariate or random
effect. An intercept-only linkage still counts as fixed.
data_check() resolves a string msmMode (e.g. "SingleSpecies") before
its multispecies stock-recruit checks; "SingleSpecies" > 0 is TRUE in R.
?build_srr: a linkage on R0 acts under mean recruitment only. Under a
curve fitted in the hindcast a single-species R0 linkage is refused; in a
multispecies model only an intercept-only one is accepted, as the initial
recruitment level. The "Starting values" section gave alpha's default
start as e^3; it is srr_prior (default 4) wherever that is an alpha.?linkage_spec: an intercept's init is on the parameter's natural scale,
a slope's on the link scale.?build_srr now leads with linkages for priors, fixed values and
covariates on R0, alpha and beta. An intercept-only linkage_spec() acts
on the parameter itself: a prior (prior_lognormal(), prior_normal()), a
fixed value (init with est_phase = 0), or one species (species =). For a
Ricker curve the lognormal linkage prior on alpha gives the same objective as
srr_est_mode = "LognormalPrior". srr_est_mode and srr_prior remain for a
Beverton-Holt steepness prior, the one a linkage cannot express. The old
example labelled "with a prior on steepness" set srr_est_mode = "Estimated", which applies no prior and turns srr_prior = 0.8 into alpha's
starting value; it is replaced. The linkages vignette's Recruitment section
gains the same recipes.
?linkage_spec: init is a named list, not a numeric vector. The old
example, c(...), is refused by the function.
srr_fun / srr_pred_fun codes 1, 3 and 5, and srr_indices, are now
errors. These were the environment-driven stock-recruit forms. 4.4.0
removed their environmental term from the template, but build_srr() kept
accepting them with only a soft-deprecation warning. A model built with them
was fitted without its covariate and reported nothing wrong: its objective and
parameter count were identical to the non-environmental model's. The 4.4.0
entry below, which says they "continue to work", was wrong.
build_srr() now stops and names the linkage that replaces them:
srr_fun = 0 with linkages = list(R0 = linkage_spec(~ <covariate>)) for
code 1, and srr_fun 2 or 4 with an alpha linkage for 3 or 5.
srr_indices = k meant the k-th env_data column after Year. A fit made
with code 1, 3 or 5 still refits, through retrospective(), run_mse() and
the other refitting diagnostics, as code 0, 2 or 4; that is the model it
actually fitted. run_mse() shows the warning, but retrospective(),
jitter(), profile() and self_test() suppress it. The one exception is
srr_fun = 0 with srr_pred_fun = 1, which scored the penalty around mean
recruitment: it refits without that penalty, so its objective changes. A code
stored in a data object's model_config is not checked and fits as 0, 2 or 4
without a warning, and a stored srr_indices is ignored. Results produced with
these codes from 4.4.0 through 5.31.0 have no environmental effect and should
be refit with a linkage. No fit without them changes.
R0), which seeds the first year and the initial
age structure, is estimated rather than derived as (alpha - 1/SPR)/beta;
every year with modelled spawners follows the curve. Any initMode is
accepted; under the fished modes (3, 4) R0 trades off against the initial
F. The initial state matters: on the Pacific hake four-species model,
"FreeParams" changed terminal SSB by -35% and "NonEquilibrium" by -25%
against the mean-recruitment fit, while the hindcast curve itself came out
flat (see inst/dev/TODO-srr-multispecies.md). Steepness is reported as 0,
the Ricker positivity penalty on alpha * SPR0 - 1 (which added to the
objective at every evaluation) is skipped, and sample_rec(sample_rec = FALSE) and retrospective() scale the projection by the mean ratio of
recruitment to the curve, since there is no unfished recruitment to scale
against. Single-species fits
are unchanged.A multispecies model can have a stock-recruit curve as a recruitment
penalty. data_check() refused every Beverton-Holt or Ricker curve under
msmMode > 0 (5.12.0), including the Ianelli configuration
(build_srr(srr_fun = "mean", srr_pred_fun = "BevertonHolt")). That
configuration never needed spawning biomass per recruit: the penalty, the
dynamic reference points and -- with proj_mean_rec = FALSE -- the
projection read alpha, beta and SSB only. It is now allowed, and so are
estimating alpha and beta, fixing alpha (srr_est_mode = "Fixed") and a
Ricker prior on alpha. Under the default proj_mean_rec = TRUE the
projection, and so the multispecies MSSB0 derived from it, runs on mean
recruitment while dynamic B0 follows the curve.
Two uses still read spawning biomass per recruit, which is undefined under
predation, and are still refused: a curve fitted in the hindcast
(srr_fun Beverton-Holt or Ricker), whose initial recruitment derives from
SPRFinit, and a prior on Beverton-Holt steepness (srr_est_mode = "LognormalPrior" or "BetaPrior").
Under predation the penalty curve's first-year R_hat is R_init, where it
was -Inf; sim_mod() and retrospective() take log(R_hat) over that
year. steepness is reported as 0, and convergence_diagnostics() records a
NOTE rather than testing the curve against the replacement line. No earlier
fit changes.
A species with input numbers-at-age (estDynamics > 0) has no
stock-recruit prior, Bmsy penalty or curve penalty. Its recruitment is a
placeholder, so those terms scored nothing real, and with an estimated
population scalar (estDynamics 2 or 3) the curve penalty pulled on that
scalar. Only a model combining a stock-recruit curve with such a species
changes. In a single-species model the population scalar is never estimated,
so no estimate changes; only the objective changes, by a constant.
fit_mod() warns when a supplied map fixes both stock-recruit parameters
while the curve still shapes recruitment, as a map reused from a
mean-recruitment fit does. build_srr() stops on srr_est_mode = "BetaPrior" with a Ricker curve, which has no Ricker form and was estimated
with no prior, and warns when Bmsy_lim is given for a non-Ricker curve,
where it is ignored.
estDynamics > 0)
at those numbers. The dynamic reference runs (DynamicB0, DynamicSB0,
DynamicSBF) projected such a species from its first-year numbers on its
placeholder recruitment, so a fixed predator collapsed in the unfished run
and took its prey's predation mortality there with it. In the Pacific hake
four-species model, with arrowtooth, sablefish and California sea lion fixed,
arrowtooth dynamic SSB was 38.6 mt in 2023 against an input 43,490 mt. Hake
dynamic SB0 in 2023 falls from 2.73e6 to 1.95e6 mt, and its depletion
against it rises from 0.516 to 0.723. Only multispecies models with a
fixed-dynamics species change: their dynamic reference quantities, and the
objective when DynamicHCR = TRUE fits the projection against them.Composition Pearson residuals divide by the effective sample size the
likelihood used, not the raw input N. residuals(type = "pearson") always
used the multinomial variance at Sample_size, for every composition source.
The likelihood does not. Under a multinomial (and the MultinomialAFSC
default) the weight column multiplies the log-likelihood, so it is an
effective sample size -- the model's own simulator draws at
n_nom * comp_weights(flt) (ceattle.cpp:3758). The reported residual was
therefore 1/sqrt(w) times the right one: too small on an upweighted fleet,
too large on a downweighted one, and reweight_comps() routinely tunes either
way. Under a Dirichlet-multinomial the proportions are additionally
overdispersed by (n + conc)/(1 + conc), so a DM fleet's residuals were
inflated and read as systematic misfit that was not there.
Age/length composition, conditional age-at-length and diet are each rebuilt on
their own likelihood's scale, because all three differ: comp and CAAL form
their Dirichlet-multinomial concentration on the offset-inflated total, while
diet renormalizes over the prey bins plus its "other prey" balance before
scaling by the stomach sample size. The composition proportion offset
(comp_offset, default 1e-5) is carried onto that scale too, which moves the
smallest bins by ~1e-4 relative even at weight 1.
Each of the three reads its family and weight from the data_list element
that actually holds them: fleet_control$Comp_distribution /
CAAL_distribution for the two composition sources, and the per-species
Diet_distribution for diet. bioenergetics_control is a workbook sheet
name rather than a data_list element, and a diet lookup routed through one
resolves to NULL, silently scoring every predator as a multinomial and then
reading Diet_comp_weights -- a log under a Dirichlet-multinomial -- as a
natural-scale multiplier on the stomach sample size. Where a
Dirichlet-multinomial weight genuinely cannot be recovered, the row falls back
to the multinomial variance and says so; it is not left on the
Dirichlet-multinomial with a substituted concentration, which would inflate
the variance by a factor that looks deliberate and is not.
Each switch is also read through its deprecated spellings (Comp_loglike,
CAAL_loglike, Diet_loglike). switch_check() upgrades those in place when
a model is built, but a fit saved before a rename still carries the old
name, and residuals() runs on the saved object -- so without the fallback an
old fit would quietly resolve to the schema default rather than to the family
it was actually fitted under.
This moves Pearson residuals on essentially every fit. It does not move any
likelihood, parameter estimate or reference point -- residuals(),
plot_comp() and the Pearson panel of plot.rceattle_osa() are the only
outputs affected. Index and catch residuals are untouched (they have no
composition family), and report_tables() reads only those.
test-likelihood-pearson-effective-n.R checks the variance by simulation --
data drawn under the family the likelihood assumes must return residuals with
sd = 1 -- rather than by restating the formula.
The OSA tail statistics and their null intervals now come from one
estimator. osa_diagnostics() reported quantile()'s type-7 interpolated
tail against a simulated null. Both sides are now the r-th order
statistic, whose exact distribution is Beta(r, n - r + 1) on the probability
scale, so the interval is closed-form and covers at its nominal rate (measured
0.94-0.96 for n = 10 to 200). Mixing the two -- an interpolated statistic
against a closed-form order-statistic null -- drops coverage to about 0.86
near n = 50, non-monotonically in n; test-diagnostics-osa-tail-nulls.R pins
the coverage so that pairing cannot come back.
The reported tail is now genuinely the r-th order statistic rather than "the
2.5% quantile", so osa_diagnostics() gains lower_r / upper_r and
lower_p / upper_p naming which order statistic was used and the nominal
probability it sits at. Short series are handled: r is clamped to [1, n],
without which qbeta(p, n + 1, 0) returns 1 and qnorm() returns Inf,
making the upper tail check pass for every series with n <= 19.
A composition fleet switched off returns no Pearson residual, rather than
zero. Comp_weights = 0 multiplies a multinomial fleet's log-likelihood by
zero, so the fleet is not fit and has no effective sample size. The residual
divided by it anyway, giving sd = Inf and a residual of exactly 0 in every
bin -- plot_comp() drew a fleet the model never saw as a perfect fit. Those
rows are now NA. A Dirichlet-multinomial reads the same column as a log, so
0 there is a weight of 1 and is unaffected.
Diet Pearson residuals group by stomach_id where the data carry it. The
grouping was read off fit$data_list, which is the pre-rearrange_data()
list and never holds that element, so the predator/sex/age/year fallback ran
even when the table had an id. Where a dataset holds more than one stomach per
predator-age-year, that pooled them into one normalizing total; the C++ scores
each stomach separately (section 13.2). No bundled dataset carries the column,
so nothing in the package changes.
osa_diagnostics(nsim=, seed=) are ignored and warn when supplied. The
tail null intervals are exact, so nothing in that function simulates. Note
the OSA residuals themselves remain randomized-quantile residuals: pass a seed
to osa_residuals() to control those.OSA and Pearson residual bubbles are drawn on a fixed [0, 6] size scale
in both plot.rceattle_osa() and plot_comp(), so two figures can be
compared by eye -- a free scale made a well-fitting fleet and a badly-fitting
one look alike, and the two figures scaled the same residuals differently.
Residuals beyond 6 are truncated with a warning naming their count and largest
magnitude; the truncation precedes the limit because scale_size_continuous()
sets out-of-bounds values to NA and drops them silently, which would hide
the very points worth looking at. The warning does not list every offending
value: dividing by the effective sample size scales residuals by sqrt(w), so
an upweighted fleet can put thousands of them past the cap at once. Bin axes
take whole-number breaks while there are fewer than 20 bins.
The OSA Q-Q panels annotate the tail statistics and their exact null
intervals in the lower right, alongside SDNR in the upper left, following
afscOSA. plot.rceattle_osa() gains add_sdnr_ci and add_qq_quantiles
(both TRUE) to suppress either annotation.
plot_comp()'s aggregated composition pools counts across years rather
than averaging proportions, so a year with 20 otoliths no longer carries the
same weight as one with 2000, and gains a 95% interval and the input /
effective sample sizes (add_agg_ci, add_agg_n, both TRUE). The interval
is the range holding 95% of the data the fitted model predicts, not a
confidence interval on the mean, so observations outside it indicate misfit.
Its variance is the exact sum of the per-year variances under each fleet's own
likelihood -- the same variances the Pearson residuals are divided by -- with
a normal approximation for the interval itself, which is poor below about 10
expected counts. It also treats the fitted proportions as known, so it is
slightly narrower than one carrying the estimation uncertainty in p_hat.
Rows past endyr are excluded, matching the C++ likelihood's own gate, so a
projection row no longer inflates the input sample size.
The annotation names two effective sample sizes rather than one, because
they answer different questions and the gap between them is the reweighting
decision. ESS (likelihood) is what the model assumed, recovered from the
same per-year variances the band is built on -- sd^2 = p(1-p)/N_eff holds
for every family, so it needs no family branching and covers a
Dirichlet-multinomial's overdispersion as readily as a multinomial weight.
ESS (McAllister-Ianelli) is the tuning target this fit's own residuals
imply, the harmonic mean across years fit_mod() already computes, which is
the unbiased scale to average a ratio estimator on (averaging Neff directly
runs about 50% high). Labelling either one "ESS" alone invited it to be read
as the other. The McAllister-Ianelli line is drawn on multinomial fleets only:
reweight_comps() names and skips a Dirichlet-multinomial fleet, because it
estimates its own weight inside the likelihood, so printing an external tuning
target beside one would invite the adjustment the package refuses to make.
The effective sample size belongs to the observation, so it is summed over
(fleet, species, sex, year) -- data_check()'s own uniqueness key -- not
over years. A fleet may record a female-only and a male-only row in the same
year, which are two observations; pooling them would report half the effective
sample size beside an ISS that correctly counted both, reading as a fleet
downweighted by half. Joint-sex rows (Sex = 3) remain one observation,
because one multinomial spans both sexes.
afscOSA's constructions were deliberately not ported here. Its band is a
binomial at the pooled sample size, and its aggregate effective sample size a
single ratio taken from the pooled composition; measured against the installed
package on data simulated with no misspecification, that ESS returns a median
of 1.16 x ISS with a range of 0.34-9.16, because pooling first discards the
between-year replication.
residuals(type = "pearson") gains an Sd column on composition sources:
the standard deviation the fleet's own likelihood assumes for the observed
proportion, i.e. the denominator the residual was divided by. NA on index
and catch. The aggregated composition band reads it, so the band and the
residuals cannot disagree about the assumed variance.
On the copy carried as attr(osa_residuals(), "pearson"), which renames every
column into that object's style, it is assumed_sd -- not sd, which on the
OSA frame beside it is the conditional standard deviation of the OSA residual,
a different quantity.
Which selectivity forms can give the two sexes different levels.
vignette("model-options-and-functionality") described Sel_norm_bin and
Sel_norm_scope as though they decided whether males and females could be
selected at different levels, and its "Max" / "AcrossSexes" row promised
that "one sex peaks at 1, the other keeps its relative level". Normalization
only preserves a ratio the form already has -- "AcrossSexes" divides both
sexes by one pooled scalar -- so on a logistic both sexes measure 1.00 and the
promise is empty.
The section now says which forms can carry a level difference and which
cannot, with the ratios measured on GOAatf: Logistic 1.00,
DoubleLogistic 1.97, 2DAR1 4.53. Going non-parametric does not help
either, which is the counter-intuitive part: those forms re-centre each sex to
a mean of one every year, so the ratio is a by-product of shape rather than
something the data inform. inst/dev/two-sex-selectivity-example.R fits the
comparison and is now tracked.
A per-sex selectivity linkage no longer fixes the other sex too. When a
linkage supplies the base level -- a fixed intercept (est_phase = 0), or a
slope-only formula -- build_map() masks the base parameter it replaces. The
selectivity branch masked it across every sex of the fleet, so a linkage
stratified on one sex (by = ~ fleet + sex, sex = 2) also fixed the other
sex's inflection or slope at its starting value. That is exactly the sex a
Stock Synthesis-style offset parameterization needs left estimated: the
reference the offset is measured from. The M and growth branches beside it
always honoured the row's own sex; selectivity was the outlier and now
matches them.
A linkage with no sex stratum -- by = ~ fleet, the default -- still masks
both sexes, because the offset then applies to the whole fleet. Only a
selectivity linkage naming a single sex changes, and nothing in the package
or the reference fits does that: the golden models' only linkages are on
composition, which this branch never touches.
test-linkage-selectivity-per-sex.R covers both cases, which nothing did
before.
Sel_norm_bin takes a word. It says where selectivity is normalized to 1,
and encoded three intents in one number with nothing in the value to say
which: blank meant "do not normalize", any negative meant "by the maximum",
and a positive number was an absolute age. It now also accepts Max, Off
(or None), and All on a LogisticPM fleet, where the column is a penalty
age-range rather than a normalization reference. Matching is case-insensitive,
and switch_check() writes the word back, so a saved workbook says what it
does. Sel_norm_bin_upper takes Off the same way.
Every existing value keeps its meaning. Blank, 0, -1, -999 and a
positive bin reach the model as exactly the code they always did; a test pins
all of them.
A normalization bin the fleet is not selected over is now an error.
Previously switch_check() silently overwrote it: anything above nages
was clamped to nages, which is a bin count used as a ceiling for an absolute
age, so on a minage = 3 stock ages 11 and 12 were valid and became 10. The
clamp also ran before data_check(), so nothing downstream could see the
original value. The bounds are now the fleet's own selected range -- below
Bin_first_selected the curve is zeroed, so a reference taken there divides
by nothing -- and a value below it is read as Max, which is what a negative
has always meant. Note Bin_first_selected is a 1-based bin ordinal while
Sel_norm_bin is an absolute age; the conversion is handled for you.
On a stock recruiting at age 0, Sel_norm_bin = 0 is the first age and is
kept as a reference bin rather than read as a flag.
An unreadable value is refused rather than read as "do not normalize".
rearrange_data() is exported and does not always run behind
switch_check(), so it checks the column itself; a typo such as Maxx used
to resolve to blank and silently turn normalization off.
An env_data year outside the model years is dropped on the linkage path,
rather than refusing the fit. Rows are matched to model years by position
(row r is model year styr + r - 1), so a row before styr shifts every
later row and feeds the wrong covariate to consumption and, under
multispecies, to predation mortality. rearrange_data() has dropped such rows
from env_index since 5.25.0, so a fit without a linkage already tolerated
them; a fit with one stopped in .check_env_data_years() with "env_data$Year
must start at the model start year". The same workbook read two ways depending
on whether it used a linkage.
Dropping a row is a warning: covariate rows are assessment input, and 15
workbooks in the sibling repositories carry pre-styr rows. A row whose Year
is NA is kept for the year check to reject by name -- it is unlabelled, not
out of range -- and a table lying entirely outside the model years is an error.
Rows after projyr are dropped too, which can move a linkage fit. They
shift nothing, since alignment runs from the front. But the fixed part of a
linkage formula goes to model.matrix(), so cut(), poly() and scale()
were computed partly over years the model never fits, and the basis they build
depends on the rows supplied. Dropping them keeps the design matrix on the
model years, and moves such a fit: measured at +434.9 on ~ cut(Year, 3) and
+823.8 on ~ scale(temp) at identical parameters, with ~ (1 | Year)
unchanged because a random effect on an unused year integrates out.
No assessment is affected. Of the 171 workbooks carrying env_data across
../Rceattle-models, ../GOA-ATF-ESP, ../GOA-multispecies-assessment and
../GOA_circlulation_study, 15 have rows before styr and none has a row
after projyr. No bundled dataset has either. Refit only a model whose
env_data runs past its projyr and carries a linkage whose fixed formula
transforms a covariate.
A standard error for the selectivity curve.
fit_control(selectivity_se = TRUE) makes [TMB::sdreport()] return
log_sel_at_age, so a fitted selectivity can be drawn with a confidence
interval instead of as a bare line. exp(value +/- 1.96 * sd) stays positive
and is right-skewed, the same treatment log_biomass, log_ssb and log_R
have had since 5.20.0. log_sel_at_age_index is reported alongside it and
gives each row's fleet, sex, absolute age and calendar year -- minage + index and styr + index, not array positions.
On the log scale, not the logit. A logit needs selectivity in (0, 1), and
the non-parametric forms renormalize to a mean of one rather than a maximum
of one: on Atka2022, 58% of sel_at_age exceeds 1 and the largest entry is
3.06, so a logit is undefined across most of the array.
Off by default. The delta method forms a Jacobian of every reported value
against every parameter, so the cost is the product of the two: Atka2022
adds 1,012 values against 584 parameters, and a 16-fleet three-species
assessment around 31,000. The objective is untouched either way -- an
ADREPORT is a report, not a likelihood term.
Only cells that are estimated and strictly positive are reported, because
one log(0) = -Inf on the tape turns every quantity in that sdreport into
NaN, biomass and ssb included, not just selectivity. So the report skips
Selectivity = "Fixed" fleets (read from emp_sel_obs, which leaves 0 wherever
a year has no empirical row -- 228 cells on BS2017SS's EIT_Pollock),
length-based fleets (sel_at_age there is sum(growth_matrix * sel_at_length),
0 for an age overlapping no selected length bin), mirrored fleets and array
padding (neither adds a curve), and every bin below Bin_first_selected, where
selectivity.hpp zeroes the curve outright. The skipped set is fixed by the
data, never by a parameter value. Nothing is lost: a Fixed curve and a mirror
both have their lead's standard error or none by construction.
A reported value can still underflow, which the skip rule cannot anticipate:
a double-normal peaking above the oldest age with a narrow ascending limb puts
exp(-x^2/2) below 1e-308, and 11 cells of Atka2022's first year go to
exactly 0 that way. The log is therefore floored at 1e-300, using a max2()
that is exact above the floor -- on a fitted Atka2022 all 1,012 reported
values are bit-identical to log(sel_at_age) taken without it.
A bin the normalization pins carries an SE of exactly 0, the honest answer for a value that is 1 by construction.
The error belongs to whichever sdreport the fit ends on, which is not
always the fit that estimated the curve. Under estimateMode = "Estimate" with
an estimating HCR, fit_mod() re-optimizes the projection with every hindcast
parameter mapped off, so the standard error of every selectivity comes back
exactly 0. Use estimateMode = "Hindcast", or
fit_control(projection_uncertainty = TRUE). estimateMode = "Projection"
estimates no selectivity at all, and projection_uncertainty cannot change
that, because build_map(debug = TRUE) has already mapped the hindcast off.
fit_mod() warns on each of these before fitting, and warns again if
getsd = FALSE leaves nothing to report.
plot_selectivity(add_ci = TRUE) draws that interval under the curve it
belongs to. It needs a fit run with fit_control(selectivity_se = TRUE) and
declines with a warning naming the option otherwise, rather than drawing a bare
line that would read as certainty -- and declines for the whole figure if any
model in an overlay lacks the errors, since a band on some panels and not
others says "this fleet is certain" rather than "this fit was not asked".
A fleet mirroring another's Selectivity_index reports no rows of its own, so
it borrows its lead's band, but only where the two curves agree: data_check()
merely warns when a shared group differs in a shaping column, and a differing
Sel_norm_bin alone rescales the curve. Every year drawn gets its own band, so
pair add_ci with minyr / maxyr on a time-varying fleet.
A fit reporting an error of 0 at every age is declined the same way, since
the band would be a hairline reading as certainty. That is what an HCR
projection returns; the warning names the estimateMode to refit with.
build_map() names the column and fleet when a switch reaches it as NA.
Selectivity or Time_varying_sel arriving unset on an active fleet failed
at the first == against it, with "missing value where TRUE/FALSE needed" and
no indication of which column or fleet. The message names data_check(), which
is the validator -- build_map() already calls switch_check(), and that
canonicalizes rather than validates. It keeps the validator's exemption: a
fleet that is Off may leave both unset, which now also survives
data_check()'s own selectivity-form checks and the random-effects sigma
loop, neither of which was gated on Fleet_type. Catchability, where NA is
legal on any fleet, is no longer compared as though it were a string.
A 2DAR1 or 3DAR1 fleet no longer warns about Time_varying_sel = "Off".
The AR1 forms supply their own annual deviations, so any other
Time_varying_sel mode is overridden and the builder says so -- but "Off"
asks for no extra deviation layer, which is what an AR1 fleet already has, so
it is not a conflict. Same class of guard as the "Hake" fix in 5.3.0.
report_tables() names its objectives for what they are. model gains
marginal_nll / joint_nll in place of marginal_objective /
joint_objective, and jitter gains best_nll / worst_nll / nll_range
in place of best_objective / worst_objective / objective_range. All six
are negative log-likelihoods on the jnll_comp scale, so a smaller value
is the better fit; _objective said nothing about which, and _likelihood
would have argued for the wrong direction. fit$opt$objective is nlminb()'s
own name and is unchanged.
fit_mod() no longer prints the "Model did not converge" banner. It fired
when an sdreport was requested and did not return, which is a Hessian that
would not invert -- not an optimizer that failed to converge, and a fit can
reach a good optimum with a non-positive-definite Hessian. The convergence
battery reports the same condition as sdreport_failed, says what it means,
and prints at verbose = 0 as well. fit$identified is unchanged.
Two long-standing R CMD check --as-cran warnings are cleared.
print.rceattle_report()'s box-drawing glyphs are written as \u2514\u2500
/ \u251c\u2500 escapes, so the R code is ASCII and the package is
portable -- the printed tree is byte-identical. tibble is declared in
Suggests, which test-data-check-age-coverage.R has used (guarded) for
several versions. CI runs error-on: "error", so neither had ever failed a
build.
quantity_dictionary() described sel_at_age as "normalized to a maximum
of one". It is normalized per Sel_norm_bin, which for the non-parametric
forms -- the default -- is a mean of one, so values above one are ordinary. It
is also 0 below the fleet's first selected bin. The units column said "proportion" for
the same reason and now says "unitless". log_sel_at_age and
log_sel_at_age_index are documented alongside them.
parameter_index() locates every estimated parameter in the model's own
coordinates. TMB names each element of obj$par after its parameter block,
so a diagnostic could report that sel_coff_dev was non-identifiable but not
which of its 392 elements. Returns one row per estimated parameter with
species, fleet, sex, age, bin, year and slot columns plus a
rendered label. Recovered through TMB's own parList(), so it tracks
build_map() exactly: mapped-off parameters are absent, and fleets sharing a
Selectivity_index or Catchability_index appear once, with n_cells
counting the cells they drive. An axis constant across the model is left out
of the label; the structured columns carry it either way.
Ages are absolute, and an axis that omits the youngest ages says so:
init_dev reports ages minage + 1 upward, because ceattle.cpp reads
init_dev(sp, age - 1) over age = 1 .. nages-1 and age minage in the
first year is recruitment. Reading that axis from minage would have named
every initial-age deviation one age too young and never named the oldest.
It describes fit$obj, which under estimateMode = "Estimate" with any HCR
but "NoFishing" is the projection object -- log_Ftarget and log_Flimit
alone, since build_hcr_map() maps every hindcast parameter off. Pass a fit
run with estimateMode = "Hindcast" to browse the hindcast parameters.
The convergence battery names the parameters it flags. estimability,
parameters_on_bounds and hessian_conditioning reported the parameter block
and left the reader to find the element. They now summarise by coordinate,
ordinal axes collapsed to ranges:
[FAIL] estimability 49 non-identifiable fixed parameter(s)
sel_coff_dev GOA_pollock_fishery, bins 1-8, 2013-2018 (41)
parameters_on_bounds gains a where column.
The three do not all count positions in the same vector: estimability and
parameters_on_bounds are captured at the hindcast optimization, while
hessian_conditioning reads the sdreport, which under an estimating HCR
belongs to the projection fit. fit_mod() now stores the hindcast index
alongside the hindcast parameter vector, and each check verifies that the index
it uses names that vector before labelling anything -- a coordinate naming the
wrong parameter is worse than no coordinate, so a mismatch prints block names
alone.
hessian_conditioning is read on the correlation matrix. The covariance
condition number is not scale-invariant (log_F near -2, sel_inf near 10),
so rescaling a parameter moved it without changing the model. The message now
leads with the square root, a ratio of standard errors -- and, because the
matrix is scaled by those errors, one measured against each parameter's own
rather than in the parameters' own units.
data$condition_number therefore means something different from 5.25.1.
Standardising lowers it by 0.4-1.1 orders on the models measured, so at
unchanged WARN 1e6 / FAIL 1e10 the check fires less readily; a badly
scaled but unconfounded fit is no longer flagged. data$se_ratio and
data$covariance_condition_number are new, the latter carrying the old value.
Code comparing condition numbers across this boundary needs the new name.
Selectivity slot labels follow the fleet's Selectivity. Slot 2 of
sel_inf is a descending inflection only for the double-logistic family: it
is logit(right_floor) under DoubleNormal and the free age-1
log-selectivity under LogisticPM.
sel_curve_pen slots are AR1 correlations. The block is estimated only
under 2DAR1 / 3DAR1, where its slots hold logit-scale correlations across
bins, years and cohorts -- not the fleet_control shape and curvature penalty
weights, which are never estimated.
rec_pars slot 2 is the SRR alpha, not steepness; steepness is derived
from alpha and SPR0. growth_log_sd is the standard deviation of
length-at-age at the minimum and maximum age (L1 / Linf), and
log_growth_pars now names its four slots rather than numbering them.
Three parameter_dictionary() dimensions were wrong. log_F declared
[n_fsh, nyrs] against an [n_flt, nyrs_hind] array; log_pop_scalar
[nspp, nyrs] against an age axis; M1_dev_log_sd [nspp, nsex, 2] against
[nspp, nsex] (also corrected in ceattle.cpp). Documentation only, no fit
changes. test-schema-parameter-index.R now checks every block's declared
rank and extent against the built array.
fit_mod() no longer prints the check_estimability table at
verbose = 1. It is one row per parameter, each named after its block, and
the convergence record now carries the same verdict by coordinate. Still at
verbose > 1; fit$identified is unchanged.
The discontinuous-likelihood warning compares two marginal objectives.
Convergence warning (8) tested nlminb's objective against
obj$report()$jnll. The first is the Laplace marginal negative
log-likelihood whenever the model carries a random effect; the second is the
joint negative log-likelihood at the random-effect mode. They differ by
the Laplace correction, which is far larger than the default rel_tol of 1 --
150.96 on a GOA pollock fit carrying two catchability linkages -- so the
warning fired on every such model regardless of whether its likelihood was
smooth. It is now tested against a fresh obj$fn() evaluation, which agrees
to machine precision on all four configurations checked (an ar1/rw q
linkage, no random effect at all, random_rec = TRUE, and 2DAR1
selectivity with random_sel = FALSE).
The comparison is made beside each optimization rather than at the end.
fit_mod() rebuilds the TMB object for the projection, and again under
projection_uncertainty = TRUE, without re-optimizing, and build_hcr_map()
maps every hindcast parameter off, so those rebuilds disagree with opt on
which parameters are random. Comparing across a rebuild is the same units
mismatch: on a BS2017SS fit with one ~ (1 | Year) catchability linkage,
opt$objective of 10207.488519 against 10228.940923 from the
all-parameters-on rebuild, a gap of 21.45 with nothing wrong with the fit.
The same mismatch reached HCR = "ConstantF" under estimateMode = 0,
which rebuilds for the projection and never re-optimizes.
The random_rec == FALSE guard is removed with it: it exempted recruitment
random effects from a check that could not read them, and the comparison now
holds for any random effect, so random_rec = TRUE models get the check
rather than skipping it. No fit changes -- the objective, the joint negative
log-likelihood and the parameter vector are identical either side of this.
No exported argument is removed or renamed. Two configurations that previously
ran are now refused, and neither produced a number anyone should use. A
workbook with the same column under two spellings is also refused where
it previously read, silently keeping one of them -- one file in the sibling
repositories does (GOA cod/Data/2024_cod_pcod_test.xlsx, whose
Bin_first_selected and Age_first_selected disagree on every row):
random_sel = TRUE with non-parametric selectivity, in two cases, each
for its own reason. Fit these with random_sel = FALSE, which is the
penalized formulation the AMAK model intends and is unaffected.
Under Time_varying_sel = "RandomWalk" the walk is scored on the per-year
renormalized selectivity, which never sees the level of each year's
coefficients. Those directions are an improper random effect, and the
deviation sd collapses: 2.7e-8 on Atka2022, a time-invariant selectivity
reported as a time-varying one, reached through 17 NA/NaN function evaluation warnings.
Under "IID" the density itself is proper, but the AMAK shape penalty beside
it is one-sided (max(d, 0)^2), whose second derivative is a step. The
Laplace correction is a log-determinant of that second derivative, so the
marginal objective is only piecewise smooth and the optimizer halts at a kink:
Atka2022 stops with a maximum gradient of 6.8 and reports a deviation sd
27% away from the value the same model reaches with Sel_curve_pen1 = 0
(maximum gradient 4e-4). Setting that penalty to 0 integrates the deviates
against the curvature penalty alone and is allowed.
Ceq outside 1-4. The template switches on it with no other branch, so an
unrecognized value left the temperature function at zero, and with it
consumption and -- under multispecies -- predation mortality.
Non-parametric selectivity no longer returns NaN when a coefficient grows
large. The forms renormalize their log-scale coefficients against
log(mean(exp(coefficients))) once per year, and the realized curve was
stored on the natural scale and re-logged by every penalty. Both halves
overflowed: the sum reached infinity once a coefficient passed about 709, and
the round trip underflowed to log(0) = -Inf from about -745, either way
giving Inf - Inf = NaN with no gradient to recover from. A new
log_mean_exp() computes the mean with a max shift -- algebraically identical,
and the shift cancels out of the derivative too -- and the centred curve is now
held on the log scale rather than recovered from exp(). Measured on
BS2017SS, the objective and gradient are finite to a coefficient of 1e8 in
both the level and the spread direction, where they previously failed at 710
and 730. The four reference models reproduce their optima to 1e-11 relative.
The non-parametric shape penalty cannot read past the last bin. Its
pairs are (bin, bin + 1), so the left bin cannot be the last one, but
data_check() bounds Sel_pen_last_bin by the number of bins. Set to the
final bin, the penalty read one bin beyond the curve -- an entry that exists
because the selectivity arrays are sized by the widest species, but that is
never written -- and charged a shape penalty against a phantom
fully-selected bin. The model clamps the range.
A model with no environmental covariate runs. Ceq 1 (Stewart et al.
1983), 2 (Kitchell et al. 1977) and 3 (Thornton and Lessem 1979) each read
env_index, which ceattle.cpp evaluates for every species on every fit, but
data_check() only guarded Ceq > 1 and described 1 as
temperature-independent. With no covariate column that read went outside the
matrix -- observed both segfaulting and returning plausible numbers on
different runs, the model being built without bounds checking. Consumption now
falls back to the constant fT = 1 of Ceq = 4 where there is no column to
read. data_check() errors on it under multispecies, where consumption drives
predation mortality and so the fit, and warns under single-species, where the
predation section does not run and fT reaches only the reported ration and
consumption_at_age. The missing-value defaults become Ceq = 4 (the only
equation that reads no index) and Cindex = 1 (it is 1-based; the documented
default of 0 was outside the matrix).
An env_data year outside styr:projyr no longer shifts the series.
env_index is read by row position -- row r is model year styr + r - 1 --
so an observation from before styr displaced every later row, feeding an
earlier year's covariate to consumption and, under multispecies, to predation
mortality. This moves real assessments. No bundled dataset is affected, but
15 workbooks in the sibling assessment repositories have pre-styr
environmental rows and need refitting: the GOA CEATTLE 2018-2023
configurations (13-14 years), AI cod and a single-species Pacific hake input
(12 years each), and three GOA circulation-study assessments (2-9 years). The
Pacific hake operating model that 04-mse.R runs is not among them --
env_data starts at its styr of 1980 -- so the MSE and predation reference
objectives are unchanged. A repeated Year, which sits inside the range and so
survived the filter, is now refused by data_check() and de-duplicated before
the model sees it.
An integer Time_varying_sel on a non-parametric fleet is a mode again.
A pre-4.4 workbook stored the two AMAK penalty weights in the time-varying
columns, and the upgrade that moves them across was triggered by the value
in Time_varying_sel -- which cannot tell a shape weight of 4 from the
RandomWalk code 4. Time_varying_sel = 4 on a modern workbook therefore
turned the walk off and overwrote the penalty weights (on GOAcod, 20 and
12.5 became 4 and the deviation sd), while the string "RandomWalk" worked.
The upgrade is now triggered by an empty Sel_curve_pen1, which is what
actually distinguishes the old format.
data_check() refuses a non-parametric fleet whose Bin_first_selected is
above its N_sel_bins, which left it with no estimated coefficients at all.
convergence_diagnostics() reports a variance parameter estimated to
zero. A deviation standard deviation at the floor means the process it
governs is not varying: a model configured as time-varying has fitted
something time-invariant. Atka2022 under random_sel = TRUE with a
non-parametric random walk reaches sel_dev_sd = 2.7e-08; the battery flagged
that fit through max_gradient, which says the optimizer did not converge
rather than naming what went wrong. The new variance_collapse check names
the parameter and the element within it, and fires whether or not the gradient
is clean -- a collapse at a converged optimum is the case with nothing else to
catch it. It reads the recruitment, selectivity, catchability, M1 and linkage
random-effect standard deviations, which are log-scale and O(0.1)-O(1) so one
threshold of 1e-3 means the same thing for each; observation sds,
growth_log_sd and the catchability prior sd are excluded. WARN rather than
FAIL, because a variance at the boundary is a well-posed optimum and a
verdict about the model rather than the optimizer.
Two composition-accumulation columns were never being read.
Accumatation_age_lower and Accumatation_age_upper are the pre-4.4 names of
Comp_accum_young and Comp_accum_old, and were missing from the schema's
alias list -- so a workbook with them got the default of no accumulation
and the setting was ignored. 147 of the 183 fleet_control sheets in the
sibling assessment repositories have them, and 79 set a young-tail value
that folds bins; the GOA pollock configuration is the AFSC one, folding ages 1-2 into
age 3 for the Shelikof survey and age 1 into age 2 for the fishery. On the
2023 GOA pollock workbook, applying it moves the composition negative
log-likelihood from 16333.05 to 16039.59 at identical parameters. Both are
1-based ordinals on the fleet's composition dimension, which equals the age
only when minage is 1 -- it is 1 on every affected workbook. No bundled
dataset has either column, and none of the 147 also has the
canonical spelling, so nothing had to be reconciled.
Est_weights_mcallister is deliberately not treated the same way. Despite
looking like a third case, it is the pre-4.12 name of an OUTPUT --
[fit_mod()] computes Comp_weights_mcallister from the fitted object and
already mirrors the old name for back-compatibility -- so a copy of it in a
workbook is a saved result, not a setting, and reading it as an input would
imply an effect on the fit that it does not have.
A deprecated column name is no longer silently discarded when its canonical
name is also present. read_data() canonicalises fleet_control, so a
script that then assigns a deprecated name creates the old column beside the
new one -- and the upgrade deleted it and kept the canonical value, behind a
routine deprecation message. The two must now hold the same setting, with an
integer switch code and its string counting as the same; anything else is an
error naming both values. Nothing is merged and no blank is filled in from the
other column: NA is a real setting in several of these (Sel_norm_bin and
Sel_cap_bin mean "do not normalize" and "no cap", Proj_F_proportion means
"no F apportioned"), so filling could not express clearing a value and would
move a number without saying so.
Check your own scripts before upgrading, because this has been hiding real
settings and the error will now find them. One script across the sibling
assessment repositories stops on it, at exactly such a setting: the Jack
mackerel 2024 bridging script assigns Sel_sd_prior = 0.35 on four fisheries
after read_data() has canonicalised the workbook's own value of 40 to
Time_varying_sel_sd, so the model was fitting effectively free deviations
where the script asked for penalized ones. The fix is to assign the canonical
name instead of the deprecated one; the error names both.
plot_catchability() draws fitted survey catchability by year, faceted by
fleet, for every fleet with index_data -- a fishery with a CPUE series as
much as a survey. It reads quantities$index_q, the realized q the model
scaled the survey by, so one figure covers an estimated or fixed mean, annual
deviations under Time_varying_q, an environmental regression, a q linkage
such as rw(1 | Year), and the closed-form "Analytical" value. Hindcast
years only: the model does not project catchability, so there is no
incl_proj. A time-invariant q draws a flat line, which is the honest
picture of a constant q rather than an empty panel.
Non-parametric selectivity accepts Time_varying_sel = "IID". The model
scores dnorm(sel_coff_dev, 0, Time_varying_sel_sd) on each estimated
coefficient, so the annual curves are independent deviations about an
estimated base curve rather than a walk. NonParametricPM still takes
"RandomWalk" only -- its deviates are the walk increments, so an
independent-deviate reading of them would not describe the curve the model
builds, and its error message says so and points at NonParametric.
This is what bundled Atka2022 is configured as, so that dataset -- and
the ADMB bridge it comes from -- could not be fitted as shipped. It now fits.
standard_output() spells the length-bin column length_bins, and emits
beg_mid. The schema stockplotr and asar read is the 34 columns of
stockplotr::example_data; convert_output() spells it len_bins internally
and renames on its last step, so 5.24.0 emitted the internal name. Those
packages index by column name, so the wrong spelling is an error in the
consumer rather than an empty column. test-report-tables.R now checks the
emitted names against that package rather than against the constant itself.
report_tables() reports convergence from the hindcast fit. max_gradient
and pdHess came from $opt and $sdrep, which the harvest-control-rule
projection re-optimizes and overwrites, with a fallback to a bare obj$gr()
evaluated at whatever parameter vector was left in place. Both now read the
.conv_hindcast snapshot [fit_mod()] takes before the projection -- the same
source convergence_diagnostics() uses -- so a maximum gradient quoted in an
executive summary describes the assessment.
The reference-point gating reads a harvest control rule held as an integer.
build_hcr_map() matches HCR by name, so a data_list holding the integer
code matched no rule and report_tables() reported NA for an F40% that was
estimated, with a basis saying it was not. HCR is normalized before the
lookup.
Spawning biomass per recruit is no longer wrong for a two-sex species. SPR
is spawning output per total recruit, so the female fraction belongs in it
for every species -- but it must enter once. A one-sex species' schedule is
sex-combined, so the age-varying sex_ratio applies at every age, which is
what mature_females folds in (section 5.4). A two-sex species' sex-0
schedule is already female, so only the recruitment split sex_ratio(sp, 0)
applies -- the same split section 6.6 uses to divide recruitment between the
sexes. Section 6.2 applied the age-varying ratio to both, which re-applied a
split already in the schedule and returned NaN wherever that table stopped
short of the species' own nages.
Found on the 2026 GOA three-species assessment, where arrowtooth flounder
(nsex = 2, nages = 21) returned NA for every per-recruit quantity and so
could not be given a Tier 3 SPR reference point. One-sex species are
unchanged in every configuration: mature_females already applies the
age-varying ratio for them, so the new formula reduces to the old one.
Which two-sex fits move. The change rescales all four SPR schedules
wherever sex_ratio varies with age, and it reaches the objective two ways:
SPRlimit / SPR0 and
SPRtarget / SPR0, so it is live only under HCR = "ConstantF" or an
SPR-based rule (HCR > 3).srr_fun = "BevertonHolt" or "Ricker", SPR0 sets
steepness and R0 and SPRFinit sets R_init, which scales the initial
age structure (section 6.5). srr_fun = "mean" ignores SPR0, which is why
the four golden references reproduce their pinned objectives.Route 1 also needs the rescale to be non-uniform. It scores a ratio of two
SPR schedules, so where sex_ratio is the same at every age with
maturity > 0 the old and new formulas give the same ratio and Ftarget /
Flimit do not move -- only the reported per-recruit quantities do. Route 1
moves a fit only where sex_ratio varies across the mature ages, or where
the schedule stops short of nages and the old formula returned NaN.
That is the split across the live assessments, all two-sex arrowtooth under
HCR = "NPFMC" with srr_fun at its "mean" default:
sex_ratio filled only to age
10 of 21, so every per-recruit quantity was NaN. It gains reference points
it could not previously form. Refit.../GOA-ATF-ESP) has the schedule fully
populated at 0.5 for age 1 and 0.353955 for ages 2-21, and maturity of 0
at age 1 -- a uniform 0.708. Its Ftarget and Flimit are unchanged; the
SPR0, SPRlimit, SPRtarget and SPRFinit it reports were 0.708 of the
correct value and now rise by 1.412. No refit; recheck any quoted SPR.Bundled GOAatf meets the condition -- sex_ratio is 0.5 at age 1 and 0.354
over the mature ages -- so the old formula rescaled every SPR by a clean factor
of 0.708 with nothing in the output to give it away. Built with
srr_fun = "BevertonHolt" and no harvest control rule, the old schedules give
steepness 0.4655 against 0.5516 and R_init 0.9505 against 1.0623 at the
starting values: -15.6% and -10.5% on a fit route 1 never touches.
test-dynamics-spr-two-sex.R checks the result against the per-total-recruit
rule.
ssb, biomass, R, F_spp, M_at_age, N_at_age, SB0 and the objective
are unchanged for a one-sex species, and for a two-sex species on neither
route.
data_check() validates nsex. The model has exactly two sex schedules --
index 0 (females, or the single combined sex) and index 1 (males) -- and reads
nsex straight into loop bounds and array dimensions, so a value outside
{1, 2} was a silent out-of-range read, and an NA aborted data_check()
with R's "missing value where TRUE/FALSE needed", naming no table. nsex is
now checked for length and value, and the M1_base / weight / ration_data
sex-consistency checks look it up by the table's own Species code rather
than by row position, so an unusable or short nsex is reported alongside
every other error instead of thrown or silently recycled.
data_check() no longer demands a full sex_ratio schedule for a two-sex
species. Every remaining use of sex_ratio on a two-sex species is
sex_ratio(sp, 0) — the age-1 recruitment split — so requiring values at every
age rejected a workbook the model never reads past the first column. A one-sex
species still needs the whole schedule, and age 1 is still required for both.
quantity_dictionary() says what every reported quantity is. fit$quantities
holds 99 derived quantities under the model's own abbreviated names, and nothing
said what they meant, what units they were in, or which had a standard error.
The new table gives each one a plain-language meaning, its units, its dimensions,
whether sdreport() provides an SE for it, and what the same quantity is called in
the NOAA standardized assessment output:
quantity_dictionary("ssb_depletion")
quantity_dictionary(process = "reference_points")
quantity_dictionary()[quantity_dictionary()$se, ] # everything with an SE
It is checked against the template and against a real fit by
test-schema-quantity-dictionary.R, so a quantity added, renamed or dropped in
ceattle.cpp fails a test rather than leaving the table quietly wrong.
Two things it records that were easy to get wrong. Units are mt and
thousands of fish throughout (numbers-at-age are thousands, weight-at-age is
kg, so their product is mt) — several C++ comments still said kg. And every
per-recruit reference point (SPR0, SPRlimit, SPRtarget, SPRFinit,
NbyageSPR) is computed only under msmMode = 0 and is exactly zero on a
multispecies fit, which reads as an estimate of zero unless you know.
report_tables() collects what an assessment reports into one set of tables.
Previously the likelihood decomposition, the time series with uncertainty, the
reference points, the fits, Mohn's rho, the jitter and the OSA diagnostics each
came out of a different call in a different shape, so comparing two models meant
assembling a dozen ad-hoc extractions by hand. Sections follow the AFSC Alaska
Groundfish Stock Assessment Guidelines for what a SAFE chapter reports, and every
table has a model column so several fits stack for comparison:
tabs <- report_tables(list(base = fit0, alt = fit1),
retro = list(retro0, retro1), osa = osa0)
tabs$reference_points
It never refits. A retrospective or a jitter is tens to hundreds of
optimizations, so they are passed in already computed; a section whose object is
NULL is simply absent. The standard harvest scenarios of guideline section
4.11.3 are not produced — they need a standard projection module Rceattle does
not have. Projected biomass under the model's own harvest control rule is in
timeseries with era = "fore".
report_tables() gains a parameters section, and reports both
objectives. Asked for directly: where are sigma_R, the estimates of M, and
the likelihood values? parameters joins coef() and vcov() to
[parameter_dictionary()], so every estimated parameter gets its
natural-scale name, its process and its standard error. Estimates are on the
parameter's own scale, so a log_ name needs exp(); sigma_R appears only
when random_rec = TRUE, and a fixed M is not there at all because it was
never estimated — read that off M_at_age.
model now reports marginal_objective and joint_objective rather than one
ambiguous objective, plus n_random. The marginal is what the optimizer
minimized and what AIC uses; the joint is what jnll_comp decomposes, so it
is what the likelihood table sums to. They are equal without random effects
and differ by the Laplace correction with them — on GOAatf with random
recruitment, 744.33 against 816.99. Reporting only one made the two tables
look inconsistent.
standard_output() emits the NOAA standardized assessment format. Relabels
report_tables() output into the schema that the stockplotr and asar packages
consume, so Rceattle results can be plotted and written into a report by the same
tooling used for SS3, BAM, WHAM and FIMS. Names are translated through the
dictionary's standard_label, so ssb becomes spawning_biomass and F_spp
becomes fishing_mortality; a quantity the standard has no word for keeps its own
name rather than being dropped.
That standard describes one stock and has no species dimension, so a
multispecies fit errors unless species selects one, rather than returning a frame
in which two stocks' biomass share a year. The likelihood decomposition is the
exception: jnll_comp counts fleets on rows 1-8 and species on rows 9-20, so
fleet rows go in the standard's fleet column whatever the species
rather than filtered away with it — on a 16-fleet assessment they are 31 of
the 38 rows.
Verified against the 2026 GOA three-species assessment (3 species, 16 fleets,
one two-sex stock, a live sdreport, and its real retrospective, jitter and
OSA objects), not only against the package fixtures. quantity_dictionary()
covers exactly the 99 quantities both its single- and multi-species fits
report.
A reference point a fit never estimated is NA with a stated basis, not
the number the array happens to hold. CEATTLE leaves a value behind in three
cases where there is no reference point, and each one reads as an estimate:
Ftarget / Flimit are estimated only under a harvest control rule that
defines them, and are switched off for a species with no projected fishery.
Unestimated they sit at exp(0) = 1, so the GOA assessment reported a
target F of 1.0/yr. The gating is taken from build_hcr_map() rather
than by reading the HCR switch a second time; the fit's own map cannot be
used, because build_map() sets both to NA in the hindcast map whatever
the HCR.msmMode > 0, SB0 / B0 are overwritten by the MSSB0 / MSB0
inputs, which stand at a 999 mt placeholder until fit_mod() derives them.
B_target = Ptarget * SB0 was therefore reported as 399.6 mt against a
true scale of 1e5-1e6 mt. MSSB0_derived is the flag that distinguishes a
placeholder from a genuinely derived value.msmMode = 0.The depletions are deliberately not blanked alongside SB0: under a
no-fishing rule in multispecies mode the model divides by biomass in the last
projection year, the equilibrated unfished reference, so that series is
meaningful.
A diagnostics list is matched to models by name. retro, jitter and
osa pair with object by name whatever the order; an unnamed list is paired
positionally and says so; a name that is not a model name is an error. That
last one catches passing a single model's osa_residuals() result stored as a
list of parts (index / catch / comp), which would otherwise be
attributed one part per model.
ceattle.cpp. No behaviour change; they were
found while documenting the quantities, and each one described the code as doing
the opposite of what it does. Flimit / Ftarget and Flimit_at_age /
Ftarget_at_age had their descriptions the wrong way round (Flimit is the FOFL
proxy, Ftarget the maximum FABC proxy); NByageF said F = Flimit where the
recursion uses Ftarget_at_age; and index_hat and catch_obs were annotated
(kg) where they are mt or thousands of fish, per the fleet's
Observation_units.parameter_dictionary() is now exported. CEATTLE's parameter vector uses
transformed, abbreviated names (log_M1, R_log_sd, index_log_q), and the
table explaining them existed only as an internal object with @noRd, so it
generated no help page and nobody outside the package could find it. It now
has a user-facing accessor that returns the internal name, the quantity on its
natural scale, the process it belongs to, a one-sentence meaning and the
dimensions, filterable by name or by process:
parameter_dictionary("R_log_sd") # -> sigma_R
parameter_dictionary(process = "selectivity")
An unknown process is an error; an unknown parameter name only warns, so
passing every name from a fit still works when that fit carries parameters
from a branch the dictionary predates. test-schema-parameter-dictionary.R
asserts that every parameter build_params() creates is documented, so the
table cannot silently fall behind the model.
AIC() replaces fit$opt$AIC in the introduction vignette. That slot is
not always there. .fit_tmb() normalizes what TMBhelper::fit_tmb() returns,
and when TMBhelper hands back a nested list the normalizer keeps the inner
opt — where AIC never lived — so the slot silently disappears. Without
TMBhelper the plain nlminb fallback has no AIC either. AIC(fit) works in
every case, because logLik.Rceattle() builds it from opt$objective and
length(opt$par), both of which survive all three paths. The vignette does not
execute by default and test-vignette-api.R checks call signatures rather than
return shapes, so the stale line was invisible to CI.
The introduction vignette now covers the fitted object itself — the S3
methods an Rceattle fit responds to (summary(), plot(), coef(),
vcov(), logLik(), residuals(), simulate(), profile(),
as.data.frame()), the classes the diagnostics return, and where the numbers
live in $quantities versus $estimated_params.
retrospective() warns when a peel is dropped, and the object remembers how
many were asked for. Mohn's rho is averaged over the peels that survive, so a
drop changes a number that gets reported — but it was announced by a
message(), which suppressMessages() hides, and nothing on the returned
object showed it afterwards. It now warns, carries $peels_requested, and
print() reads "3 of 5 peel(s)" with a NOTE and says what rho was averaged
over. jitter() and self_test() still message.
Worth knowing: getsd = TRUE also turns on the non-positive-definite Hessian
check in .refit_converged(), so it can drop peels that getsd = FALSE
keeps, and rho can differ between the two runs of the same model.
A retrospective peel now carries uncertainty on its hindcast. Each peel is fitted twice — a peeled hindcast, then a forecast refit that estimates only the peeled years' F — and the second holds the hindcast fixed, so it reported a standard error of zero for every hindcast year. Peels plotted with no confidence band while the unpeeled model had one.
Under getsd = TRUE the peel is now reported from the same parameters with the
hindcast free. Nothing is re-estimated: on BS2017SS under an NPFMC HCR, Mohn's
rho, every peel's SSB and every peel's objective are unchanged. SSB standard
errors in a common year now fall as data are retained — 248,693 with two years
peeled, 220,476 with one, 179,416 with none — instead of reading zero. Costs
one extra model build per peel.
fit_control(projection_uncertainty) survives a refit. It is carried on
data_list and read back by .refit_like(), so a peel, jitter or MSE refit
inherits how the model was fitted rather than taking fit_control()'s FALSE
default — the failure the bias-adjustment flags already guard against. Under an
HCR it decides whether a fit reports 6 free parameters and no hindcast
uncertainty, or 316 and real errors. Forwarded only when getsd = TRUE.
profile(joint =) moves a whole set of cells on one grid. slots cross
through expand.grid(), so moving a ten-age M schedule together over 13
values was a ten-dimensional factorial — 13^10 fits, not 13. That made an
empirical age-based M, the usual case when M is supplied as a vector rather
than estimated, impossible to profile the way it is normally reported.
joint = "multiply" applies one grid to every named cell as a multiplier on
its current natural-scale value, preserving the shape of the schedule
(ratios between ages) and moving it up or down as a whole; 1 is the fitted
model. joint = "add" adds the grid value instead, preserving the
increments between cells; 0 is the fitted model. joint = "value" sets
every cell to the same value. "none" is the default and unchanged.
Both act on the natural scale, so they need transform = "log" or
"identity" — applying a multiplier to a stored log is a different model,
not a different parameterisation, and an arbitrary transform cannot be
inverted. On a log-scale parameter a grid value that would drive a cell to
zero or below is an error naming the cell and the value, not an NaN fit.
profile(param = "q") profiles a fleet's base catchability.
index_log_q is one entry per fleet, named by Fleet_name, so slots takes
a fleet name as well as an index.
Fleets sharing a Catchability_index share one q parameter — the map copies
the lead fleet's slice across the group — so fixing one member's cell alone
would leave the rest estimating a common q, which is not that fleet's q
profiled. Naming any member now profiles the whole group: the others are
added to slots with a message, and joint becomes "value" so the group
takes one q rather than a factorial over its members.
profile() returns $joint alongside $alias, so print() and the figure
axis can say the grid holds a multiplier or an offset rather than a parameter
value. A grid running 0.7 to 1.3 under an axis reading M1[1, 1, 1] would be
read as an M of 0.7. A catchability axis names the fleet, not its number.
plot_profile(relative = "scaled") puts every component on its own scale.
On a real profile one component routinely dwarfs the rest — on a Bering Sea
pollock M profile the bottom-trawl composition moves 60.5 objective units
while the bottom-trawl index moves 0.697, a factor of 87 — so the large one
sets the y axis and every other curve is drawn as a flat line along the
bottom. Where those components prefer the parameter cannot be read at all,
which is the one thing the figure exists to show.
"scaled" re-zeroes each series at its own minimum and then divides by its
own change over the grid, so every curve runs 0 to 1 and the minima can be
compared directly.
It discards magnitude: a component moving 0.02 and one moving 40 draw
identically, so the figure stops saying whether a component matters and says
only where it points. Raise minfraction alongside it. minfraction and the
legend ordering are computed on the raw change over the grid, before any
re-zeroing or rescaling — computed afterwards every span would be 1, the
filter would drop nothing, and scaling would have quietly removed the guard
at exactly the moment it is needed.
relative keeps "own" (the default, unchanged), "minimum" and "none".
run_mse() no longer expands a time-invariant weight or ration_data
series into the projection. A series supplied at Year 0 is time-invariant
— rearrange_data() fills every hindcast year from the single row — so it
needs no projection rows. Expanding it replaced one legal row with a series
naming the projection years and no year before them, and data_check() reads
any index carrying more than one year as time-varying and requires it to span
styr..endyr. The first assessment advances endyr, the check runs, and the
operating-model refit dies with
Weight data for index = 4 & sex = 1 does not span all hindcast years
recorded as a bare "OM" failure with the message discarded, so every
simulation returned use_sim = FALSE and mse_summary() had nothing to
summarise. Hit on a Pacific hake MSE whose workbook supplies weight indices
4–6 at Year 0; a workbook that dates every weight row never reached it. Both
expansions now skip a group whose rows are all Year 0 and are shared by the
operating and estimation models through one helper.
Two further defects in the same expansion, found reviewing the fix. It read
the group's last table row rather than its last row at or before endyr,
so a series running past its own terminal year projected a value the hindcast
never fitted — not the terminal hindcast year ?run_mse documents holding.
And it added a row for every projection year without checking whether one was
already there: rearrange_data() assigns row by row into the weight array, so
the appended duplicate sorted last and its value silently replaced the
observation. A workbook with catch and weight through 2023 against endyr
2019 hit both, losing four years of weight-at-age. clean_data() already
skips an already-present projection year for catch, and the NByageFixed
expansion beside this one does the same.
run_mse() refuses an operating and estimation model whose terminal or
projection years disagree. The assessment loop takes the row positions of
the catch rows to fill from the estimation model's table and indexes the
operating model's max_catch_hat with them, so the two tables have to
describe the same rows in the same order. Mismatched years read the
exploitable-biomass limit off the wrong years, or off NA, and the run
completed either way — catch advice that is wrong without looking wrong. The
requirement was always there; now it is checked.
run_mse() no longer skips every assessment whose year already carries
catch. The fill that hands the control rule's advice to the operating model
selects projection rows on is.na(Catch). A projection year arriving with a
number in it was passed over: no catch was written, the interval's total came
out 0, and the operating model took the no-catch path — rebuilt at
estimateMode = 3 with its map dropped, rather than advanced. The next
section then overwrote that year with the rebuilt model's realized catch, so
the recorded value did not survive either; the assessment simply did not
happen.
This is what a workbook looks like when endyr has fallen behind the catch
series. On a Pacific hake MSE workbook carrying catch through 2023 with
endyr still 2019, four of six assessments (2020–2023) ran that way, because
clean_data() creates an NA projection row only for a year with no row
already — leaving 2024 and 2025 as the only fillable years. Nothing in the run
reported it.
Catch past either model's terminal year is now blanked to NA at setup, with
a warning naming the model, the years, and the terminal year it read against
— the three things needed to tell a stale endyr from data that belongs in
the projection. Nothing the model uses is
lost: the likelihood scores only Year <= endyr, and no control rule reads
observed catch (HCR 1, constant catch, sums catch_hat). The blanking runs
ahead of the Proj_F_proportion check, which compares recorded catch against
projected exploitable biomass and was tripped by the same rows into forcing a
rebuild the model did not need.
never eaten, exerts no predation) rather than a trailing
clause. Same rule, same species reported.profile_components() and plot_profile(): read a profile for conflict,
not just for precision. profile() returned the total negative
log-likelihood, which answers how well the data determine a parameter but
not whether the data sources agree about it. A smooth, well-behaved total
can be two surveys pulling in opposite directions, and nothing in the total
says so.
profile_components() pulls quantities$jnll_comp out of every grid fit and
returns it as one long data frame — one row per grid point per component,
each cell labelled with the fleet or species it belongs to. That labelling is
the work: the columns of jnll_comp count fleets on the data, selectivity
and catchability rows and species on the priors, penalties and predation
rows, so a cell read off the wrong axis would attribute a survey's likelihood
to a species.
plot_profile() draws it, following r4ss::SSplotProfile(): change in
negative log-likelihood on the y axis, every series re-zeroed at its own
minimum with a point marking it, the total overlaid in black, and components
moving less than minfraction of the total dropped. The spread of the points
along the x axis is the disagreement. plot() on a profile is the same
thing. relative = "minimum" re-zeroes at the fitted optimum instead, and
passing a list of profiles facets them side by side.
Under random_rec = TRUE the total is the Laplace-approximated marginal
likelihood while the components are the inner joint negative
log-likelihood; they will not sum, and profile_components() says so rather
than reconciling them.
The QAR1 process error is reported as a deviate, not as a prior. The AR1
density on index_q_dev (Rogers et al. 2024) was scored into the
"Catchability prior" row of jnll_comp, where it read as a prior on log q.
It now scores into "Catchability deviates", and accumulates rather than
assigns, so it cannot erase anything already in the cell. No fit moves:
the objective is jnll_comp.sum(), so relocating a term between rows leaves
it bit-identical, and the branch is in any case unreachable —
data_check() refuses Catchability = 6 and the live QAR1 form is a q
linkage, ar1(1 | Year) with observe, which scores under "Linkage random
effects". Fixed rather than left in place because a component profile reads
these row labels to say which term is in conflict.
tests/testthat/test-schema-jnll-rows.R is new and pins the three-way
registry the jnll_comp rows depend on — the JnllRow enum, the row labels
in R/6-rename_output.R, and the column axis each row is indexed by. Nothing
linked them at build time before, so a row added to one and not the others
was silent.
profile() reports the parameter in the units you profiled it in.
profile(fit, param = "M1", ...) resolves param to "log_M1", so
print() reported a minimum of, say, 0.28 next to the name log_M1 when
0.28 was an M and not a log M. The alias is now kept in the returned object
as $alias, and both print() and the figure's axis use it. $param is
unchanged — it is still the internal slot the fit mapped off.
self_test() and profile() join the display contract. 5.16.0 gave
convergence_diagnostics(), osa_diagnostics(), retrospective() and
jitter() one way of reporting — an overall status in the same four words,
a one-line verdict, then a compact table. These two were left out, so
vignette("model-diagnostics") claimed a contract the family did not all
keep.
self_test() now reports the fraction of simulations that reached an
optimum, which the returned list could not say on its own: non-converged
runs are dropped before it is returned, so the number of fits returned is
not the number attempted. The number attempted is carried in
attr(, "nsim"). Beneath it, a tally of each returned fit's own
$convergence$status — a separate question, since a replicate can reach a
zero gradient and still carry a NOTE — and a line naming what generated
the replicates, because with processes held fixed the spread across them
carries observation error only and is a lower bound on estimation
uncertainty.
profile() now reports whether the grid brackets the minimum. A profile
whose lowest point is its first or last grid value has run out of grid rather
than found the optimum, and it plots as a perfectly ordinary curve, so
nothing in the numbers says so. For a one-dimensional profile it also gives
the values within 1.92 of the minimum — the 95% profile-likelihood cutoff,
\eqn{\chi^2_1(0.95)/2} — marked open on either side the grid does not close.
A cross-profile gets the edge test but no interval.
Return values are unchanged. Both objects are the list they always were:
sims[["Sim_1"]], length(sims), prof$grid$slot_1 and
prof$nll - min(prof$nll) all index as before, c(sims, list(fit)) and
sims[i] return a plain list of fits, and plot_biomass() and
compare_sim() are unaffected. Only class() gains an entry and print()
gets an opinion.
R/9-retro_and_jitter.R was split into 9-diagnostic_helpers.R
(rceattle-refit-args, .parallel_lapply()), 9-retro_and_jitter.R,
9-self_test.R and 9-profile.R. No code moved between functions and no
behaviour changed.A maturity or sex_ratio held as a matrix no longer aborts
data_check(). The per-species age-coverage check added in 5.19.0 read the
Species column with $, which on a matrix is an error rather than NULL:
the check died with $ operator is invalid for atomic vectors and took every
error accumulated before it with it. read_data() returns both tables as
data frames, but a hand-built data_list may hold either as a matrix, and
every other check tolerated one. Read by column name now.
A Species column that is not a species number is reported. It was
compared with as.integer(), so a column of species names gave NA, NA != i gave NA, and which() dropped it — leaving the row order unchecked,
which is the one thing that block exists to check. A factor column is now read
by its label rather than its level code: the two agree only while the numbers
run 1..nspp, so a table carrying species 1, 3, 4 read as level codes 1, 2, 3
and passed.
Every nages NA reports rather than aborting. 5.19.0 added na.rm to
the age-coverage comparisons, which covers one NA among several; with
every value NA the maximum is -Inf, and two checks downstream still
consumed it — the CAAL column list as 1:-Inf, which errors with "result
would be too long a vector", and NByageFixed as a demand for -Inf
columns. Both are gated on a finite maximum now, so nages is reported by
the check that owns it.
vignette("hcrs-and-mses") attributed the per-fleet Index_distribution
index draw to 5.13.0; it landed in 5.9.0. It also had 5.13.0 breaking
reproducibility for every seeded run_mse(), where it breaks it for two
kinds of run — one using a scalar cap, and one whose control rule reads
reference points from an estimated stock-recruit curve. Both the vignette and
the 5.13.0 NEWS.md entry now say which.
The 5.18.0 entry gives the runtime the common-random-numbers fix cost:
refitting the operating model over its whole projection was worth 9% on a
Bering Sea multispecies MSE projecting to 2040, growing with the length of
the projection. inst/dev/TODO-mse-horizon.md carries the design for getting
it back.
The version-numbering note at the top of this file had a sentence with words missing.
data_check() checks maturity and sex_ratio age coverage per species,
not only against the widest one. The existing checks compare the table's
column count with max(nages), so a table wide enough for the longest-lived
species passes even when a shorter-column species is left short of its own
bins — and the [0, 1] range check then reads over the NAs with na.rm.
Both tables are summed over age. mature_females is maturity, times
sex_ratio where a species is modelled one-sex (ceattle.cpp 5.4), and feeds
hindcast ssb; spawning_biomass_per_recruit() (src/TMB/spr.hpp) sums the
same schedule for SPR. So a maturity gap makes SSB NA for any species, and
a sex_ratio gap does the same for a one-sex species.
On a two-sex species sex_ratio reaches only SPR — which is how this one
hid: the model fits cleanly, and the failure waits until a harvest control
rule asks for reference points, when nlminb reports NA/NaN gradient evaluation, naming neither the table nor the species.
Found on the GOA multispecies workbook, where arrowtooth sex_ratio was
filled to Age10 against 21 age bins. That same data now reports:
sex_ratio is missing values for species 2, ages 11-21 of 21. Spawning
biomass and spawner-per-recruit both sum over every age, so a gap leaves
SSB or SPR0 NA.
Ages past a species' own nages are padding and are not checked, so a ragged
multispecies workbook still passes — asserted against the bundled datasets in
test-data-check-age-coverage.R.
Rows are read by position, since rearrange_data() drops the Species
column and hands the model a matrix whose row i is species i. A
Species column that disagrees with the row order, and a table with fewer
rows than species, are both reported.
An NA in nages no longer aborts data_check(). Six age-coverage
comparisons (weight, ration_data, age_error, maturity/sex_ratio
widths, NByageFixed, CAAL columns) evaluated any(... < nages) or
max(nages) without na.rm, so a single NA made the if() condition NA
and raised "missing value where TRUE/FALSE needed" — discarding every error
accumulated up to that point, including nages' own "not included for all
species".
The workbook column descriptions are reference entries, not essays. The
meta_data sheet's Description column is what an assessment author reads
while filling in a workbook, and several cells had grown past 1,500 characters
— Catchability and Catchability_index the longest. Each now gives what the
column means, its allowed values and its default, then points at
vignette("model-options-and-functionality"), which already carried the
sharing rules and the lead-fleet resolution in full. No allowed value, default
or switch code was dropped, and inst/extdata/meta_data_names.xlsx is
regenerated from the schema. The same trim is applied to the matching
?BS2017SS field dictionary in R/data.R.
"The template" is called "the model" in user-facing text. It is TMB build
jargon for ceattle.cpp, and it collided with the Excel template
write_template() writes — two different objects under one word, in help
pages and error messages. data_check() and sim_mod()'s messages, the
vignettes and the roxygen now say "the model"; write_template()'s own
documentation still says "template", meaning the workbook.
Code comments state current behaviour rather than what an earlier release did, per the repository's comment standard. No behaviour, return value or number changes anywhere in this release.
The NEWS.md entries for the unreleased 5.9.0–5.18.0 line are condensed:
every breaking change, deprecation, moved number and reproducibility boundary
is kept, while derivations and design rationale defer to the vignettes that
carry them.
The SB0/SBF-under-predation limitation in ceattle.cpp is marked
KNOWN LIMITATION and names DynamicSB0 / DynamicSBF as the consistent
alternative, rather than carrying a review-process TODO tag.
Two run_mse() schedules run on the same seed are now on common random
numbers, so a paired comparison of two assessment schedules measures the
schedules. Each assessment's observation draws are seeded on its own year,
from a table built once per replicate, and the operating model is refit over
its whole projection rather than only as far as the next assessment. On
BS2017SS, contamination in a year whose advice is identical by construction
falls from 2.1% to 0.003%; the remainder is optimizer noise.
Common random numbers are still incomplete. One sim_mod() call draws
every year in an assessment interval under that assessment's seed, so a year
sitting inside a longer interval is drawn under a different seed than the same
year in a schedule that assessed it directly. ?run_mse,
vignette("hcrs-and-mses") and inst/dev/TODO-mse-horizon.md say so.
Every seeded run_mse(simulate_data = TRUE) result changes. Runs stay
reproducible from a given seed within a version, and simulate_data = FALSE
is unchanged to the bit.
It costs runtime. Refitting the operating model over its whole projection
rather than only as far as the next assessment was worth 9% on a Bering Sea
multispecies MSE projecting to 2040, growing with the length of the
projection; single-species models saw little change. projyr sizes the AD
tape, so the saving was real -- it just made the draws depend on when the next
assessment fell, which is the one thing a comparison of two schedules has to
hold fixed. inst/dev/TODO-mse-horizon.md carries the design for recovering
it without that dependence.
run_mse() no longer refits the operating model on a shortened horizon.
That saving was 9% on a Bering Sea multispecies MSE projecting to 2040; it
goes because the shortened horizon was what made the draw count depend on the
schedule. inst/dev/TODO-mse-horizon.md records how to recover it.
tools/verify/verify-mse-om-horizon.R is replaced by the gap_crn_clean
check in tools/verify/verify-mse-schedule.R.run_mse(assessment_period =) takes an explicit vector of assessment
years. A single number is the period it always was; a vector is the exact
years an assessment is completed, which is how a skipped assessment is
expressed — a period cannot represent one gap in a regular cycle.
biennial <- seq(om$data_list$endyr + 2, om$data_list$projyr, by = 2)
run_mse(om, em, assessment_period = setdiff(biennial, 2031))
Every year must fall after the operating model's terminal year and within the projection horizon; a year outside it is an error rather than being dropped. A schedule must name two or more years, since one year cannot be told from a period. A scalar must be a whole year of at least 1 that leaves room for one assessment inside the projection. Nothing that produced a usable number is refused.
A schedule that stops short of the projection horizon now warns. Catch is
filled only up to the last assessment, so trailing years keep the NA their
projection rows were created with, while mse_summary() summarises over the
whole projection — understating Average Catch, Catch IAV and P(Closed)
with nothing in the table to say so. A biennial cycle over an odd number of
projection years lands here every time. The warning names the years and both
fixes: extend the schedule, or set projyr to the last assessment year.
run_mse(catch_mult =) takes a data.frame of Year, Species and
mult. A single number or a vector of length nspp is unchanged and
applies in every projection year; a data.frame applies only to the year and
species pairs it lists, and every omitted pair is multiplied by 1. This
expresses a buffer held only while advice is stale, which the vector form
cannot.
buffer <- expand.grid(Year = 2032:2033, Species = seq_len(om$data_list$nspp))
buffer$mult <- 0.90
run_mse(om, em, assessment_period = setdiff(biennial, 2031),
catch_mult = buffer)
Mind which years are stale: the assessment in year Y sets catch for Y+1
onward, so a missed 2031 assessment leaves 2032 and 2033 on 2029's advice.
Species is the species number. An uncovered year, an out-of-range species, a
non-finite or negative multiplier, and two rows for the same year and species
are all errors — each would otherwise read as no reduction in a run that
finishes and looks ordinary. The year bound is the last assessment year, not
projyr. A vector catch_mult is now checked for finiteness and sign.
Note this multiplies catch, not the ABC the control rule produces. Where
realized catch sits well below ABC — GOA arrowtooth flounder, for one —
reducing ABC changes removals only to the extent the fishery attains it.
Scale by recent attainment, 1 - (1 - mult) * attainment, or report the
result as an upper bound. ?run_mse and vignette("hcrs-and-mses") carry the
caveat.
No stored MSE result changes. assessment_period = 1 and a scalar
catch_mult take the same paths they did, and
tools/verify/verify-mse-schedule.R checks that a schedule given as years
reproduces the equivalent period exactly.
Documented: two schedules on the same seed are not on common random
numbers. Measured at 2.1% on a year whose advice was identical by
construction. Pre-existing, and fixed in 5.18.0.
vignette("hcrs-and-mses") gains a worked missed-assessment scenario, and its
reproducibility section now names 5.13.0 alongside 5.9.0. Both moved the
random-number stream, so a seeded run_mse() reproduces across neither.summary() on a fit returns a summary object, not the fit. It was
print(object). It now prints the spec tree, a parameter table with standard
errors, and the likelihood components and their total, returning a
"summary.Rceattle" with $coefficients, $jnll_comp, $objective and
$convergence.
x <- summary(fit) followed by x$quantities no longer works; use fit
directly. Swept across ../Rceattle-models and ../GOA-ATF-ESP: every call
is a bare summary(mod) at top level, so nothing downstream is affected.
Time_varying_sel = "AR1" and Time_varying_q = "AR1" (value 2) are
removed. data_check() errors on them, naming the fleet and the
replacement.
Neither was ever an AR1: the model scored value 2 with the same independent
normal penalty as value 1, and neither deviation block has a correlation
parameter. Nothing that produced a usable number is refused — every fit
under value 2 was an IID fit, and "IID" reproduces it exactly. Of 173
fleet_control sheets across the ecosystem, zero set "AR1" on either
switch.
Time_varying_q is exempt where it is not a mode: under
Catchability = "Environmental" the column holds an env_data column index,
so a literal 2 there is a series number, not a switch value.
The replacement is the linkage grammar, which fits the stationary AR1 these
names promised — sigma is the marginal sd, reducing to IID at rho = 0:
build_catchability(linkages = list(
q = linkage_spec(~ ar1(1 | Year), by = ~ fleet)))
build_selectivity(linkages = list(
inf_asc = linkage_spec(~ ar1(1 | Year), by = ~ fleet)))
It is also more expressive: per selectivity parameter rather than per fleet, with the deviation sd estimated or fixed, priors, bounds and a phase.
The unfished-reference placeholder is recognised by a flag, not by its
value. mse_summary() compared the reported SB0 to the 999 mt
placeholder — a float equality test that would also null a legitimately
derived 999 mt, and that was blind to a workbook supplying its own MSSB0.
clean_data() now seeds a per-species MSSB0_derived, which fit_mod() sets
where it projects under no fishing. A fit saved before the flag existed falls
back to the value test.
print() on a fit closes its tree and lines up its values. Run time is
signif() to match the objective rather than seven digits of wall clock, and
an estimateMode that runs no projection reports
HCR : (not applicable -- hindcast only) rather than NoFishing, which read
as a management choice nobody had made.
Time_varying_sel_sd is validated, as Time_varying_q_sd already was.
build_params() takes its log, so a blank or non-positive value gave a
non-finite objective from inside MakeADFun, naming neither the fleet nor the
column.
Required only where the model reads the sd, which is a property of the
(Selectivity, Time_varying_sel) pair: the logistic family under IID /
RandomWalk / RandomWalkAscending, Hake under IID, and
NonParametric/NonParametricPM under RandomWalk. LogisticPM, Block,
Fixed, 2DAR1 and 3DAR1 are excluded.
Of 173 fleet_control sheets across the ecosystem, one is refused —
Pacific hake/Data/Old files/hake_from_ss.xlsx, whose Hake_fishery pairs
Selectivity = "Hake" and Time_varying_sel = "IID" with a sd of 0. That is
dnorm(dev, 0, 0) and cannot produce a finite objective. Nothing that fitted
is refused.
The model spec tree shows a parameter carrying more than one linkage. The
grammar lets one parameter hold a list of specs — a shared prior plus a
fleet-specific random walk, which is how GOA pollock 2025 is configured — and
print() rendered those rows as NULL. Each spec now gets its own line,
tagged [i/n] when stacked.
plot(fit, what = "ssb_depletion"). The S3 dispatcher offered only
"depletion", which is biomass depletion. In a package whose vocabulary is
the Amendment-56 proxies, spawning-biomass depletion against B40% is the
management quantity, and the generic could not draw it.
The diagnostics share one display contract. osa_diagnostics(),
retrospective(), jitter() and mse_summary() now open with the header
convergence_diagnostics() uses — the object, a status, and a one-line
verdict — before the detail.
No return value changed. retro$mohns, jit$nll, diag$sdnr,
summ$species and every column index as before; only class() gains an entry
and print() gets an opinion.
osa_diagnostics() leads with how many sources failed and the overall SDNR
against its null interval, then a severity-tagged line per source, worst
first.retrospective() judges the terminal peel against print(retro, band = ),
default +/- 0.2. Forecast-skill peels are reported but not judged: a rho
over a forecast horizon is not the quantity that rule was calibrated on.jitter() gains njitter, so the fraction of starts reaching the best
optimum has a knowable denominator — non-converged starts are dropped, so
the count of returned fits is not the count attempted.mse_summary() reports its four blocks and their dimensions, and says that
as.data.frame() on the whole object will not work.The golden regression now runs in CI, and the multi-OS NOT_CRAN guard is
deterministic. test-golden-regression.R carries both skip_on_cran() and
skip_on_covr(), so it fired in neither existing job and the committed
backstop for golden equivalence was never automated. A new deep-checks
workflow runs it nightly and asserts the file produced assertions, since a
skipped file otherwise reports as a pass.
The NOT_CRAN=false override moved to step-level env: on
check-r-package, where GitHub's precedence makes it unshadowable. Through
$GITHUB_ENV it did not hold reliably, and when it slipped the Windows worker
died with 0xC0000005.
The model can be built with array bounds checking.
RCEATTLE_SAFEBOUNDS=true builds with TMB's safebounds, turning an
out-of-range access from a silent write into adjacent memory into an R error
naming the object. tools/verify/verify-safebounds.R drives it, with a
positive control asserting -DTMB_SAFEBOUNDS reached the compile line. The
default is unchanged and no fit gets slower.
The ten vignette chunks the weekly job could not see now run.
RCEATTLE_EVAL_VIGNETTES flips each vignette's chunk default, but a chunk
carrying its own eval = FALSE overrides it — so ten chunks across four
vignettes stayed unexecuted in the one job built to execute them, including
the OSA and process-residual workflows and three run_mse() variants. Exactly
one chunk keeps an explicit eval = FALSE, the install block, and says so.
Cost on an M-series Mac: hcrs-and-mses 2.7 -> 19.2 minutes,
introduction 2.7 -> 3.8, model-diagnostics 1.9 -> 2.3. The weekly job's
timeout went from 120 to 180 minutes, since a timeout loses every vignette's
result rather than one.
?mse_summary says when om_terminal_depletion is NA — any
multispecies run without a harvest control rule, which derives no unfished
reference. Points at om_terminal_depletion_dynamic, which is unaffected.
vignette("model-parameterizations") gives the fishery index predictor.
It still showed the survey snapshot for CPUE, which 5.9.0 replaced for a
fishery with the year-average N̄ = N (1 − e^{−Z}) / Z. The predictor is now
split by Fleet_type, with the Baranov derivation and the SS3
correspondence. The survey half was wrong independently: it wrote
e^{-Month·Z} where the model computes exp(-(Month/12)·Z), an exponent out
by a factor of twelve.
Same vignette: the Selectivity list gained DoubleNormal (8),
NonParametricPM (9) and LogisticPM (11) and the note that there is no 10;
N_sel_bins is described in bins rather than ages; and the four
"Time_varying_sel = 2, sigma is estimated" equation blocks now name what
actually does that, fit_mod(random_sel = TRUE).
Three switch-table rows in vignette("model-options-and-functionality") no
longer break their table. A | inside a table cell is a column separator
even inside a code span, so linkage_spec(~ ar1(1 | Year)) rendered a
spurious fourth column. The replacement code moved out of the cells into
fenced blocks below each table. Nothing catches this today: R CMD check does
not execute the vignettes, test-vignette-api.R parses only the R chunks, and
pandoc renders a ragged row without complaint.
vignette("introduction") plots SSB depletion where it says it does. The
"SSB depletion" example called plot_depletion() — biomass depletion, the
same call as the line above it — so the two examples drew identical figures
under different labels. It calls plot_depletionSSB() now.
The Time_varying_sel soft-deprecation message now names ar1(1 | Year)
alongside rw(1 | Year), as the Time_varying_q message already did.
DynamicSB0 and DynamicSBF decay to spawning on their own mortality.
Both are carried forward on predation mortality solved at their own fishing
rate (M_at_age_dB0, M_at_age_dBF) but then applied M_at_age — the
fished mortality — over the fraction of the year to spawning. They now use
the same M they were propagated on.
Only a model with a non-zero spawn_month is affected, and only under
predation: with spawn_month = 0 the term is exp(0), and with msmMode = 0
the two mortalities are identical. No bundled dataset carries both, so
/golden-check is bit-identical.
The equilibrium pair, NByage0/SB0 and NByageF/SBF, is not fixed
here. SB0 escapes it under msmMode > 0, being overwritten with the
MSSB0 input, but SBF does not: it applies the realized fished-predator
M2 to an F-target equilibrium numbers-at-age array. HCR 5 (NPFMC Tier 3)
reads SBF and SB0 together, so under predation the two legs of that rule
sit on different mortality bases. msmMode = 0 is unaffected — there is no
M2. DynamicHCR = TRUE uses DynamicSB0 / DynamicSBF, which are
internally consistent. Marked KNOWN LIMITATION in ceattle.cpp.
A multispecies model can report depletion against DynamicSB0. With
HCR = 0 the projection runs unfished, so SSB in the last projection year is
multispecies SB0 once projyr is far enough out to equilibrate, and that is
the reference ssb_depletion uses. That arm ran unconditionally, so it also
overwrote the DynamicHCR = TRUE result, leaving no way to get a dynamic-B0
depletion out of a multispecies model. It is now skipped under DynamicHCR.
Runs with DynamicHCR = FALSE are unchanged.
A multispecies fit now carries its unfished SSB (MSSB0) on the returned
object. Under msmMode > 0 the model discards its own equilibrium SB0
and reads the MSSB0 DATA_VECTOR instead, so ssb_depletion is
ssb / MSSB0. No workbook can supply it — clean_data() seeds it at a 999 mt
placeholder and fit_mod() derives the real value by projecting under no
fishing. That derived value was written only into the reorganized copy the
projection refits from, so the returned data_list kept the 999, and
everything that refits from a fitted object — .refit_like(), remove_F(),
and every run_mse() projection — re-entered the model with the
placeholder.
Any multispecies refit carrying a harvest control rule moves. SB0 is not
only the depletion denominator: it is also the HCR threshold and the
posfun floor on ssb_depletion - Plimit, so a refit was comparing spawning
biomass against 999 mt. A single fit_mod() call was already correct — the
fit itself used the derived value — so /golden-check is bit-identical and
no hindcast changes.
mse_summary() no longer reports SSB/999 as a terminal depletion. An
unfished reference is only derived for a run carrying a harvest control rule,
so under HCR = "NoFishing" MSSB0 is still the placeholder. The
multispecies arm divided by it anyway: on the Pacific hake three-species
model OM: Terminal SSB Depletion read 2.68e3 while
OM: Terminal SSB Depletion (Dynamic), which uses the model's own
DynamicSB0, read a sane 0.96. It is NA now, which is what an undefined
reference point should report. The dynamic column is unchanged.
That arm also indexed SB0[sp] on an nspp × nyrs matrix, reading year 1
rather than the terminal year. Masked today because the multispecies
overwrite makes every year identical; corrected so it stays right if that
changes.
vignette("model-parameterizations") wrote "specificed" for the
Time_varying_sel = 3 block form in three places, and both that vignette and
build_map_selectivity() wrote "selecitivty" for the
Selectivity = "NonParametric" form of Ianelli et al. (2018). Text only; no
switch, default or fit changes.random_q = TRUE now estimates the catchability deviation standard
deviation. random_sel has always freed sel_dev_log_sd alongside the
selectivity deviates it integrates out; random_q integrated the catchability
deviates out but left index_q_dev_log_sd mapped, so the deviations were
smoothed at whatever Time_varying_q_sd the workbook happened to carry. The
two are symmetric now: integrating the deviates out is what turns their sd into
a marginal variance rather than one half of a degenerate joint mode.
Any fit using random_q = TRUE moves; every other fit is unchanged, since
the parameter stays mapped without the flag. index_q_log_sd, the prior sd the
assessor sets on q itself, is still never estimated.
How well the sd is informed depends on the series. On a 40-year index with the observation sd held fixed and q deviations injected at a true sd of 0.4, the estimate reaches its lower bound when the observation sd is 0.1 or larger, and recovers 0.06 at 0.05 -- i.e. a short or noisy index can return a value that reads as a constant q. That is visible in the estimate and its gradient, so check both rather than assuming the sd is informed.
A shared selectivity or catchability group now says that no fleet keeps its
own deviation sd. Fleets sharing a Selectivity_index estimate one
sel_dev_log_sd between them. TMB collapses a shared parameter to the mean of
its members' starting values, and that sd is held on the log scale, so the
group starts at the geometric mean of their Time_varying_sel_sd — a
value none of the rows asks for, reached in silence. build_map() now warns
once per group, naming its fleets, their differing values and the geometric
mean the group starts from, and does the same for Time_varying_q_sd across a
shared Catchability_index. Both read the finished map, so they cover only
fleets whose sd is actually estimated: with it fixed, or on an Off fleet,
each fleet keeps its own value and there is nothing to report. No fit changes.
rearrange_data() no longer iterates once over an empty age_error. The
loop used 1:nrow(), which runs for i = 1 and then i = 0 on a frame with
no rows. It is seq_len() now. No bundled dataset ships an empty age_error,
so no fit that works today changes.
R/0-build_srr_and_M.R is split into one file per process. It had reached
1,497 lines and 52 top-level objects carrying six unrelated constructors, only
three of which its name described: 612 lines were catchability, selectivity and
composition linkage machinery. It is now 0-build_srr.R, 0-build_m1.R,
0-build_growth.R, 0-build_catchability.R, 0-build_selectivity.R and
0-build_composition.R. .coerce_switch_arg moves to 0-switches.R beside
the other switch coercion helpers, and the three linkage stratum helpers to
0-build_linkage.R. A pure relocation: every one of the 441 top-level object
bodies is unchanged, and the multiset of non-blank lines across R/ is
identical before and after. Exports, NAMESPACE and the pkgdown reference
index are untouched, since all three index topics rather than files.
The composition and CAAL row normalization is one helper. The two blocks
were the same five lines twice; .normalise_rows() replaces both. It divides
by the row sums instead of going through t(apply()), which returned a
transposed matrix for a single-bin composition; every multi-bin result is
unchanged. The zero-row guard stays.
Four stale or incorrect source comments corrected, in build_params(),
clean_data(), the model's fishing-mortality section, and the two
shared-parameter blocks in build_map() that still claimed fit_mod()
suppresses their warnings. Documented in inst/dev/CLEANUP_BACKLOG.md.
Reference points ignored an estimated stock-recruit curve unless the
projection also used it. Reference-point recruitment is set by two arms:
one for mean recruitment, requiring proj_mean_rec = TRUE and no curve, and
one for the curve, requiring proj_mean_rec = FALSE. A model with an
estimated Beverton-Holt or Ricker and proj_mean_rec left at its default of
TRUE matched neither, so NByage0 and NByageF stayed 0 for every year
after the first and SB0 was the initial cohort decaying away — on a five-age
fixture, 3.344 falling to 1.230 over six years instead of holding at 3.344.
SB0 in the terminal year is the depletion reference HCR 5 and HCR 6 read, so
this inflated perceived depletion and the catch advice that follows from it.
The curve arm now applies whenever srr_pred_fun is a stock-recruit form.
Projection recruitment is unaffected — that switch is read separately, so a
mean-recruitment projection stays mean. No bundled dataset or golden reference
reaches the gap, because the default srr_pred_fun is mean recruitment.
Ten writes to the male slot ran past the end of the sex dimension. Arrays
carrying a sex are dimensioned by max_sex, so where every species is
one-sex that dimension has length 1 and sex index 1 does not exist. The "males
take 1 - sex_ratio" half of each recruitment and sex-ratio assignment wrote
it unconditionally. The value is 0, but the index is out of range and the
model is compiled without bounds checking, so it was a silent write into the
next age of the same species and year. Reproduced on BS2017SS with
safebounds = TRUE; guarded, the same fit is clean at an unchanged objective.
Affects BS2017SS, BS2017MS, and any model whose species are all one-sex.
comp_data$Sex was never checked against nsex. M1_base, weight and
ration_data are all validated; composition was not. A joint row (Sex = 3)
on a one-sex species is the damaging case, because the two registries
disagree: check_composition_data() sizes it at nages while the model
writes it out to nages * 2, so the surplus landed in the next observation's
predicted composition and silently changed that observation's likelihood.
Sex = 2 on a one-sex species is the same class — the model reads a sex
index that does not exist. Both are now refused with a message naming the
species. Sex = 0 and Sex = 1 are unaffected.
build_srr(proj_mean_rec =) was documented backwards. The roxygen said
0 = mean recruitment and 1 = the stock-recruit relationship; the model
reads 1 as mean recruitment and 0 as the relationship, and the argument
defaults to TRUE. Anyone who set it from the documentation got the opposite
of what they asked for. Corrected, with the reference-point behaviour stated.
Recruitment at minage > 1 read memory instead of spawning biomass.
Recruits arriving at age minage in year yr were spawned in yr - minage,
so for the first minage - 1 hindcast years the stock-recruit relationship
indexed ssb() at a negative column. Eigen does not bounds-check in a release
build, so this did not error -- it returned whatever was in adjacent memory.
On a nages = 5 Beverton-Holt fixture, recruitment in those years came back
as 8.6e-314, is.finite() reported TRUE, and the objective moved from
14407.38 (minage = 1) to 15162.60 (2) and 17532.15 (3) purely from the
values read. It was only reachable with an active stock-recruit relationship:
srr_fun = 0 never calls the relationship with a spawning biomass.
Those years now take R_init, equilibrium recruitment at F = Finit, with
the recruitment deviation already estimated for them — the same
R_init * exp(rec_dev) the first year is built from, so the whole
pre-start-spawned block rests on one equilibrium. This follows Stock
Synthesis, which covers the pre-start period with an equilibrium age
composition plus early recruitment deviations rather than extending the
relationship backwards; WHAM avoids the boundary by fixing the recruitment lag
at one year regardless of recruit age. Not R0, which in a later year is
still the starting value. On the same fixture the objective now moves
14407.38 / 13959.97 / 13587.67 across minage 1/2/3.
All four sites are guarded. Expected recruitment takes the same R_init
anchor, so the stock-recruit penalty scores the deviation alone — there is no
spawning biomass to carry information about the curve's level. The reference
points keep the relationship and only clamp the lag into range.
No fit anyone runs today changes: every bundled dataset and all three live
assessments are minage = 1, where the guard cannot fire. The four
golden-reference models are unmoved.
CAAL_distribution = "MultinomialAFSC" now works. CAAL_distribution
shares comp_loglike_map with Comp_distribution, so the value passed
validate_switches() and then died inside the model with
Invalid 'caal_ll_type' -- the CAAL dispatch had only cases 0 and 1. The AFSC
multinomial pseudo-likelihood is now implemented for conditional age-at-length
in the same form already used for age composition, reading the CAAL row's
proportions with its own sample size and weight. As for age comps it is a
pseudo-likelihood rather than a density, so OSA residuals are taken under the
full multinomial and the simulation draw is multinomial.
A model with nlengths < nages wrote past the end of two matrices.
age_hat and age_obs_hat are indexed by age -- up to nages * 2 for
joint-sex composition -- but were sized from comp_obs, whose width is the
number of Comp_ columns in the workbook. For a model whose composition data
are all length-based with fewer length bins than ages, that width is
nlengths and the writes ran off the end. Unchecked in a release build, so
the effect was a silent write into adjacent memory rather than a crash. Both
are now sized from the widest age index the model's own composition rows will
be written at -- nages, or nages * 2 for a joint-sex row -- and never
narrower than the observations. Sizing them at nages * 2 unconditionally
would also close the overrun, but age_hat is REPORTed and assessment scripts
read it, so a single-sex model must not gain trailing all-zero columns it
never had.
A scalar cap is now an annual ceiling, not an assessment-interval one.
run_mse() filled catch for every year of the assessment interval and then
tested the cap against the sum over all of them, so at
assessment_period = 2 a one-year cap was applied to a two-year total,
roughly halving projected catch. The cap is now applied within each projection
year.
The same line carried a second and larger defect, so any stored MSE result
using a scalar cap changes, at every assessment_period including 1, and
whether or not the cap ever bound. It was
ifelse(sum(x) > cap, cap * x / sum(x), x) with a length-1 test, and
ifelse() returns the shape of its test -- so whichever branch was taken was
truncated to its first element and then recycled across every row. Two species
landing 80 t and 20 t came out as:
| scalar cap | old | new |
|---|---|---|
| 50 (binds) | 40 t, 40 t -- 80 t total, over a 50 t ceiling | 40 t, 10 t |
| 500 (never binds) | 80 t, 80 t -- 160 t total, catch invented | 80 t, 20 t |
The non-binding case is the more serious one: a cap set generously enough to
be inert still rewrote every species' catch to the first species' value. Only
an exactly equal split across species was unaffected. The species-specific
vector form of cap was always per row and is untouched.
mse_summary() scored HCR 2 against a hardcoded depletion. The estimation
model's P(SSB < SSBlimit) used a literal 0.5 * 0.35 under HCR 2, on the
depletion scale, while the operating-model arm scored the same rule as an
absolute 0.5 * SBF -- so the EM/OM cross-tab compared two different
criteria. In single-species mode the EM now reads ssb_limit_thresh(), the
helper the OM already uses, so both sides apply one rule and it cannot drift.
Plimit is not that rule: it defaults to 0, which would report a default
single-species ConstantF run as never overfished.
Under msmMode > 0 both sides still fall through to Plimit, so a
multispecies ConstantF run left at the default Plimit = 0 reports
P(SSB < SSBlimit) as 0. That is the operating model's own long-standing
multispecies rule and the two sides agree, so it is left alone here -- but set
Plimit explicitly on a multispecies ConstantF run.
A reproducibility boundary, for two kinds of run. A seeded run_mse()
from 5.12.0 does not reproduce here if it used a scalar cap — the rewrite
above changes the catch taken, cap binding or not — or if its control rule
read reference points from an estimated stock-recruit curve, which the
NByage0 / SB0 fix above moves. A run with neither is unaffected by this
release. That is unlike 5.9.0 and 5.18.0, which move the random stream for
every seeded run; vignette("hcrs-and-mses") lists all three.
fit_mod() no longer suppresses every warning build_map() raises. The
suppression hid messages that exist to be seen -- an M1 model asking for
sex-specific mortality on a single-sex species, a Time_varying_sel the
selectivity form ignores, a Laplace request the map cannot honour. Each
changes what is estimated. They are now de-duplicated and passed through, so
a retrospective or MSE that re-enters fit_mod() once per peel or simulation
prints each distinct warning once rather than hundreds of times.
random_sel = TRUE is now refused on a fleet with Time_varying_sel = "Block", instead of producing a NaN objective. Selectivity blocks are fixed
effects — one slope and inflection per block, with no penalty — but
fit_mod() added the block parameters to TMB's random regardless, and the
model scores selectivity deviates only under "IID", "RandomWalk" and
"RandomWalkAscending". They were integrated against no density, with
sel_dev_log_sd mapped out so there was no variance to estimate either, and
the fit died with TMB's NA/NaN gradient evaluation — which names neither
selectivity nor random_sel. The error now says which fleets and what to
change.
No bundled dataset or reference model reaches this, and no fit that works
today changes. "Block" remains fully supported with random_sel = FALSE.
A Beverton-Holt or Ricker stock-recruit curve can no longer be combined
with msmMode > 0. data_check() now refuses it. Those curves anchor
steepness, R0 and R_init on spawning biomass per recruit, and SPR is not
defined under predation: total mortality carries M2, which scales with
predator abundance, so per-recruit spawning output is not a property of the
prey stock alone. The model has always declined to compute it under
multispecies mode -- but the code that consumes it was never gated to match,
so SPR0 = 0 reached it. With srr_fun >= 2 that is a division by zero:
R0 and R_init came back -Inf and the objective NaN. With the Ianelli
configuration (srr_fun mean, srr_pred_fun Beverton-Holt or Ricker) it was
worse, because the fit ran to completion and reported a finite objective
with steepness silently 0 and R_hat -Inf.
Nothing that produced a usable fit is refused. Use
build_srr(srr_fun = "mean", srr_pred_fun = "mean") under predation, or fit
the stock-recruit curve in single-species mode.
Three fleet_control columns are now validated: Selectivity_dimension,
Sel_shape_dir and Sel_shape_mode. A typo in any of them previously
resolved to NA rather than erroring -- Selectivity_dimension became a
missing selectivity dimension, the two Sel_shape_* columns a missing penalty
mode -- and nothing downstream re-checked it, so the model fitted and reported
a number on an input nobody had accepted. A workbook carrying an invalid value
will now refuse to load.
Each column is checked against exactly what its consumer implements, so no
working spelling is refused: Sel_shape_mode accepts "Smooth", "smooth"
and 1; Sel_shape_dir accepts "Increasing", "increasing" and "-1"
(the ADMB sign convention). Selectivity_dimension accepts only "Age" and
"Length", the only values rearrange_data() matches.
A blank Selectivity_dimension cell takes the schema default ("Age") rather
than erroring, so the partial-assignment idiom keeps working, and on a model
that estimates growth the fill names the fleets it applied to. This can move
a fit on a model that carried blanks: those fleets previously reached the
model as a missing dimension, which sizes selectivity by nlengths rather
than nages and so changes a max-normalized curve. No bundled dataset or
golden model carries the column at all.
Pre-flighted over every workbook in the ecosystem: 196 carry a
fleet_control sheet and not one value the new check rejects.
write_data() writes fleet_control in schema column order, as the
control and bioenergetics sheets already did. Values are unchanged and
round-tripping is identical; only the column order in the workbook moves.
The schema order was also tidied so the workbook reads sensibly: Month now
sits with the fleet identity columns rather than behind 26 selectivity
columns, and Index_distribution heads the index-observation block with the
standard-error settings it governs.
Catchability = "AR1" (the QAR1 form of Rogers et al. 2024) is now an
error. It never worked. build_map() gates the log-q deviates on
Time_varying_q %in% c("IID", "AR1", "RandomWalk"), but under this form
Time_varying_q holds an env_data column index rather than a mode -- so
the deviates were never estimated and q came back constant. The fit ran and
reported a time-invariant catchability where a time-varying one was asked
for, silently.
It errors rather than warns because a warned fit still returns a summary()
that looks ordinary, and nothing downstream can tell that its q is constant or
its objective inflated. On BS2017SS fleet 7 the Catchability deviates
likelihood row accumulates 54.8 from deviates that are identically zero,
so the reported objective is not comparable with any other model's, and
index_q_dev_log_sd is left free with nothing opposing it, driving its sigma
to zero.
The Rogers form is available as a linkage. observe is what makes it QAR1
rather than a free AR1 on q — it names the environmental series the deviates
are observed against — and obs_sd is that series' measurement SD, which the
legacy switch never asked for and must now be supplied:
build_catchability(linkages = list(q = linkage_spec(
~ ar1(1 | Year), by = ~ fleet, fleet = <Fleet_code>,
observe = "<that fleet's env_data column>", obs_sd = <its measurement SD>)))
and pass it to fit_mod(qFun = ). GOA pollock/2025/04-fit-and-diagnostics.R
in ../Rceattle-models is a worked example: it runs exactly this form.
The schema marks the code removed too, so ?BS2017SS and the workbook meta
sheet say so before a model is built rather than only at fit time.
Note this is not Time_varying_q = "AR1", a different switch sharing the
same string, which puts an AR1 structure on an ordinary "Estimated" q and
works correctly.
Estimate_catch_sd = "Analytical" now works. The option was documented in
the schema, accepted by validate_switches(), and treated as a real mode by
build_map() (which maps catch_log_sd out) -- but the model's dispatch
had only cases 0 and 1, so a model using it passed every R-side check and then
died inside the fit with Invalid 'Estimate_sigma_catch'. It is now
implemented, mirroring Estimate_index_sd = "Analytical", and the fitted
value is reported per fleet as catch_analytical_sd.
Note this concentrates sigma out of the catch likelihood, so the catch data no longer pins F as tightly as a fixed sd does, and the objective is not comparable to a fixed-sd fit. A model that does not ask for it is unaffected.
Both analytical observation sds now account for the lognormal bias
adjustment, so Estimate_index_sd = "Analytical" changes fits that use it.
Rceattle fits an index or catch series to a mean-unbiased prediction,
log(obs) ~ N(log(pred) - b*sigma^2/2, sigma) with b = bias_adjust_obs
(default 1). The Ludwig and Walters (1994) form
sigma = sqrt(mean((log(obs) - log(pred))^2)) is the sd that minimises that
density only when b = 0, so on a default fit the option did not return the
concentrated estimate it advertised. The sd used is now
sigma^2 = 2*S / (sqrt(1 + b^2*S) + 1), S = mean((log(obs) - log(pred))^2)
which reduces to the Ludwig-Walters form at b = 0. On the three BS2017SS
fisheries the old expression sat 4-17% high and left up to 1.6 units of
negative log-likelihood on the table. No bundled dataset, reference model or
live assessment sets either switch to "Analytical", so no fit anyone runs
today moves; a model that does use it will. The one workbook in the ecosystem
that sets it (Pacific hake/Data/Bridging/hake_bridging6_2022.xlsx, read only
by a deprecated script) returned a NaN objective before this change too --
its second index fleet has a zero predicted index in every hindcast year.
A fishery asking for an analytical catch sd with nothing to estimate it
from is now refused by data_check(), as is the index equivalent. With no
fitted positive observation the sd is undefined, and it used to fall through
as 0 -- which the likelihood never reads, but the reported index_sd /
catch_sd and the sim_mod() draw both do, and rnorm(mean, 0) is a
deterministic observation. A zero-catch year is legal input, so this is
reachable on the catch side; on the index side data_check() already refused
a non-positive observation, and the matching guard added to the index
estimator is parity rather than a live fix.
write_data() no longer fails on a workbook that predates a control or
bioenergetics switch. Both sheets were assembled from the full schema
object list -- the control sheet by rbind(), which drops a NULL, and the
bioenergetics sheet by hard-coded row index into a fixed-height matrix. An
absent object therefore aborted the whole write with arguments imply differing number of rows or number of items to replace is not a multiple of replacement length. The live Pacific hake workbook hits this: it predates
alpha_wt_len / beta_wt_len, so read_data() then write_data() could not
round-trip it. Both sheets now write the objects the data_list actually
carries, in schema order, as fleet_control already did.
Absent objects are dropped rather than written at their schema default: the
default belongs to the model, and baking one into a workbook would turn a
value switch_check() announces at fit time into one the file asserts.
Keying the bioenergetics rows by name also retires the duplicate row-order
list that had to be kept in sync with the schema by hand.
switch_check() accepted the selectivity spelling "Non-parametric" while
validate_switches() rejected it, so a model written that way loaded, was
normalised to nothing, and then failed its own data check. It is now upgraded
to "NonParametric" on the way in.
write_template() writes every column the schema defines. The
hand-written list had fallen 14 columns behind, and a column missing from the
template is how a user never learns an option exists. The penalty weights
Sel_curve_pen1/2/3 are deliberately left blank rather than seeded with their
schema default of 0: switch_check() converts Sel_shape_sd /
Sel_curvature_sd / Sel_devmag_sd into a weight only where the raw weight
is still blank, so a seeded 0 would silently disable that interface and leave
a non-parametric fit with no shape penalty at all.
The model-level switches have one table, .rce_model_switch_schema(), and
the comments save_config() writes are projected from it. estimateMode's
DebugOptimize was a real mode the comments never mentioned, and msmMode
was described in prose naming none of its canonical values. Each comment now
gives the integer code beside the readable name
(SingleSpecies (0) / MSVPA (1) / TypeIIIMSVPA (2)), since a saved config
writes the switch as the number.
quantities$NbyageSPR now carries dimension names. Its first dimension is
a reference point rather than a species -- c("F0", "Flimit", "Ftarget", "Finit") -- so reading it no longer means knowing that slot 3 is Finit.
Species and age bins are labelled to match the other numbers-at-age arrays.
Positional indexing is unchanged.The vignettes can now be executed. They still render without running their
code by default -- several chunks fit real models -- but
RCEATTLE_EVAL_VIGNETTES=true turns evaluation on, and a new weekly
vignettes.yaml CI job runs them for real. Nothing executed them before, so an
API break in a vignette was invisible until a user copied the code. That is how
hcrs-and-mses.Rmd kept calling knitr::kable() on mse_summary() for several
releases after it started returning a list; that code is fixed, and the
per-entity structure ($species, $fleet, $total, $meta) is now explained.
The switch tables document every value the model accepts. Selectivity
stopped at 7, omitting DoubleNormal (8), NonParametricPM (9) and
LogisticPM (11); Catchability stopped at 6, omitting AnalyticalArith (7).
PowerEquation is now marked as not implemented, and the 5.8.1 deprecation of
"Environmental" (deprecated in 4.9.0) is recorded with the linkage that replaces it.
Index_distribution is documented. It had no vignette coverage at all,
despite deciding whether an index is scored on the log or the natural scale --
and despite a second, hand-synced registry (.index_rows_natural_scale()) that
a new natural-scale family must also be added to, or it silently gets the
log-scale residual formula.
reweight_comps(fleets =) is documented, as is the log scale Comp_weights
takes under a Dirichlet-multinomial. fit_mod()'s initMode moves from an
849-character @param into an Initial age structure section, and
suitMode / avgnMode now say "declared but not implemented" in one voice
rather than three. Examples added to build_srr(), build_hcr(),
mse_summary(), sim_mod(), model_average(), osa_residuals() and
run_mse().
The vignettes and build_data()'s help no longer tell users to call
data_check(), which is internal; they point at build_data(.check = TRUE).
Removed flt_sel_ind. rearrange_data() computed it from Fleet_code on
every fit and nothing read it -- it was declared in no DATA_ object and
referenced nowhere in the package or the assessment repos.
The C++ dispatch branches are pinned to the R maps that select them
(test-schema-cpp-dispatch.R). The pinned exemptions are a machine-checked
inventory of what is implemented in the model but unreachable from R
(non-parametric growth; msmMode 3-9) and what R can encode but the dispatch
never sees. The schema's tmb_target field, previously set on no rows, now
carries the rename for 27 columns and is checked against both ends of it.
R/data.R's field dictionary is pinned against the switch maps: a documented
code the map does not define, or a map value the dictionary omits, now fails.
It also may no longer present a deprecated alias as a canonical column.
.rce_config_schema() reads estimateMode_map and msmMode_map instead of
hardcoding them, so the comments save_config() writes list every valid value.
estimateMode was missing DebugOptimize.
The diagnostics all take the fitted model as object. The same argument
was previously spelled Rceattle in retrospective(), jitter(),
self_test(), sim_mod(), sample_rec(), remove_F() and
model_average(), and fit in reweight_comps(), osa_residuals() and
process_residuals() -- so which name to use depended on which diagnostic you
reached for, and Rceattle collided with the package name
(Rceattle::retrospective(Rceattle = mod)). All ten now take object, matching
convergence_diagnostics() and R's own summary()/coef()/vcov() methods.
Existing scripts keep working unchanged. The old names are still accepted
and are silent in this release; they begin warning in a future version. To find
your own call sites early, set
options(Rceattle.warn_deprecated_args = TRUE). Supplying both names for one
model is an error rather than a guess.
Positional calls are unaffected: the old name is the last formal on every one
of the ten, and the new phase and fit_control arguments below are appended
after the arguments that predate them rather than inserted among them.
profile() keeps fitted because it is an S3 method for stats::profile(),
and mse_summary(mse =) and compare_sim(operating_mod =) keep their names
because they take an MSE result and a simulation set, not a fitted model.
sim_mod(), sample_rec(), remove_F() and model_average() now say what
is wrong when they are given something that is not a fitted model. They
previously failed further in with argument is of length zero or
invalid 'type' (list) of argument.
fit_mod() reports a composition weight of 1 on a Dirichlet-multinomial
fleet. That likelihood reads Comp_weights,
CAAL_weights and Diet_comp_weights as the LOG of the starting weight, so
the value write_template() seeds (1) is a starting weight of e, and a weight
of 1 is written as 0. A model built from the model and switched to a
Dirichlet-multinomial previously started at e with nothing saying so. Only an
exact 1 is reported, so a deliberate value is left alone, and Off fleets and
fleets carrying no composition data are skipped. It fires once per fit, and
the diagnostic refits suppress it the way they already suppress
data_check(). No value and no fit changes -- this is a message.
retrospective(), jitter() and self_test() accept
fit_control = fit_control(...), matching fit_mod(). Only phase and
getsd are read, because those are what these diagnostics forward to each
refit; setting any other field is an error rather than a silent no-op.
Supplying it replaces the defaults they otherwise infer from the fitted model.
Omitting it -- the default -- leaves every existing call behaving exactly as
before.
Which fields you asked for is read from what you set -- named in the
fit_control() call, or assigned to afterwards (ctl$getsd <- TRUE) -- not
from whether the value differs from a default. So fit_control(getsd = TRUE)
and fit_control(phase = FALSE) do what they say even though TRUE and
FALSE are those fields' defaults. A field you never touch keeps the
diagnostic's own default, which is not always fit_control()'s:
retrospective() phases its peels where fit_control() does not, so
fit_control(getsd = FALSE) asks about standard errors and cannot also
flatten Mohn's rho.
?rceattle-refit-args documents the vocabulary retrospective(),
jitter() and self_test() share -- object, cores, fit_control, and
what fit_control() reaches through a refit -- the way
?rceattle-plot-args already does for the plotters. phase, getsd and
timeout stay on each function, because their defaults and what they mean for
that diagnostic differ.
retrospective() takes phase, the value it previously fixed internally.
The default is TRUE, which is what it always used: a peel restarts from the
unpeeled fit's starting values with a year removed, so without phasing the
parameters barely move, the peels sit on top of the full model, and Mohn's rho
is biased towards zero. No peel changes unless you set it.
?jitter now records that attaching Rceattle masks base::jitter(). The
function keeps its name; call base::jitter() for the base-graphics
behaviour.
retrospective(), jitter() or self_test() called under the
deprecated Rceattle = name no longer sends the fitted model to each Windows
worker twice. The PSOCK path exports the caller's whole frame, and both
argument names were bound to the same model. macOS and Linux use FORK and were
never affected. No result changes -- this is transfer time and worker memory.Rceattle and fit as the fitted-model argument of the ten diagnostics
above. Accepted silently now, warning from 5.13.0, removed in 6.0.0. (5.11.0 and
5.12.0 are both unreleased on this line, so the silent grace period has not
yet reached a user; the warning moves with it.)rearrange_dat() now names its removal version (6.0.0) rather than
deprecating open-endedly. Use rearrange_data().The timeseries, predation and selectivity plotters now share one set of
arguments, and use them. Only plot_timeseries() ever honoured line_col,
lwd, lty and alpha; the others declared them and ignored them, so
colours and line widths silently did nothing. They are now resolved in one
place, documented once in ?"rceattle-plot-args", and applied consistently.
line_col accepts colour names, hex codes, or base-graphics palette indices
(line_col = 1), and supplies the palette for whichever variable the figure
maps to colour; on plot_selectivity()'s year fan it gives the ramp anchors
instead. lwd keeps the base-graphics scale, where the default 3 is a
standard-weight line. The remaining plotters -- plot_mortality(),
plot_maturity(), plot_comp(), plot_data(), plot_stock_recruit(), the
index, catch and diet families -- still take their own arguments.
The predation plotters honour the shared arguments. plot_b_eaten(),
plot_b_eaten_prop(), plot_m_at_age(), plot_m2_at_age_prop() and
plot_ration() now use line_col, lwd, lty, minyr, maxyr and
incl_mean (and alpha, where the figure has a ribbon), all of which they
previously declared and ignored. They also accept maxyr, lty, incl_mean
and top_adj where those were missing entirely, so scripts passing them no
longer stop with unused argument. species and spnames worked before and
now additionally take names, a logical mask and "all", and validate.
line_col and lty follow the figure, not the model: colour separates
predators in plot_b_eaten_prop() and plot_m2_at_age_prop(), and line type
separates the sexes in plot_ration() and plot_m_at_age(). Each function's
help says which. Too few colours are recycled, now with a warning naming what
they coloured, and a varying lty whose key has one level warns rather than
being dropped in silence.
add_ci = TRUE says when it cannot draw an interval. None of the
predation quantities carry standard errors -- M_at_age and
B_eaten_as_prey are REPORTed but not ADREPORTed, and consumption and the
M2 proportions are products and ratios of such series -- so the argument was
silently doing nothing. It now warns once and draws no ribbon.
plot_selectivity() draws every model, on the right dimension, and uses its
arguments. It previously read only the first fit, so a list of models
silently lost all but one; model_names, line_col, lwd and species were
declared and ignored; and species defaulted to three hard-coded Bering Sea
species names.
It now takes line_col, lwd, lty (which separates the sexes), species,
spnames, minyr, maxyr and alpha. With one model colour is still the
year, so the figure is unchanged apart from line width; with several, colour
separates the models and the year fan moves to transparency, keeping the
curves superimposed for comparison. colour_by forces either.
Length-based fleets are drawn on length bins. plot_selectivity() read
sel_at_age for every fleet and labelled the axis "Age", so a fleet whose
Selectivity_dimension is "Length" showed the growth-matrix conversion of
its curve rather than the curve that was fitted. Each fleet is now drawn on its
own dimension, and a model mixing the two returns one figure per dimension (a
named list) rather than putting ages and length bins on one axis.
species and spnames mean the same thing across these plotters.
species selects -- by index, name, logical mask, or "all", in the order
given -- and spnames labels. Several plotters previously read species as
display labels; a character vector giving one label per species is still read
that way, with a message. plot_timeseries() and its wrappers
(plot_biomass(), plot_ssb(), plot_recruitment(), the depletions,
plot_exploitable_biomass(), plot_f()) gained selection by name.
data_check() warns when diet_data does not cover every age of an
empirical-suitability predator. Under suitMode = 0 suitability is read
straight out of the diet data, so an age with no diet row is switched off
rather than estimated: a predator age with no rows gets suit_other = 1 and
exerts no predation mortality, and a prey age with no rows is never eaten.
Neither raises an error or moves the likelihood, so a diet table truncated at
the wrong age silently drops part of the predation. The warning names the
species, sex and age range for each role.
Only prey-at-age-in-predator-at-age rows count toward coverage: the
aggregated diet formats feed the diet likelihood but are skipped when the
suitability array is filled, so they cannot close a gap. Parametric
suitability (suitMode > 0) and single-species models are unaffected. A
species with no prey rows at any age is not reported — nothing eating it is a
modelling choice, not a truncated table.
Three predation plotters drew quantities that did not match their axis labels. All are corrected, so figures regenerated from them will differ from earlier runs of the same model.
plot_m2_at_age_prop() draws a share, not a contribution. M2_prop holds
each predator's contribution to M2, which sums over predators to M2_at_age,
so the plotted "proportion" reached 1564 on BS2017MS. The contributions are
now divided by their total, giving shares in [0, 1] that sum to 1 across
predators for each prey age and year; a prey age with no predation in a year
leaves them undefined and draws nothing. The y axis reads "Share of M2 at age
<age> by predator".
plot_ration() multiplies the ration by average numbers-at-age, not
biomass-at-age. consumption_at_age is one fish's annual ration in kg and
numbers-at-age are in thousands, so the product is mt -- the way the model
forms total consumption (avgN_at_age * ration, predation.hpp). Multiplying
by biomass instead weighted the age-sum by weight-at-age, so "million mt"
described nothing it computed. Average numbers rather than start-of-year
numbers, so the series reconciles with plot_b_eaten(); under the default
avgnMode = 0, N_at_age would overstate it by 1 / ((1 - exp(-Z)) / Z). On
a fitted BS2017MS the first year drops 38.3% for pollock, 20.1% for cod and
13.5% for arrowtooth flounder.
plot_b_eaten() is in million mt. It plotted B_eaten_as_prey in the mt
the model reports it in, while plot_b_eaten_prop() -- the same quantity
broken down by predator -- was in million mt, so the two could not be read
side by side. Both are now in million mt, the display unit the timeseries
plotters use. p$data moves by the same factor of 1e6.
The diet_data age-bound check compares each species against its own
nages. It matched them by position, so a species missing from the Pred
or Prey column shifted every later species onto the wrong age limit and
could reject a valid table. Out-of-range ages are still caught.
model_names given as a list() works again, in every plotter that labels
models. The package's own vignettes build it that way, and a list produced a
one-element list per model that the plot frame could not bind. Supplying fewer
names than models now warns instead of silently drawing two models as one
series.
A fleet with Selectivity = "Fixed" is drawn on ages whatever
Selectivity_dimension says. Empirical selectivity is read into sel_at_age
only, so such a fleet on a "Length" dimension would have been drawn as an
identically-zero curve -- and scripts commonly set Selectivity_dimension
across every fleet at once.
The projection divider is drawn at the latest hindcast year across the models
plotted, not at whichever model came last in the list. On a retrospective peel
the peels end in different years, so the divider could land mid-hindcast and
label real data as projection. Figures overlaying models with different endyr
-- including any using reference = -- will show the divider in a new place.
It is also omitted when the last hindcast year falls outside a minyr/maxyr
window, rather than stretching the axis back to it.
An invalid line_col, lwd or alpha now stops with a message naming the
argument. Previously an NA colour or width drew nothing, and a transparency
outside [0, 1] saturated the ribbon or errored inside the device, in both
cases with no explanation.
A spnames of the wrong length now stops rather than recycling, which had
labelled one species with another's name. A species string matching no
species now stops rather than silently plotting everything.
minyr and maxyr narrow the data rather than only the axis, so a panel with
a free y scale rescales to the window. Clipping the axis alone left the scale
trained on the hidden years, which on a series spanning orders of magnitude
squeezed the requested window into the bottom of the panel. They also now work
on the predation plotters, which declared them and ignored them, and
plot_timeseries(save = TRUE) writes the same window it plots.
plot_f() keys its Ftarget and Flimit reference lines to the species it
drew. It indexed Ftarget and the facet labels with the raw species
argument, which works for indices but gives an NA facet key for a name --
on the same argument that newly accepts names.
plot_depletionSSB() draws the Ptarget and Plimit lines of one model, per
species. The two were collected into a models-by-species matrix and then
subset with the species indices, which flattens column-major: on a two-model
overlay whose models carry different Ptarget, species 2's line came from
model 2's species 1. Models in one figure normally share their reference
points, and then every row of that matrix is identical, which is why it went
unseen. The values now come from the first model.
A single lty reaches the figures that map line type themselves --
plot_ration(), plot_m_at_age(), plot_b_eaten_prop(),
plot_m2_at_age_prop() and plot_selectivity(). It is applied to every level
of whatever the figure keys line type on, and warns when that key has more
than one level, since they are then drawn alike. The default lty = 1 leaves
the figure's own line types alone.
incl_mean averages each model over its own hindcast, not the first
model's. On a retrospective peel the answer otherwise depended on the order of
the list.
plot_m2_at_age_prop() orders its prey panels the way species asked for,
like the other plotters, instead of alphabetically.
plot_b_eaten(mse = TRUE) draws its projection ribbon at alpha (default
0.4) rather than a fixed 0.3, honours incl_mean, validates lwd and lty
as the other paths do, and says that line_col does not apply when the
simulations are summarized into one band.
plot_timeseries(save = TRUE) writes the years alongside the values, names
the columns after the models, names the file after the species, and writes
only the species plotted. It previously wrote unlabelled columns for every
species, with no year column.
The pkgdown site builds again. simulate.Rceattle (added in 5.9.0) was not
in _pkgdown.yml, and pkgdown stops on a documented topic missing from its
reference index. ?"rceattle-plot-args" is listed there too, so the topic the
plotter help and the vignettes point at has a page on the site.
age, minage and maxage are ages, not age-bin indices.
plot_m_at_age(age =), plot_m2_at_age_prop(age =), plot_ration(minage =)
and plot_mortality(maxage =) were passed to the at-age arrays as subscripts.
The arrays are indexed 1 to nages, while a species' ages run minage to
minage + nages - 1, so on any species with minage != 1 the figure drew a
different age from the one its axis named -- and an age past the plus group
ran off the end of the array rather than being reported. Each is now resolved
against each species' own age vector, which is also what lets one figure hold
species with different age ranges. Only a model with minage != 1 changes:
every bundled dataset is minage = 1, where the two readings coincide.
A species that has no such age is dropped with a warning naming it, rather than drawn at a shifted age; an age no species carries is an error. Requesting an age past the plus group of some but not all species therefore now draws the species that have it instead of failing.
plot_selectivity()'s species used to be an ignored label argument whose
default was c("Walleye pollock", "Pacific cod", "Arrowtooth flounder"). It
now selects. Passing species names that only partly match the model's own
stops with a message naming both readings, rather than guessing -- which is
what a call copied from the old default does on a model whose species are
named differently. Pass labels as spnames, or the model's own names as
species. A call passing one label per species, none of which match, is still
read as labels.
plot_timeseries(save = TRUE) needs a file stem, and stops without one. It
previously wrote to a file called NULL_...csv.
plot_f() is now built by the same factory as the other timeseries plotters,
so it takes their full argument list -- it gains lty, save, reference,
legend.pos and ylab, none of which it accepted before. plot_timeseries()
gains two internal arguments, ref_lines and suffix, which the factory uses
to attach the F and depletion reference points; the reference-point layers are
consequently later in p$layers on the depletion plots than they were. The
rendered figures are unchanged.
plot_selectivity()'s p$data names the x variable Bin, not Age, and
carries a Dimension column ("Age" or "Length"); the column holds an age
or a length-bin ordinal depending on the fleet, so it is no longer named for
one of them. It also gains a Model column, now that every model is drawn.
plot_b_eaten()'s value column is in million mt (see above).
ggplot2 now requires >= 3.5.0.Every observation type is now simulated by the TMB model, in a SIMULATE
block beside the density that scores it — survey index (under each fleet's
own Index_distribution, including the correlated MVN/MVNORM draw), total
catch, age/length compositions, conditional age-at-length, and, for the first
time, stomach contents. sim_mod() no longer re-implements any observation
model in R, so the draw and the density cannot be changed on one side only.
The two copies had diverged: every survey was drawn as independent lognormal
whatever its Index_distribution, and diet was not drawn at all, so a
multispecies self_test() recovered suitability from stomachs that never
varied.
Compositions and CAAL are drawn in raw bin space, before tail accumulation and
before comp_offset. Both families are closed under merging categories, so
drawing raw and letting the refit fold again is exact.
Comp_weights / CAAL_weights / Diet_comp_weights now enter the draw as
they enter the density: as an effective sample size for the multinomial
families (N * weight), and through the concentration for the
Dirichlet-multinomial. The old R draw used the nominal Sample_size
regardless of the weight, so a self_test() on a re-weighted model was
handed data more informative than the estimator treats them as being. Any
model with a composition weight other than 1 gives a different self-test.
A row whose sample size times its weight rounds below one observation comes
back empty, with Sample_size dropped to zero. Rows the model cannot draw —
a predator under empirical suitability, a covariance fleet outside its fitted
window — keep their observed values. sim_mod() warns for each case.
Simulated quantities are reported under names ending _sim (catch_obs_sim,
index_obs_sim, comp_obs_sim, caal_obs_sim, diet_obs_sim), since TMB
never clears its report environment.
This changes results. A seeded self_test() or
run_mse(simulate_data = TRUE) will differ from earlier releases: the draws
moved into obj$simulate() and the random-number stream moved with them.
Simulating is also slower, since it evaluates the compiled model rather than
reading $quantities.
sim_mod() can redraw process error as well as observations, via the new
process argument. Observations are always drawn; process error is a
choice, because redrawing it changes what self_test() measures — from
recovering parameters to recovering a process. Pass "recruitment", "M",
"growth", "catchability" or "selectivity", the groupings "dynamics" or
"observation", or TRUE for all. The default FALSE keeps the previous
behaviour.
Recruitment covers the initial age structure, since the initial
numbers-at-age are recruitment from before styr. Natural mortality covers
all six M1_re modes. Growth, catchability and selectivity are drawn wherever
they are written as a random linkage.
Three cases are not drawn, each with a warning: the equilibrium
initial-condition modes, which fix init_dev; observed AR1 (Rogers QAR1)
linkages, whose latent state is measured by a covariate series; and the legacy
Time_varying_q / Time_varying_sel deviations and the AR1 selectivity
forms, which carry densities but no draw yet.
When a process is redrawn, the deviations that generated the data come back as
attr(x, "process_sim"). Compare estimates against those, not against the
source model's fitted deviations, which would report bias by construction.
Each deviation arrives with a same-shaped _drawn logical marking the cells
the draw touched. compare_sim() compares against the operating model and is
therefore not valid for process-drawn replicates.
Index_distribution = "TruncatedNormal" fits and simulates a
natural-scale normal left-truncated at zero, so the density is renormalized as
log f(x) = log phi(x; mu, sd) - log Phi(mu/sd) and the draw is taken by
inverse CDF on (0, Inf). It is the only natural-scale family whose simulator
and likelihood are the same distribution: "Normal" and the covariance
families have to redraw the non-positive draws data_check() would refuse,
which samples from a truncated normal while the likelihood scores an
untruncated one. Prefer it unless an exact ADMB comparison is needed —
"Normal" is unchanged and still reproduces AMAK avo_like / cpue_like
term for term.
TruncatedNormal is registered as a natural-scale family in the
diagnostics, so residuals(type = "pearson"), plot_index()'s observation
interval and plot_indexresidual() give it the natural-scale treatment. A
test enumerates index_distribution_map, so a future family that misses the
predicate fails in the suite rather than in a residual plot.
simulate() works on a fitted model, as the stats generic.
simulate(fit, nsim = 10) returns a list of nsim data sets — a list at
nsim = 1 too, so callers never special-case the length. seed follows the
stats::simulate() convention. Use sim_mod(simulate = FALSE) for expected
values rather than draws.
self_test(process = ) passes through to sim_mod(), so a self-test can
ask whether the estimator recovers a process it was not shown. Each
replicate's true deviations are returned as attr(result, "process_sim")[[k]].
sim_mod() warns when a simulated observation is one the model cannot be
refit on. A non-finite or negative draw would otherwise be rejected by
data_check() and counted by self_test() as a convergence problem rather
than a data one. The usual cause is a fleet whose observation standard
deviation never got a value.
quantities$log_index_sd and log_catch_sd are renamed index_sd and
catch_sd. Neither was ever a log: both hold the observation standard
deviation the likelihood used for that row. Its scale depends on the family —
log-scale for Lognormal, an absolute sd in the units of the index for a
natural-scale Index_distribution.
The old spellings are still returned this release and will be removed in the
next. Fits saved before the rename keep working. The TMB parameter
index_log_sd is a different object and is unchanged.
A fishery carrying index_data now gets an estimable catchability. The
model has always fitted such a row, but build_map_catchability() entered its
block only for Fleet_type == "Survey", so a fishery's index_log_q,
index_q_dev and index_log_sd stayed mapped out and
Catchability = "Estimated" did nothing.
In reverse, a survey with no index rows no longer gets an estimated catchability — a q with no index to inform it is a flat direction in the likelihood. Sharing takes precedence and is unchanged.
No bundled dataset is affected, so /golden-check is silent. To keep the old
behaviour set Catchability = "Fixed", or give the fleet its own
Catchability_index to estimate it separately. print() marks the case:
q: Estimated (fixed: no index data).
A fishery's index is predicted from the year-averaged numbers. A survey
index remains a snapshot at its observation month, N exp(-(Month/12) Z). A
fishery index is CPUE, which integrates over the year alongside the catch:
C/E = q * sum_a sel_a * Nbar_a * w_a, Nbar_a = N_a (1 - exp(-Z_a)) / Z_a
the same mean-numbers term the catch equation and the fishery's age
composition already use, so a fleet's index, catch and comps are now
consistent. Month is not read for a fishery's index rows.
This affects trend as well as scale, so catchability cannot absorb it; the
size over a range of Z is tabulated in
vignette("model-options-and-functionality"). Stock Synthesis splits the same
way on its per-fleet survey timing.
Only a fishery carrying index_data is affected, so /golden-check is
bit-identical. For a snapshot-timed index on a fishery's selectivity — the
AMAK/ebswp form the EBS pollock ADMB bridge needs — put the index on its own
Survey fleet and point that fleet's Selectivity_index at the fishery.
sim_mod(simulate = TRUE) no longer works on an averaged model.
Simulating evaluates the compiled model, so it needs one. A fit whose $obj
was dropped is rebuilt from its data_list and estimates. model_average()
output cannot be rebuilt — its quantities are an average over models, so
there is no parameter vector to draw around. Simulate from one of the
underlying fits. sim_mod(simulate = FALSE) is unaffected.
sim_mod() errors instead of recycling when catch_data no longer lines up
with the fitted model. The write-back is a row-position copy, and the old
draw passed mismatched vectors to rnorm(), which recycled the shorter one
and returned a full-length, wrong answer. The error names both row counts.
The growth_re switch, growth_indices, and the log_growth_par_devs
parameter array are removed. Nothing consumed growth_re: the deviation
array was mapped off in every configuration and given no density, and the
parameters its documentation claimed to map were never declared. Setting it
changed no fit, and removing it leaves every fit bit-identical. A data_list
still carrying either name gets a deprecation message. Time-varying growth
goes through build_growth(linkages = ). fit_mod() drops retired parameter
blocks from inits, so warm starts from older fits keep working.
data_check() requires the catchability columns each switch actually
reads — Index_sd under Estimate_index_sd = "Estimated",
Catchability_prior_sd under Estimated-with-prior and AR1,
Time_varying_q_sd under a penalized Time_varying_q, and
Catchability_init for every form that reads it. All are taken as log()
when the parameter list is built, so a blank or non-positive entry gave a
non-finite objective reported by TMB, naming neither the fleet nor the column.
Each condition matches build_map()'s own gate, so settings that read no
starting value are not asked for one.
data_check() requires the catchability settings wherever an index is
fitted. Catchability, Catchability_init and Estimate_index_sd have no
schema default, so on a fishery they arrived NA and the fit died inside
build_map(). The error now names the fleet and the column. Those
configurations did not run before either.
Two limits worth knowing. An index needing a selectivity different from
the fishery's still has to be its own fleet: sel_at_age is indexed by fleet.
The same is true of flt_units, so a CPUE series in numbers cannot sit on a
fleet whose catch is in weight.
A missing Ceq is reported instead of crashing data_check().
data_check() reports fleets that share a Selectivity_index but disagree
on the columns that shape the curve, which are read per fleet and so do not
give one shared curve despite the shared parameter block. A blank counts as a
value: an empty Sel_norm_bin means "do not normalize".
data_check() reports a shared Catchability_index that does not share a
catchability. Analytical and AnalyticalArith solve q from each fleet's
own index observations, bypassing the group parameter, so it is reported on
the form rather than only on a disagreement.
A catchability linkage that names only part of a shared Catchability_index
group is reported. The offset is added per fleet and not reconciled across
the group, so the fleets end up with different catchabilities while
fleet_control still says they share one.
A catchability linkage on a fleet whose Catchability is "Environmental"
or "AR1" is now refused. Those forms overwrite index_q, so the linkage
offset never reached the likelihood and its parameters were free with an
identically-zero gradient. Express the whole relationship as the linkage and
set Catchability = "Estimated".
The diagnostic refits no longer repeat data_check()'s warnings, which
describe the data_list and would otherwise appear per peel, jitter or MSE
iteration. fit_mod() gains quiet_data_check (default FALSE), set by
every refit. Errors still stop the fit.
A fishery's index now appears in the index diagnostics. plot_index()
selected Fleet_type == "Survey"; both it and plot_indexresidual() now
select the fleets carrying index_data. plot_indexresidual() previously
applied no fleet filter at all, so it also drew residuals for Off fleets —
GOA2018SS fleet 7 has nine such rows.
Diagnostics for a natural-scale survey index no longer use log-scale
formulas. Lognormal carries a log-sd; MVN, MVNORM, Normal and
TruncatedNormal carry an ABSOLUTE sd. Applying the lognormal form to those
does not error, it returns nonsense, because sigma^2 / 2 is a number the
size of the index squared.
residuals(type = "pearson", source = "index") returned the same large
constant for every row of a natural-scale fleet (about +75 for an absolute
sd of 150, whatever the fit). It now standardizes as (obs - hat) / sigma.plot_index() drew its observation interval with qlnorm(), giving bands
like [0, 1e130]. Natural-scale fleets now get obs +/- 1.96 * sigma,
clamped at zero.plot_indexresidual() now uses the plain difference where the fleet is
fitted on the natural scale, labelled "Index residual".plot_indexresidual() plotted the negative of a residual. It drew
predicted - observed while residuals() returns observed - predicted.
Both are now observed - predicted, the usual assessment convention (WHAM,
SS3). Index residual plots made before 5.9.0 are mirrored about zero; a
figure carried forward needs redrawing and any written interpretation of its
sign needs rereading. Lognormal fleets are unchanged.
Estimate_index_sd = "Analytical" no longer passes silently on a
natural-scale index family. The analytical sd (Ludwig and Walters 1994) is
accumulated from squared log residuals. Normal and TruncatedNormal read
it as an absolute value, so the likelihood was evaluated on the wrong scale —
data_check() now refuses that combination. MVN and MVNORM score through
index_cov and never read the scalar sd, so their fits are unaffected, but
the diagnostics divide by it, so those warn rather than refuse. Lognormal is
untouched.
TruncatedNormal OSA residuals were the untruncated family's. The
truncation constant is a function of the prediction, not the observation, so
it drops out of any method reading the curvature of the density in the
observation — including oneStepPredict()'s default. osa_residuals() now
residualizes this family in its own call with method = "oneStepGeneric" and
a range starting at zero. The correction is the size of the truncated mass: on
a fleet predicting 100 with an absolute sd of 150, an observation exactly at
the prediction moves from 0 to -0.44. "Normal" is untouched; both are
pinned against their closed forms in
tests/testthat/test-likelihood-index-truncated-normal.R.
Three consequences, all in ?osa_residuals: exact integration does not always
converge on a random-effects model, in which case the fleet falls back to
TMB's spline approximation with a warning; sd is NA for the group and
predicted is the truncated conditional mean; and adding a truncated fleet
shifts the other fleets' residuals on a random-effects model — a different
conditioning sequence, not a wrong value. attr(x, "method") records what ran.
compare_sim() warns when it is handed process-drawn replicates, since
every statistic it reports is a deviation from operating_mod, which is the
truth only when the replicates redrew the observations alone.
osa_residuals() reported observed = NA for any group residualized with
oneStepGeneric. Affected the discrete composition path as well as the new
truncated one.
M1_re = 3 and 6 estimated far fewer deviations than intended. The
age-by-year map index was built from a vector of length nyrs_hind and
assigned into an nages x nyrs_hind block, which R recycled silently — on
GOA2018SS, 42 free deviations where 882 were meant. Fits in these two
modes change; the other M1_re modes recycle deliberately and are unaffected.
These deviations are integrated out rather than estimated as fixed effects, so
the model class is sound, but a free age-by-year mortality field with a single
standard deviation and no prior on M is strongly confounded with selectivity
and recruitment in a single-species assessment, and the latent dimension is now
over twenty times larger. Read fit$convergence and the estimability table
before trusting such a fit, and consider a prior on M.
The 2D-AR1 correlations acted on the wrong dimensions. SEPARABLE(f, g)
applies f to the outermost array dimension and g to the fastest-running
one, and both 2D-AR1 densities pass their fields as (bin, year) or
(age, year), so the two correlations were exchanged. Affected M1_re = 6
and Selectivity = "2DAR1". The 3D-AR1 form was already bin/year/cohort.
Most fits do not move. Both correlations start at 0 and TMBphase() holds
them there, so SSB, F and reference points are unchanged and only the two
reported estimates exchange names. A fit moves only where the two differ as
starting values — for selectivity, unequal Sel_curve_pen1 /
Sel_curve_pen2.
Warm starts across this release are the case to watch. inits from an
older fit carry M1_rho and sel_curve_pen under the previous convention, so
a refit starts from a mirrored point. That covers retrospective(),
jitter(), run_mse() and any two-stage bootstrap. Refit from
inits = NULL, or transpose the two slots, if the starting values differ.
A composition or CAAL row with no sample size came back holding the
predicted proportions. comp_hat rows are normalized to sum to one, so a
row with Sample_size = 0 was returned as noise-free proportions —
indistinguishable from a real full-weight composition. run_mse() reaches
this state directly, so the next assessment would have been handed a
perfectly-observed composition for a year that was never sampled. Such a row
now comes back empty.
obj$simulate() returned the drawn compositions under the observed data's
own name. comp_obs and caal_obs are now drawn into copies, as the catch,
index and diet draws already were. fit$quantities was never affected.
The natural-scale survey draw did not update obsvec, leaving the OSA
path scoring different data from the draw.
Recruitment is no longer redrawn when two densities score it. Under
srr_fun = 0 with srr_pred_fun > 0 — the AMAK/Ianelli configuration —
rec_dev is scored by both JNLL_REC_DEV and JNLL_SRR_PENALTY. Two terms
on one latent do not compose into a distribution to draw from, so
self_test(process = "recruitment") would have measured recovery against a
process the model does not assume. The deviations are left at their fitted
values and sim_mod() says why; a random linkage on a recruitment parameter
is a separate latent and is still drawn.
The correlated survey draw could write an unchecked non-positive index. The rejection loop tested its budget before testing the draw, so exhausting the budget wrote out whatever had last been drawn.
data_check() rejects a diet_data that is not sorted by stomach_id.
The diet likelihood walks diet_ctl with a single forward cursor, so the ids
have to run 0, 1, 2, ... in order and with no gaps. Out of that order whole
stomachs drop out of the likelihood with no warning and a lower objective:
re-sorting a cleaned BS2017MS diet table by predator age leaves 3 of its
45 stomachs in the fit, and reversing the rows leaves 1. clean_data()
sorts by stomach_id; this catches a hand-built or re-sorted table.
data_check() allows a stomach summing to 1 within floating-point
tolerance (> 1 + 1e-12 rather than > 1). Real excesses are still
rejected.
sim_mod() sizes the truncation warning correctly, per family. For the
independent "Normal" family the warning reports the gap as
P(draw <= 0) = Phi(-mu/sd) on the worst row. The previous test counted
rejections, but each row is drawn once per call, so the ratio could only be
0, 1/2, 2/3, ... — an indicator rather than a rate, against an unreachable 2%
threshold. A covariance fleet is judged on the JOINT rate, since it rejects
the whole vector whenever any row is non-positive: an 8-row fleet whose worst
margin is 31% is rejected about 92% of the time.
The developer guide covers the SIMULATE layer: what a new likelihood term
owes its simulator, the *_sim reporting convention, and the two registries a
new or retired parameter block has to be added to.
vignette("model-diagnostics") documents process = , the natural-scale
index families, and what a redrawn M can and cannot be read as. On BS2017SS
with a year-varying M random effect, refits recover the simulated deviations
(correlation around 0.5, positive in every replicate) but recover their
standard deviation poorly — roughly 70% low, collapsing to zero in a third of
replicates. That is an ordinary variance component estimated from little
information, not a fault in the simulation;
tools/verify/verify-sim-recovery-M.R reproduces the numbers.
vignette("model-options-and-functionality") records which Sel_curve_pen
slot is which correlation under the AR1 selectivity forms, and the
estimability caveat on M1_re = 3 / 6. vignette("growth-estimation")
gains a time-varying growth section. vignette("hcrs-and-mses") notes that
seeded MSE results are not comparable across 5.8.x to 5.9.0.
Documentation-only release. Four changes that landed earlier in the 5.x line
reached main without a NEWS entry; they are recorded here so the release
notes describe what actually shipped. No code changes.
The Dirichlet-multinomial composition likelihood uses a different effective
sample size. The concentration is now built from the observed-count total,
sum(comp_obs) = N * (1 + offset * nbins), rather than the raw input N.
That is the same N ddirmultinom() reads back from obs.sum(), so the
alpha and the density are now consistent, and it matches the CAAL
Dirichlet-multinomial and the AFSC linear-DM parameterization. Any fleet
with Comp_distribution = "DirichletMultinomial" will move; multinomial
fleets are unaffected. Refit before carrying a DM-weighted model forward.
build_growth(sd_plus_group = ) defaults to NA ("inherit") rather than
"WHAM". NA takes data_list$growth_sd_style when the data list carries
one, falling back to "WHAM" otherwise, so a fresh fit is unchanged. The
fix is for refits: a diagnostic that rebuilt growth through
build_growth(fun = ) previously reset an SS3-style plus-group SD back to
WHAM without saying so. Retrospectives, jitters and self-tests of an SS3
growth model change accordingly.
Catchability = "Environmental" is deprecated. It still fits with its
existing numerics and results are unchanged, but it names covariates by
position in Time_varying_q rather than by formula, and cannot carry priors,
bounds or an estimation phase. Use a catchability linkage instead:
build_catchability(linkages = list(q = linkage_spec(~ temp, by = ~ fleet))).
switch_check() now names the affected fleets.dmultinom_osa(), which
renormalizes the predicted proportions; the previous dmultinom() did not.
The reported per-row NLL therefore shifts by an additive constant,
Neff * log(1 + n_comp * offset). The gradient, the MLE and every derived
quantity are unchanged -- only the printed likelihood value moves. Compare
likelihoods within a version, not across this boundary.fleet_control$Sel_norm_scope controls whether selectivity normalization
pools its reference across sexes. Normalization makes two independent
decisions -- where the reference is taken and whose scale it sets -- and
until now the second was a silent passenger on the first: a named
Sel_norm_bin always used a per-sex reference, and max-normalization always
pooled across sexes, so two of the four combinations were unreachable. The new
column is orthogonal to Sel_norm_bin, so the two combine rather than
multiplying into more columns:
"AcrossSexes" (default) -- one reference pooled over the sexes, so the
less-selected sex stays below 1 and relative sex-specific selectivity is
retained. This is what a dimorphic stock usually wants."WithinSex" -- each sex divided by its own reference, so both reach 1 and
only the shape differs by sex.The newly reachable combination worth knowing about is max-normalization with
"WithinSex": each sex is scaled at its own plateau, wherever that falls. For
a stock whose sexes plateau at different ages that is more robust than naming
a bin, because a named bin stops being the plateau once selectivity is
time-varying while the maximum tracks it.
The scope has no effect on a one-sex species, or where Sel_norm_bin is NA
and nothing is normalized. The one behaviour change is a two-sex fleet
normalizing at a named bin: it previously used a per-sex reference and now
pools, and switch_check() emits a message naming the fix (set
Sel_norm_scope = "WithinSex") when it sees that configuration. Max-normalized
fleets -- including the operational arrowtooth configuration -- already pooled
and are unaffected. No bundled dataset is in the affected case.
Biomass, SSB and recruitment confidence intervals are now taken on the log
scale. The model ADREPORTs log_biomass and log_R alongside the existing
log_ssb, and plot_timeseries() builds exp(log(x) +/- 1.92 * sd_log) from
the delta-method SD of log(x) whenever a model reports one. sdreport()
linearizes once about the MLE, so its SD is exact only for a linear function
of the parameters; these three series are built multiplicatively
(R = R0 * exp(rec_dev), and n-at-age is a product of survivals), so log(x)
is close to linear in the estimated parameters where x itself is
exponential. The interval is therefore both better approximated and
right-skewed, and cannot cross zero the way the symmetric natural-scale
interval did for weak year classes and depleted stocks. The log-scale SD is
also the CV, the form the ABC / OFL buffer calculations want. Natural-scale
biomass, ssb and R are still ADREPORTed, so existing callers of
sdrep$value are unaffected. Models fit before this release have no log_*
rows, but still get the log-scale interval -- see below.
The plot_*() timeseries wrappers take a ylab argument. It is appended
to the argument list, not inserted, so positional calls are unaffected. Left
NULL (the default) the axis label is derived from the series and the model's
minage, so a model whose recruitment is at age 3 no longer gets an axis
labelled "Age-1 recruits". Only recruitment names an age now -- it is an age
class, so the age is information. The biomass and SSB axes drop their Age-1+
prefix and read Biomass (million mt) and SSB (million mt); the prefix was
noise on an aggregate, and wrong on SSB, which is mature females rather than
the minage+ stock.
plot_exploitable_biomass() and the two depletion plotters accept
add_ci = TRUE, on the log scale. exploitable_biomass, ssb_depletion
and biomass_depletion are now ADREPORTed, so an interval can be computed at
all. They are ADREPORTed on the natural scale only, deliberately:
exploitable_biomass sums over fisheries alone, so it is exactly 0 for any
model without projection F (Proj_F_proportion = 0, which is every bundled
dataset -- all 216 species-years of BS2017SS), and the depletions divide by
B0 / SB0; log() of either would put -Inf on the AD tape and turn the
entire sdreport, not merely that row, into NaN. They still get a
log-scale interval, because the delta method defines sd(log x) = sd(x) / x
exactly -- the plotters recover the log-scale SD from the natural-scale one
wherever the series is positive. Where it is 0 the quotient is undefined and
the (degenerate) symmetric interval stands. sdreport grows about 20% on
BS2017SS for the three extra series.
Models fit before this release get log-scale intervals too. The same
sd(log x) = sd(x) / x recovery applies to any strictly positive series with
a natural-scale SD, so an existing fit with no log_* rows plots a
right-skewed interval without being refit. Where the model does report
log_biomass / log_ssb / log_R, those are used directly; the two agree to
machine precision (2e-16 on BS2017SS).
as.data.frame(fit) and the plotters now report the same interval. The
extractor still built a symmetric natural-scale interval for biomass, ssb
and R while plot_biomass() drew the log-scale one, and it returned NA
standard errors for exploitable_biomass and the two depletions -- the three
series this release just ADREPORTed. Both paths now go through one helper.
The plotters also switch from the rounded 1.92 to qnorm(0.975), so the
band labelled 95% is a 95% band: ribbons are about 2% wider than in 5.7.0,
and any interval read off a figure or table should be regenerated. The point
estimates are unchanged.
add_ci = TRUE on a fit made with getsd = FALSE no longer errors. It
warned that it would draw no interval and then failed on the next line
indexing an absent sdreport. It now warns and draws the series without a
ribbon, as the message says. The warning is also gated on whether the series
is ADREPORTed at all, so plot_f(add_ci = TRUE) no longer warns on every fit
about standard errors F_spp can never have.
rearrange_data() and data_check() work again on a fleet_control that
has not been through switch_check(). Sel_norm_scope was read with no
missing-column guard, so a data list read straight from a workbook failed with
a cryptic "column not found". It and Index_distribution are now filled from
the column schema. A blank Sel_norm_scope cell is filled too -- an NA
reached TMB as NA_integer_, which the C++ reads as WithinSex, the opposite
of the default.
The Sel_norm_scope behaviour-change notice fires only where it applies.
It was skippable (leaving one cell blank flipped that fleet silently) and it
fired on Hake and LogisticPM fleets, which use Sel_norm_bin as a penalty
age-range rather than a normalization reference, and on Off fleets. The gate
now matches the one in selectivity.hpp.
A one-species plot drops the redundant species strip, and absolute y-scales start at zero so panel heights are comparable. Depletion and other relative series are unaffected.
Diet defaults are announced only when the model reads them, so a single-species fit no longer reports settings it never uses.
model_config() names the mistake when handed a data_list. It takes
model settings, not data, but almost every other entry point takes a
data_list first -- so model_config(my_data) bound the list to msmMode and
reported only "msmMode must be a single value". It now points at
build_data(base = , model_config = ). The check itself was correct:
msmMode is a model-wide scalar (unlike the per-species suitMode), and a
single-species msmMode > 0 cannibalism configuration was never blocked.
A confidence interval that cannot be drawn now warns instead of vanishing.
Requesting add_ci = TRUE for a series with no standard errors indexed out of
range, filled the interval with NA and rendered an invisible ribbon -- no
error, no warning, just a missing interval. That is how the three plotters
above hid their missing ADREPORT. plot_timeseries() now says which series
and which model lacks standard errors.
Recruitment was plotted 1000x too high, and the stock-recruit panel 1000x
too low. The model carries numbers-at-age in thousands and weight-at-age in
kg, so biomass comes out in mt and recruitment in thousands of fish.
plot_recruitment() never applied the matching /1e3, plotting thousands of
recruits under an axis reading "Age-1 recruits (million)", while
plot_stock_recruit() divided by 1e6 under an axis reading "Recruitment
(millions)" -- so the two panels disagreed with each other by a factor of a
million. Both now plot millions of recruits. plot_exploitable_biomass() was
labelled "million mt" with no rescaling applied at all and is now divided by
1e6 like the other biomass series. Divisors and axis units are now held in
one table (.RCE_TS_RESCALE / .rce_ts_ylab()) that plot_stock_recruit()
reads too, and are asserted against each other in the test suite. Any figure
or number read off these three plotters needs regenerating. Model results
are unchanged -- this is display-only.
How sex-specific selectivity works is now documented. Every selectivity
form is sex-specific by default in a two-sex model. Sel_norm_bin says only
where the normalization reference is taken; whether the relative scale
between sexes survives is Sel_norm_scope (above). Relative sex selectivity
is only informed where the composition is joint (comp_data$Sex = 3);
sex-specific rows each sum to 1 and carry no sex-ratio information, and
Selectivity = "Hake" always normalizes within sex regardless of either
column. New "Sex structure and relative selectivity" section in
vignette("model-options-and-functionality"), with the details on the
Sel_norm_bin and Sel_norm_scope field dictionary entries.
The mt / thousands-of-fish convention is now stated consistently. The
biomass / ssb / catch_hat declarations in ceattle.cpp, the
index_data / catch_data / Observation_units field dictionary entries and
the Stock Synthesis catch-conversion table all said kg. Section 12 of
vignette("model-options-and-functionality") now covers the display units,
ylab and the log-scale intervals, and names the discrete palette correctly
(Okabe-Ito, not viridis).
Sel_norm_bin and Sel_norm_bin_upper now say that they are bin indices on
the fleet's own Selectivity_dimension -- an absolute age for an age-based
fleet (6 means age 6, not the sixth bin), a 1-based length-bin ordinal for a
length-based one.
fleet_control$Sex is documented as inert. It is read nowhere in the model;
sex is set per observation on comp_data$Sex. Retained only so older
workbooks still read.
vignette("model-parameterizations") no longer claims the model is fit to
sex-ratio data. There is no sex-ratio likelihood component -- the text
described the ADMB implementation. Sex-ratio information enters through joint
composition data.
plot_timeseries() documents the unit convention it assumes: numbers-at-age
in thousands and weight-at-age in kg, hence biomass in mt and recruitment in
thousands of fish.
The remaining bundled datasets use the canonical fleet_control column
names. The 4.10.0 renames (Q_prior -> Catchability_init,
Comp_loglike -> Comp_distribution, ...) had reached BS2017SS,
BS2017MS and GOA2018SS only; Atka2022, GOA2018SS, GOAatf,
GOAatf2023, GOAcod, GOApollock, GeorgesBank3spp,
NorthernRockfish2022 and whamGrowthData still shipped the legacy
spellings, so loading one printed a dozen upgrade messages. A script reading a
legacy name off one of these objects directly -- e.g.
GOApollock$fleet_control$Comp_loglike -- now gets NULL. Use the canonical
name; a workbook on disk still reads fine, since the aliases are upgraded on
read.
mse_summary()'s metric columns now have syntactic names. They were
display strings held together by check.names = FALSE, so reading one meant
summ$species[["OM: Terminal SSB Depletion (Dynamic)"]]. They are now
ordinary names -- avg_catch, catch_iav, p_closed, om_terminal_ssb,
om_avg_depletion and so on -- with an om_ prefix for the operating model's
truth and em_ for the estimation model's perception. The four
misclassification metrics become p_overfishing_false_pos / _false_neg and
p_overfished_false_pos / _false_neg, and the three collapse metrics become
om_sims_collapsed, om_no_f_sims_collapsed and om_sims_collapsed_from_f
-- named sims because they are counts of simulations, not probabilities,
which the old names did not say. The long display strings are kept on a
"labels" attribute of each frame, so plots and SAFE tables can still print
them: attr(summ$species, "labels"). Species, Fleet_code and Fleet_name
are unchanged. Values are unchanged. Done in the same release as the
per-entity reshape so mse_summary() breaks once rather than twice; no script
in Rceattle-models or GOA-ATF-ESP reads a metric by name.
data_requirements() argument selectivity is now Selectivity, and
index_distribution is now Index_distribution. They stand in for the
fleet_control columns of those names, and the rest of the package's
arguments already mirror their source exactly (estDynamics, Ceq,
growth_model, and model_config()'s mirror of fit_mod()). No deprecation
path: data_requirements() is new in this release line and has never been on
a released version.
self_test() no longer fails on every replicate for a model with a
natural-scale survey index. sim_mod() drew Normal and MVN index
observations from the untruncated natural-scale normal, so a fleet with an
absolute sd near its index -- an AVO-type index with sd 0.25-0.80 against
observations from 0.70 -- drew non-positive most replicates. data_check()
rejects those, so the refit failed and the replicate was counted not
converged, reading as a convergence problem rather than a simulation one.
Non-positive draws are now redrawn, correlated fleets jointly. On EBS pollock
self_test() goes from 0 of 50 replicates to all of them, recovering SSB with
a median bias of -0.4%.
Redrawing samples from a normal truncated at zero, not the normal the likelihood maximises, so a fleet that needs it is self-testing against a different data-generating process. At index 0.70 with sd 0.80 a fifth of draws are rejected and the truncated mean is 39% high. Two warnings now fire: the existing one when a fleet still cannot draw positive, and a new one when any row is redrawn more than 2% of the time. Either means the absolute sd is too large relative to the index for a natural-scale normal.
Lognormal fleets are unaffected, including the seeded all-lognormal fast path,
and seeded results are bit-identical for any Normal/MVN fleet that never
rejects. A fleet that does reject consumes extra random numbers, shifting that
replicate's comps and catch and every later replicate in a seeded loop.
data_check() no longer errors on a data_list that carries no HCR.
HCR is a fit_mod() argument, not a data field, so a list read straight
from a workbook has none. NULL %in% ... is logical(0); TRUE && logical(0)
is NA and x & logical(0) is logical(0), so data_check() died with
missing value where TRUE/FALSE needed and validate_switches() with
argument is of length zero. fit_mod() sets HCR before checking and was
unaffected.
run_mse() no longer errors when a survey fleet has NA
Proj_F_proportion. The check that some fleet takes projected F summed the
column without na.rm, so the NA that fleets taking no catch legitimately
carry made the sum NA and the if failed with missing value where TRUE/FALSE needed before the MSE started.
plot_indexresidual() now honours incl_proj. The argument was accepted
and then ignored, so projection years were always plotted -- the opposite of
its documented default, and of plot_index() / plot_catch(). A projection
row's "observation" is whatever placeholder its workbook carries (the
roll-forward scripts in Rceattle-models write 99999), which drew as an
enormous spurious residual. This removes rows from the default output for
any model whose index_data runs past endyr; pass incl_proj = TRUE to
keep them. No bundled dataset has such rows -- clean_data() extends
catch_data to the projection horizon but not index_data, and run_mse()
marks its projection index rows with a negative Year, which was already
filtered -- so in practice this bites only workbooks that carry their own.
Applies to the default residual_type = "pearson"; the "osa" path returns
osa_residuals() directly and takes none of the plot arguments.
plot_catch(incl_proj = TRUE) no longer errors on projection rows.
clean_data() gives projection years an NA catch, and the lognormal
error-bar mask in .fleet_fit_df() passed that NA to a subscript. The mask
now excludes NA observations, and those rows draw the fitted line with no
error bar, as non-positive observations already did. plot_index() shares the
code path and is fixed with it, though only a workbook that supplies NA
index observations can reach it.
data_check() now warns when CAAL data sit on a fleet whose
Selectivity_dimension is not "Length". Conditional age-at-length is the
age composition within a length bin, so the model predicts it from
selectivity-at-length convolved with the growth matrix. An age-dimensioned
fleet has no selectivity-at-length, so the prediction is zero -- and the
likelihood still runs, scoring the observations against the flat composition
the offset leaves behind. Those data then add a term to the objective that no
parameter can move: on the test fixture the CAAL component costs 64.6 with
uniform observations and 325.8 with realistic ones, and the whole objective
shifts by exactly that amount. Selectivity_dimension defaults to "Age", so
this was the default outcome for anyone adding CAAL data. It warns rather than
errors because such models currently fit; the sibling case (empirical growth,
which zeroes the prediction for the same reason) has always errored, and this
should tighten to match once configurations have been checked.
data_check() now rejects a diet_data that is not sorted by
stomach_id. The TMB diet likelihood walks diet_ctl with a single forward
cursor, taking stomach i's prey as the rows where stomach_id == i, so the
ids have to run 0, 1, 2, ... in order and with no gaps. Out of that order the
cursor runs past whole stomachs, which then drop out of the likelihood with no
warning and a lower objective: re-sorting a cleaned BS2017MS diet table by
predator age leaves 3 of its 45 stomachs in the fit, and reversing the rows
leaves 1. clean_data() sorts by stomach_id, so anything that came through
it already satisfies this; the check catches a hand-built or re-sorted table.
mse_summary() now reports the catch metrics for a species that has no
fishery. A multispecies model can carry a predator purely for its
consumption -- arrowtooth and sablefish in the hake model do exactly this --
and such a species has no catch_data rows at all. Every per-species catch
metric then operated on a zero-length vector: Average Catch warned
"argument is not numeric or logical: returning NA" and returned NA, while
Catch IAV and P(Closed) divided by a length of zero and returned NaN
silently. They now report Average Catch = 0 -- an unfished species really
does have zero catch -- and NA for the two ratios, which are not defined
when there is no fishery to vary or to close. A species that does have a
fishery but no landings in the projection years is reported as NA with a
warning naming the species and the years, because that is a data problem
rather than a modelling choice. Metrics for fished species are unchanged
(verified bit-identical on the 3-species hake MSE), as are $fleet and
$total.
?osa_residuals now explains the negative composition predicted values it
warns about, in a new section: why a numerical conditional mean overshoots
below zero on a near-empty bin, that it follows the bin's count rather than
the composition's sample size (so a rare age in a well-sampled year does it),
and that those residuals are biased positive. The warning itself is now two
sentences pointing at that section rather than carrying the whole explanation,
because it fires once per call on real assessments -- 269 rows on GOA-ATF-ESP
2025, 98 on GOA pollock 2025.
Corrected the comp_osa.hpp note claiming the pUsed squeeze in
dmultinom_osa() matches WHAM. WHAM does not squeeze pUsed; it squeezes
1 - pUsed at point of use, which Rceattle also does. The extra squeeze is
redundant and leaves an O(A * eps) difference from WHAM on an A-bin
composition. Comment only -- no value changes.
Fixed two doubled words in user-facing text: "the The Pacific Fishery
Management Council" in ?build_hcr, and "the the diet" in the diet_data
schema description, which is written verbatim into the bundled
meta_data_names.xlsx (regenerated to match).
fit_control(verbose = 0) now silences the estimability table printed when
a fit does not converge. TMBhelper emits it with print(), which
suppressMessages() cannot catch, so it appeared whatever verbose said --
one row per fixed parameter, repeated for every refit a self_test(),
jitter() or retrospective() run makes. It and the "Model did not converge"
banner now appear only under verbose > 0. The verdict is unchanged on
fit$identified and in fit$convergence.
A covariance (MVN/MVNORM) survey Sigma that is symmetric but not
positive definite is now caught by data_check(). The covariance index
likelihood factorizes Sigma, so symmetry is not enough. An indefinite or
singular matrix used to clear every check -- presence, squareness, dimension,
symmetry -- and then fail inside the TMB objective, in a message that named
neither the fleet nor Sigma. The error now names the fleet and reports the
smallest eigenvalue, so it is clear how far off the matrix is.
An index_cov entry for a fleet that is not using the covariance likelihood
now warns. It was ignored in both data_check() and the internal aligner, so
supplying a covariance but leaving Index_distribution unset gave a silent
lognormal fit with no indication the matrix had been dropped. Entries keyed by
either Fleet_name or Fleet_code are recognised, so a correctly configured
fleet does not warn.
self_test(debug = TRUE) returns every simulation, not just the converged
ones. When a self-test comes back short, the dropped runs are the ones worth
looking at, and each already carries its own $convergence diagnostics --
they were simply discarded. Under debug, Sim_i is simulation i (so it
pairs with the seed seed + i that produced it) and the verdicts are in
attr(sims, "converged"). The default return is unchanged. A refit that
errors outright is now caught and returned as its condition rather than
aborting the whole call (and, under a cluster, every other replicate with
it) -- previously the hardest failures were the ones you could not inspect.
jitter(timeout = ) and self_test(timeout = ) bound a single run.
.fit_tmb() optimizes with eval.max = iter.max = 1e9, so a replicate that
wanders somewhere pathological has no bound and one stall blocks the whole
run -- a failure no convergence check can reach, because the fit never
returns. A run past the limit is stopped, counted as non-converged, and
reported separately from the gradient drops, since the fix is different
(raise the limit, versus look at the model). The limit is approximate:
setTimeLimit() is checked when control returns to R, so it fires between the
optimizer's function evaluations rather than inside one. Both functions now
also trap errors per run, so one bad replicate cannot abort the rest.
self_test(start = ) chooses the starting values. "initial" (the
default, and the previous behavior) starts each refit from
initial_params -- what the original fit started from -- so the estimator
covers the same ground on simulated data that it did on the real data.
"estimated" warm starts from estimated_params, which is faster and far
more likely to converge but asks the estimator to travel no distance: it tests
that the optimum is stable under resampling, not that it is reachable.
reweight_comps(fleets = ) accepts fleet names as well as codes, matching
linkage_spec(fleet = ). Passing a name previously did not error: it fell
through as ineligible and surfaced as a warning about having no fitted
composition data, followed by "No fleets to reweight". An unknown name is now
reported against the model's own fleet list, and mixing ids and names in one
vector is caught with the coercion hint.
self_test() can now phase its refits, and inherits the setting by
default.
Every refit starts from the source model's starting values, so it covers the
same ground the original fit did -- but self_test() had no way to phase it.
retrospective() fixes phase = TRUE ("phasing, or the parameters dont wanna
move") and jitter() exposes the argument; self_test() took
.refit_like()'s phase = FALSE default with no argument to change it. A
model that needed phasing to fit its real data therefore had to reach the same
optimum in one unphased pass for every simulated one. On a GOA pollock
configuration that fits at a maximum gradient of 2e-4, 5 of 6 simulations
ended between 4e9 and 2e13; with phasing all 6 converged to ~1e-4. (That model
carries an sdrep, so the self-test ran at getsd = TRUE and those five were
dropped through the non-positive-definite path below -- under getsd = FALSE
the old gate would have returned all six and counted them as converged.)
phase reads the setting fit_mod() recorded on the source fit, so a model
fitted under the package default is still refitted unphased; pass
phase = TRUE for a model that needs phasing but was not fitted with it.
The re-fitting diagnostics' convergence gate now tests the gradient.
retrospective(), jitter(), self_test() and profile() each drop the runs
that did not converge, by comparing opt$Convergence_check against the string
TMBhelper::fit_tmb() uses for a non-invertible Hessian. That comparison could
not work in either direction. With getsd = TRUE, fit_tmb() returns before
assigning that string when the Hessian fails chol() -- so runs were dropped
by the enclosing is.null() guard, incidentally rather than by the test, and
by a criterion (positive-definiteness) that is not the one intended. With
getsd = FALSE that assignment is unreachable, so nothing was ever dropped
and a run ending at a maximum gradient of 1e13 counted as converged.
(Convergence_check itself is always set, but to one of the two gradient
verdicts, neither of which the comparison matched.)
Both paths now go through one gate that reads the hindcast maximum gradient,
using convergence_diagnostics()'s own FAIL threshold. Because the gate can
now actually drop, all three report how many runs they dropped rather than
returning a thinned list in silence -- for jitter() and self_test() a
thinned list is a biased sample, since the failures are exactly the runs that
would have shown the spread.
retrospective() survives a dropped peel. It named its output
Year_(endyr - peels):endyr -- always peels + 1 labels -- and looped
1:(length(mod_list) - 1), both of which assume every peel converged. One
dropped peel errored with 'names' attribute [3] must be the same length as the vector [2]; all of them dropped made 1:0 count down and index past the
end. The assumption held only because the old gate could not drop anything
(above). Entries are now labelled from each model's own endyr_peel, so a
dropped peel leaves a gap rather than shifting every later label onto the
wrong model.
retrospective() now judges the peeled hindcast, not just the F refit.
Each peel fits twice: the peeled hindcast, then a refit with only the peeled
years' log_F free. Only the second was gated, and its gradient says nothing
about whether the hindcast converged -- so a peel whose hindcast diverged was
kept as long as the handful of F parameters settled, and its parameters went
into Mohn's rho. Both refits are now gated.
A non-positive-definite fit no longer returns a malformed fit$opt.
On that early return TMBhelper::fit_tmb() hands back
list(opt = , h = ) rather than the estimates, so fit$opt$objective,
$max_gradient and $Convergence_check were all NULL. fit_mod() now
unwraps it, keeps the Hessian as $hessian, and records the verdict
fit_tmb() gives when sdreport reports pdHess = FALSE. fit$sdrep stays
NULL, so the sdreport_failed diagnostic is unaffected. A no-op for every
converged fit.
data_requirements() and build_data()'s pre-check read an attached
model_config(). They classified from the top-level switches only, so an
object carrying model_config(msmMode = 1) reported the predation inputs as
Ignored while print() on the same object showed it as multispecies.
fit_mod() resolves the configuration onto the data list, and the requirement
layer now sees the same thing. The overlay covers what model_config() carries:
estDynamics and Ceq have no field there, so NByageFixed and the
bioenergetics inputs still classify from the top-level slots.
The pre-check classifies an integer-coded fleet_control. It runs before
switch_check(), so a Selectivity still holding the code 0 never matched
the string "Fixed" and emp_sel was not required. The predicates now accept
the code, the canonical string, and the numeric-looking string a workbook read
can leave behind.
A mistyped mode switch is caught before the fit, not after it.
model_config() validates initMode, avgnMode and suitMode against the
switch maps, and fit_mod() checks them before doing any work. fit_mod()
rebuilds a model_config() after the optimization to record the run
configuration, so validating only there meant avgnMode = 3 -- a switch the
model never reads -- threw away a converged fit and its sdreport. That call
is now guarded as well, so recording the configuration can never cost a caller
a fit. The checks accept a per-species vector and an integer code, since
.refit_like() feeds resolved values back.
run_config() errors on an unrecognized field instead of silently dropping
it, naming the valid set and pointing at fit_control(), where the mistake
usually belongs -- run_config(fit, getsd = FALSE) was a no-op. Names are
validated, not values.
The linkage grammar documented a term that does not exist. A time block was
described as ~ block(Year) in the roxygen, the developer guide and the
vignettes, but there is no block(): the fixed part of a linkage formula is
passed to model.matrix(), so a model carrying that term failed at fit time
with could not find function "block". Documented as ~ cut(Year, ...), with
a note that anything model.matrix() accepts will work.
summary() on a data object no longer prints the model_config block.
print() still does, and both take config = to override, so
summary(dat, config = TRUE) and print(dat, config = FALSE) give the other
behavior. An object with no configuration prints identically either way.
read_data(), clean_data() and combine_data() return an
"Rceattle_data" object. A data list keeps its spec-tree printer through a
write_data() / read_data() round trip instead of only when it came from
build_data(). The visible consequence is that these now print an indented
specification tree at the prompt rather than dumping the ~40-element list. The
tag is inert: every consumer treats the object as a plain list, and it is
stripped before anything reaches MakeADFun(), so no fit changes.
data_requirements(index_loglike = ) is now index_distribution = ,
matching the canonical fleet_control column. The function has not been
released, so there is no deprecation path.
Runnable examples for linkage_spec(), write_template(), load_config()
and run_config(), none of which had one. linkage_spec() is the entry point
to the whole linkage grammar; its examples show the covariate, per-year,
penalized random walk and intercept-prior forms.
reweight_comps() is in the reference index (its absence aborted the pkgdown
build), is referenced from the composition-diagnostics workflow in
vignette("model-diagnostics"), and cites Francis (2011) as the alternative.
examples/McAllister-Ianelli-reweighting.R now uses it rather than teaching a
hand-rolled tuning loop.
Reference documentation describes current behavior rather than narrating past refactors, and the release notes for 5.3.0 and 5.4.0 are condensed to what changed, who is affected, and what to do about it.
A failed parallel OSA loop no longer crashes the R session.
osa_residuals() runs the one-step-ahead loop in parallel by default, via
forking, and retried serially if that failed -- but reusing the object the
failed attempt had used, which ended the session with "An irrecoverable
exception occurred" rather than recovering. The retry now builds a fresh
object first. Found on GOA pollock 2025, where osa_residuals(mod) was
unrunnable; it now returns its 1100 residuals, the same as
parallel = FALSE gave.
Forked workers abort on that model at any width above one core, and this is
usually absorbed: a direct one-step-ahead call over the same observations
returned all of them at 1, 2, 4, 8 and 11 cores. Occasionally the absorption
fails, and that case did not reproduce on demand -- it appears to depend on
machine load rather than on the model -- so the fix makes the retry safe
rather than preventing the underlying failure. The aborting worker still
prints its own message, which comes from C and cannot be suppressed; it does
not mean the call has failed. parallel = FALSE avoids the attempt entirely.
A negative expected composition count now warns (issue #108). predicted
is an expected bin count and cannot be negative, but it goes slightly negative
where a bin holds almost no fish: compositions enter as counts
((proportion + comp_offset) * N), and the one-step-ahead conditional mean is
a numerical step from the observation that overshoots below zero when the
count is near it. This follows the count in the bin, not the sample size of
the composition -- on EBS pollock the affected rows have a median observed
count of 0.05 against 4.9 elsewhere, while their sample sizes span the same
range as everything else, so a rare age in a well-sampled year does it too.
The affected years are named, and the warning notes that the residual on those
rows standardises the observation against the same mean, so it is biased
positive -- every affected row was positive in both Rceattle and WHAM. Nothing
is dropped or clamped: the negative value is the signal that the bin is too
sparse, and which rows to set aside is the analyst's call. This is not specific
to Rceattle; WHAM produces negative expected counts on its own example, with
the same default one-step-ahead method and composition counts on the same
scale.
OSA residuals on an accumulated composition are labelled by the age they
represent (reported as issue #108). Tail accumulation folds the tails into a
boundary bin, so a fleet with Comp_accum_young = 3 on a 10-age model is fit
on 8 bins -- but those were numbered 1 to 8, which labels the boundary residual
"age 1" when it stands for ages 1-3 combined and puts every other bin two ages
out. The residuals themselves were right; only the labels were positional.
age_length_bin now gives the age or length each residual belongs to, and a
new accumulated column marks the bins that carry a folded tail. A fleet
without accumulation is unchanged.
residuals() reports composition residuals on the bins the likelihood
fit. Tail accumulation folds the tails into a boundary bin and restricts the
likelihood to that window, but residuals() knew nothing about it -- so a
fleet folding ages 1-2 into age 3 still returned residuals for ages 1 and 2,
against a model that never fit them separately, and the boundary bin's
observed proportion excluded the tail it had absorbed. Both the composition
and the "pearson" attribute of osa_residuals() now fold first, so the
Pearson and OSA views of a fleet describe the same bins. The residual formula
is unchanged, and a fleet without accumulation is bit-identical. A new
Accumulated column marks the bins carrying a folded tail.
residuals() no longer returns composition residuals for observations the
model did not fit (issue #108). A row with Sample_size 0 enters no
likelihood -- the TMB guard is Neff > 0 -- but the Pearson path reported one
anyway, so a diagnostic plot showed residuals for data the model never used.
On the 2025 GOA pollock assessment that was 531 of 1568 rows: every length
composition (all carried at Sample_size 0) plus 8 unfitted age rows. The OSA
path already applied the guard, which is why the two disagreed. Same rule now
on both, for comp and caal.
The "pearson" attribute of an osa_residuals() result uses the same
column names as the result itself. It came from residuals(), which names
columns in the data-sheet style (Fleet_code, Year, Observed), so one
object carried two conventions and a reader had to know which half they were
holding. Shared names do not mean a shared scale, and ?osa_residuals now
says so: the attribute's observed/predicted are proportions with the
sample size alongside, since composition Pearson residuals are defined on
proportions, where the OSA columns carry bin counts.
sim_mod() draws the survey index under the fleet's own
Index_distribution. Every fleet was drawn as an independent lognormal
whatever it was set to, so an MVN/MVNORM survey was simulated on the wrong
scale and without the correlation its covariance carries: a survey with an
absolute sd of 20 and correlation 0.6 came back with a log-scale sd of 0.1 and
no correlation. Normal was likewise drawn on the log scale. Nothing errored,
so self_test() reported recovery against a data-generating process the
likelihood never assumed. Affects self_test(), jitter() and
run_mse(simulate_data = TRUE) for MVN/MVNORM/Normal surveys only --
all-lognormal models draw exactly as before. A natural-scale draw can come out
non-positive, which data_check() rejects; that now warns, since otherwise
the refit simply fails and the run is reported as not converged.
A parallel worker that dies no longer aborts retrospective(), jitter()
or run_mse(). Each worker fits whole models, so running several at once can
exhaust memory and the machine kills one. The call then stopped with
Error in unserialize(node$con) : error reading from connection, discarding
every peel or simulation finished up to that point. The tasks are now
recomputed sequentially instead -- slower, but the call returns -- and the
message suggests a lower cores. An error raised inside a worker still
surfaces as itself. These functions parallelize differently on Windows than on
macOS and Linux, so the same model can hit this on one and not the other.
Each retrospective() peel reports its own terminal year, so the peels show
up in a plot. Every peel carried the terminal year of the unpeeled model, and
plots take their year axis from it, so each peel was drawn across the full
series and they all lay on top of one another.
plot_biomass(retro$Rceattle_list) now gives the usual fan with no extra
arguments. Mohn's rho is unchanged.
A peel still estimates the years it dropped -- they are its retrospective
forecast, fit to the observed catch -- so endyr marks what the peel was fit
through, not how far it was estimated. incl_proj = TRUE plots those years,
and data_list$endyr_full gives the unpeeled terminal year where they end.
?osa_residuals no longer carries a broken link to parallel::mclapply().
The parallel argument's cross-reference had been generated with a
Windows-specific file anchor (parallel:mcdummies, where Windows documents the
forking dummies); mclapply lives in mclapply.Rd on Unix, so the link
resolved nowhere on macOS or Linux and R CMD check reported it as a missing
link. It now points at the topic rather than a platform's file, so it resolves
everywhere.
Released from main as 4.9.1, before this development line was merged back.
The two sections carry the same changes under different version numbers.
The stock-recruit convergence check now runs under the Ianelli
configuration. .check_stock_recruit() was keyed on srr_fun, so with
srr_fun = 0 and srr_pred_fun > 1 it returned no result at all -- the case
where 5.5.0 made \eqn{\alpha} estimable against a steepness prior, and so
exactly the case most in need of checking. It now reads srr_pred_fun there.
Steepness is the complete test: for Beverton-Holt
\eqn{h = \alpha \phi_0 / (4 + \alpha \phi_0)}, so \eqn{\alpha < 1/\phi_0} is
precisely \eqn{h < 0.2}. The reported \eqn{R_0} is not checked in that
configuration, because there it is the mean-recruitment level rather than a
value the penalty curve implies.
build_srr() warns when a steepness is passed as srr_est_mode = 0's
\eqn{\alpha}. srr_prior means steepness for srr_est_mode 2 and 3 but
\eqn{\alpha} for mode 0. Since 5.5.0 fixes \eqn{\alpha} at that value under
the Ianelli configuration, a steepness supplied by mistake is now pinned
rather than merely used as a starting value, putting the curve under the
replacement line with a negative implied \eqn{R_0}. A Beverton-Holt
srr_prior in (0, 1) under mode 0 now warns and gives the conversion
\eqn{\alpha = 4h / (\phi_0 (1 - h))}. It warns rather than errors because
\eqn{1/\phi_0} is not known until the model is built, and a small
\eqn{\alpha} is legitimate for a stock with a large \eqn{\phi_0}.
Released from main as 4.9.0, before this development line was merged back.
The two sections carry the same changes under different version numbers.
build_srr(srr_alpha_init = , srr_beta_init = ) sets the stock-recruit
starting values. The package defaults (\eqn{\alpha = e^3}, \eqn{\beta = 3})
are placeholders that carry no knowledge of the stock's scale. \eqn{\beta} sets
the density dependence in \eqn{R = \alpha S / (1 + \beta S)} and needs to be of
order \eqn{(\alpha - 1/\phi_0)/R_0} -- typically \eqn{10^{-3}} or smaller for a
stock in tonnes -- so the default sits three orders of magnitude away and
Beverton-Holt fits returned NA/NaN gradient evaluation before reaching a
first useful step. Both arguments take natural-scale values, one per species.
For a Beverton-Holt seeded from steepness \eqn{h} and unfished spawning biomass
per recruit \eqn{\phi_0}:
alpha <- 4 * h / (phi0 * (1 - h))
build_srr(srr_fun = "BevertonHolt",
srr_alpha_init = alpha,
srr_beta_init = (alpha - 1 / phi0) / R0)
init_dev is no longer declared a random effect under
initMode = "FreeParams" (0). ceattle.cpp applies the initial-deviate
density dnorm(init_dev, -sigma^2/2, R_sd) only when
initMode > 1 && initMode != 5, but fit_mod() added init_dev to the
Laplace random block whenever any element was free. Under mode 0 that made the
approximation integrate over an improper (flat) prior rather than estimate the
initial age structure as fixed effects, which is what mode 0 means and what
comparable platforms (WHAM's N1_model = 0, FIMS's log_init_naa) do.
Only initMode = 0 combined with random_rec = TRUE is affected, but for
that combination results move substantially -- on a small test model the
objective went from -91.6 to -83.3 and SSB by ~59%, because a flat prior and a
fixed effect are different models. No assessment in the ecosystem uses that
combination (every initMode = 0 model sets random_rec = FALSE), so nothing
in practice changes; refit if you do use it.
build_params() no longer seeds the stock-recruit \eqn{\alpha} from
srr_prior when srr_prior is a steepness. srr_prior is not the same
quantity for every curve: for Ricker it is a prior on \eqn{\alpha}, but for
Beverton-Holt it is a prior on steepness and must lie in (0, 1). Seeding
\eqn{\alpha} with log(steepness) started the optimizer at a near-zero
recruits-per-spawner. The seeding is now applied only where srr_prior is
genuinely an \eqn{\alpha}, matching the prior gates in ceattle.cpp.
This changes results for Beverton-Holt models with srr_est_mode 2 or 3,
which are the only configurations where srr_prior is a steepness. Six of the
24 srr_est_mode x srr_pred_fun combinations move; the other 18 are
byte-identical. In the ecosystem that is BSAI Atka mackerel
(srr_est_mode = 2, srr_prior = 0.8), where the starting \eqn{\alpha} goes
from \eqn{\log(0.8) = -0.22} to the package default and the fit moves by
~6e-4 in relative SSB. Refit affected models rather than assuming the old
numbers carry over.
Removed the spurious "alpha was not initialized to srr_prior" message.
fit_mod() copies recFun$srr_prior into the data_list only after the
caller has already built parameters, so the message fired on the documented
workflow and was not actionable.
A prior on steepness is now applied under the Ianelli configuration.
Steepness belongs to the stock-recruit curve, but it was only ever derived
inside the switch(srr_fun) block, so with srr_fun = 0 and
srr_pred_fun > 1 -- the AMAK/Ianelli setup, where the curve is estimated as a
recruitment penalty -- it stayed at the mean-recruitment constant 0.99. The
stock-recruit prior (srr_est_mode 2 or 3) is evaluated on steepness, so it
was being applied to a constant: no gradient, no effect on \eqn{\alpha}, and a
large fixed offset added to the objective. Steepness is now derived from the
penalty curve's \eqn{\alpha} in that configuration. \eqn{R_0} is deliberately
left alone, since recruitment there is \eqn{R_0 \exp(\mathrm{rec_dev})}.
This changes results for any model combining srr_fun = 0,
srr_pred_fun 2 / 3, and srr_est_mode 2 / 3 -- in the ecosystem that is
BSAI Atka mackerel (srr_prior = 0.8, srr_prior_sd = 0.0001), whose
steepness prior previously contributed a constant ~2.27e6 to the objective and
did nothing else. It now constrains \eqn{\alpha} as intended. Refit and expect
the reported objective to change substantially, mostly through the removal of
that offset. Models with srr_est_mode = 1 (no prior) are unaffected, as are
models whose hindcast already uses the curve (srr_fun > 1), where steepness
was always derived.
srr_est_mode = 0 ("fix alpha to prior mean") now works for Beverton-Holt.
The gate in fit_mod() was Ricker-only, so a Beverton-Holt fit that supplied
inits -- the normal warm-start workflow -- had \eqn{\alpha} mapped out by
build_map() but never set to the prior mean, leaving it pinned at
build_params()' placeholder \eqn{e^3}. build_params() and fit_mod() now
share one rule (.srr_prior_is_alpha()) so the two paths cannot disagree.
build_map() now keys the \eqn{\alpha} mapping on srr_pred_fun, the curve
\eqn{\alpha} belongs to, so srr_est_mode = 0 also fixes it under the Ianelli
configuration. \eqn{R_0} stays keyed on srr_fun and remains estimated there.
No model in the ecosystem currently uses srr_est_mode = 0.
srr_alpha_init / srr_beta_init survive a refit. .refit_like(), which
backs retrospective(), jitter(), self_test(), profile(), run_mse(),
remove_F(), sample_rec() and reweight_comps(), rebuilds the
stock-recruit specification from the stored data_list and now carries both
starting values through. Note they override a supplied inits, so
warm-starting from a previous fit's estimated_params restarts
\eqn{\alpha}/\eqn{\beta} at the specified values rather than the fitted ones;
leave them NULL to warm-start from the fit.
build_srr() validates the steepness prior instead of failing silently.
For a Beverton-Holt curve, srr_est_mode 2 and 3 put the prior on steepness,
so srr_prior must lie in (0, 1) -- this is now checked. The beta prior
(srr_est_mode = 3) additionally converts (mean, sd) to shape parameters by
moments, which are positive only when \eqn{sd^2 < \mu(1-\mu)}; outside that
range the shapes go negative and the prior is meaningless. It failed silently
rather than loudly, because TMB's lgamma-based dbeta returns a finite
value for negative shapes where R's dbeta returns NaN. Note the package
default srr_prior_sd = 1 is never valid for a steepness and now errors with
the largest permissible value.
fit$initial_params now records the parameters the model actually started
from. It was captured before the blocks that overwrite proj_F_prop,
log_Ftarget, log_M1 and the stock-recruit alpha / beta, so it reported
values no fit ever used. This was not only cosmetic: retrospective() and
jitter() reuse initial_params as their refit starting values
(R/9-retro_and_jitter.R). Those overrides are deterministic functions of the
data_list / HCR and are re-applied on every refit, so no fit changes --
confirmed bit-identical across retrospective, jitter, self_test,
profile, remove_F and three run_mse variants via
tools/verify/verify-refit-like.R.
steepness is reported for every year, not just the first. It is declared
[nspp, nyrs] but only column 1 was ever assigned, so
fit$quantities$steepness was a value followed by structural zeros. It is now
filled from the per-year alpha, which also generalizes to a time-varying
recruitment linkage.
R0 is deliberately left as it was -- column 1 carries the curve-derived value
under Beverton-Holt and the later columns the exp(rec_pars) level. Filling it
per-year makes R0 an AD function of alpha and beta across the whole series,
which feeds the reference-point and projection recruitment calls and costs
roughly 6x in fit time (a random-effects fit on a 30-year, 12-age model went
from 14 s to 91 s, with an identical objective) for a quantity that
Beverton-Holt recruitment does not use.
New stock_recruit convergence check. Rceattle parameterizes
Beverton-Holt by \eqn{\alpha} and \eqn{\beta} and derives steepness as
\eqn{h = \alpha \phi_0/(4 + \alpha \phi_0)}, with no lower bound -- unlike
WHAM, which builds \eqn{h} on (0.2, 1) by construction. Since rec_pars
carries no default bounds, the optimizer can reach \eqn{\alpha < 1/\phi_0},
where the stock cannot replace itself and the implied unfished recruitment
\eqn{R_0 = (\alpha - 1/\phi_0)/\beta} turns negative, propagating into the
initial age structure as NaN. fit$convergence now reports this as a FAIL
naming the species and the offending steepness / \eqn{R_0}.
linkage_spec(integrate = FALSE) estimates a random-effect linkage term as a
penalized fixed effect. A ~ (1|Year) / rw(1|Year) / ar1(1|Year) term is
normally integrated out by the Laplace approximation. Many reference
assessments -- ADMB/AMAK, goa_pk, and Rceattle's own legacy
Time_varying_sel / Time_varying_q switches -- instead treat such deviations
as penalized fixed effects: the deviations sit in the objective as a plain
penalty with a fixed SD and are not integrated. These are different models, not
a reparametrization, so a rw() could not reproduce them. On the GOA pollock
fishery selectivity walk, integrating moved the fit by ~119 jnll units and
shifted SSB ~14%.
build_selectivity(linkages = list(
slp_asc = linkage_spec(~ rw(1 | Year), fleet = "GOA_pollock_fishery",
init = list(sigma = 0.05), integrate = FALSE)))
Permitted only with a fixed SD, and rejected with observe = . See
?linkage_spec for the restrictions and
vignette("environmental-linkages-and-priors") for when to prefer it.
The linkage table records where each deviation is stored (re_pos).
re_index is the deviation's slot in the global random-effect numbering, but
the deviations are split across beta_linkage_re and beta_linkage_re_pen, so
the position within the vector that actually holds one is a different number as
soon as a model uses both treatments. re_pos gives that position, alongside
re_integrate for which vector, so setting inits by hand is
beta_linkage_re_pen[re_pos + 1] rather than a reconstruction of the split.
Indexing a mixed model by re_index writes to the wrong deviation without
erroring.
osa_residuals() falls back to serial when the parallel loop fails.
TMB::oneStepPredict(parallel = TRUE) forks via mclapply, and on some
model/observation combinations a worker aborts rather than returning. The dead
worker left an error object where a gradient belonged, which surfaced from deep
inside TMB as non-numeric argument to mathematical function -- a message with
no visible connection to the cause. The parallel loop is now retried serially,
reporting that it did so, so the call returns residuals instead of failing.
Seen on the GOA pollock model's index residuals; the other sources, and every
model tested with a linkage on catchability, run in parallel unaffected.
A selectivity prior on a limb the fleet's curve does not have is now
rejected. A Logistic fleet has no descending limb and a DescendingLogistic
fleet no ascending one, so those slots never reach selectivity-at-age -- they
sit at their build_params() defaults. A prior on them was still added to the
objective, shifting the reported likelihood by a constant that moved with an
unrelated default while doing nothing to the fit. fit_mod() now names the
fleet and parameter and says which limb to prior instead. This is how a
reconciliation against another model picks up an unexplained offset: on the GOA
pollock bridge it was worth 13.19 likelihood units.
Parameter bounds are applied to the parameter they were written for.
fit_mod() assembled lower/upper in build_params()'s parameter order,
but TMB orders obj$par by the sequence the PARAMETER_* macros appear in the
template -- and build_params() lists the linkage coefficients after log_F
while ceattle.cpp declares them before it. Both vectors have the same length
either way, so nothing downstream could notice: the box constraints landed on
the wrong parameters. The orders disagree in four places, so this reaches well
beyond linkages: rec_dev/R_log_sd, the growth block against
log_Flimit...log_F, index_q_beta/index_q_rho, and
sel_curve_pen/sel_coff_dev. Fits of growth-estimating and
linkage-carrying models can therefore change, since bounds that previously
bound the wrong parameter now bind the right one; refit them. The reference
fits are unaffected (their misplaced bounds never bound). Bounds are now
aligned against names(obj$par) after MakeADFun() and asserted non-NA,
since nlminb accepts an NA bound silently and returns objective = Inf
with par = NA rather than erroring.
est_phase no longer silently fails to fix a random-effect term.
est_phase reaches only beta_linkage, the fixed-effect coefficient vector; a
random effect's deviations live in a separate vector it never touches, so
est_phase = 0 on a formula containing (1|g) / rw() / ar1() left every
deviation estimated -- the opposite of the documented "fix at init" contract.
It is now an error naming the supported alternatives (drop the term, or fix a
small SD). Values above 1 remain inert for every linkage row.
Also covers the unreleased 5.2.0-5.2.4 development increments, whose entries were folded into this section.
reweight_comps() runs the iterative McAllister-Ianelli tuning loop.
Every fit reports the implied weights in
fleet_control$Comp_weights_mcallister, but tuning on them has meant copying
them across and refitting by hand, over and over. reweight_comps(fit) does
that loop: it refits until the largest relative change in any weight falls
below tol, returns the model fitted with the settled weights, and attaches
the per-iteration history as fit$reweight. Dirichlet-multinomial fleets
estimate their own weight, so they are named and skipped.
intercept is accepted wherever `(Intercept)` is. The key naming a
linkage's intercept came straight from model.matrix(), so setting an init,
bounds or a prior on it meant writing list(`(Intercept)` = ...) --
backticks, a capital I, and parentheses, with no error if any of it was wrong.
intercept now means the same thing in init, bounds and priors on every
process. The original spelling still works, and a covariate actually named
intercept still refers to itself.
An init, bounds or priors key that names nothing is now an error. A
misspelled key was ignored without complaint, and the model went on to fit from
a starting value, or under a prior, that was not the one asked for. Keys are
now checked against the linkage's own design columns (plus sigma / rho
where the structure has them); an unrecognized key stops the fit and lists the
keys that were available.
An intercept init or bounds reaches the base parameter for all six
processes. Setting the intercept of a recruitment, natural-mortality or
growth linkage set that parameter's starting value and bounds, but the same key
on a catchability, selectivity or composition linkage was accepted and then
dropped. All six now behave alike. Values are given on the parameter's natural
scale and stored on whichever scale the parameter uses, so a catchability of
0.8 or a von Bertalanffy K of 0.25 is written as given.
Three cases cannot be honoured, and each now stops the fit with a message
naming the linkage: a value of zero or less on a parameter held as a log; a
natural-scale value for a sel_inf slot that holds a logit or a log instead of
an inflection (DoubleNormal, LogisticPM); and a fleet that mirrors another's
selectivity or catchability, where the value would be overwritten by the fleet
leading the shared block.
osa_residuals() gains an "ecov" source for the state-space covariate of
a QAR1 catchability (build_catchability(..., observe=), Rogers et al. 2024).
It one-step-ahead residualizes the covariate observation with
TMB::oneStepPredict(), first and against its own series, as in WHAM's
make_osa_residuals(); the fit itself is unchanged. The AR1 latent is
zero-mean, so standardize the covariate -- build_catchability() warns
otherwise.
linkage_spec() selects species and fleets by name. species = now accepts
species names matching data_list$spnames, and fleet = accepts fleet names
matching fleet_control$Fleet_name, in place of 1-based ids:
q = linkage_spec(~ ar1(1 | Year), by = ~ fleet, fleet = "Shelikof")
M1 = linkage_spec(~ temp, by = ~ species, species = c("Pollock", "Cod"))
Ids continue to work unchanged. Names are resolved when the model is assembled
in fit_mod(); an unrecognized name -- or one that is not unique in
fleet_control -- errors and lists the model's own names, whereas a
Fleet_code that is wrong but in range silently attaches the linkage to a
different fleet and still fits. Matching is exact after trimming whitespace, and
a misspelled name is rejected even when the corresponding filter would be a
no-op. Named specs round-trip through save_config() / load_config().
run_mse() refits the operating model only as far ahead as the next
assessment needs. Between assessments the operating model has to reach one
assessment step past its terminal year -- far enough for the exploitable-biomass
cap on the next TAC -- but it was rebuilt over the whole projection every time,
and projyr sizes the AD tape, which costs roughly a fixed amount per model
year. The refit now runs on the shortened horizon and the operating model is
restored to the full projection afterwards; the final assessment keeps the full
horizon, so the returned operating model is unchanged. Measured at a 9%
saving on a Bering Sea multispecies MSE projecting to 2040, growing with the
length of the projection. Single-species models see little change.
A run with simulate_data = TRUE will not reproduce results generated before
this version, because the operating model carries fewer projection rows while
the refit runs and sim_mod() therefore draws a different number of random
values. Runs remain fully reproducible from a given seed within a version.
With simulate_data = FALSE results are unchanged to the bit.
Time_varying_sel = "RandomWalkAscending" no longer penalizes the descending
limb it never estimates. The random-walk selectivity penalty was gated on the
selectivity type alone, so on a double-logistic fleet the descending-limb term
was accumulated under mode 5 as well as mode 4. Mode 5 varies the ascending limb
only, so with random_sel = FALSE those deviates sat at their init of 0 and the
term contributed a pure constant -- on GOA2018SS (fleet 8, 42 years, one sex)
exactly 113.459 objective units. Models with no mode-5 fleet -- including
BS2017SS / BS2017MS -- are bit-identical.
A constant shift can still move the fit, and on GOA2018SS it does.
nlminb stops on an objective-relative tolerance, so an additive constant
changes where it halts and which local optimum it reaches. The fit now lands on
a converged optimum 52.9 units better (12868.005 rather than 12920.897 on the
same surface), and the derived quantities move with it:
GOA Pacific cod SSB in the final projection year (2050) falls 14.1%
(395,627 to 339,897 t), while SSB, recruitment and F differ by up to 40%, 39%
and 230% across the hindcast. Anyone carrying GOA numbers forward should
refit.
With random_sel = TRUE the removed term is not constant: sel_dev_log_sd
is estimated for a mode-5 fleet, so removing the term moves the estimated
selectivity-deviation SD upward and rescales shrinkage on the ascending walk.
The direction is right -- penalizing a deviate pinned at 0 is a spurious
-log sigma -- but it is a behavior change for such models.
All four pinned reference objectives are restated: the reference recipe now
Newton-polishes each fit to a stationary point (gradients ~1e-11) instead of
stopping on nlminb's relative tolerance, and test-golden-regression.R
asserts convergence alongside each pinned value. Note this certifies
convergence, not uniqueness -- use a jitter or multi-start check for that.
fit_control()'s newtonsteps = 0 default is unchanged; this affects the
reference recipe only, not any user fit.
retrospective() and run_mse() run again on a model with an observed
(state-space covariate) linkage. The map for the QAR1 observation SD
(log_obs_sd_linkage) was stored as a factor, where every other entry of
mapList is a raw integer. Both diagnostics build a debug map for each peel /
assessment, which maps the whole list out at once, and on a factor that step
aborted with argument "list" is missing, with no default -- so every peel of
such a model failed, reported under the parallel dispatch only as 5 nodes produced errors. Models with no linkage_spec(observe = ) group were never
affected, and no fit changes: the map TMB receives is identical.
A diagnostic refit keeps the source model's bias adjustment. fit_mod()
resets data_list's bias_adjust_obs / bias_adjust_proc and then re-applies
the values from fit_control(), whose defaults are TRUE. .refit_like()
built a fresh fit_control(), so a model fitted with bias adjustment off
refitted with it back on -- worth ~880 jnll units on BS2017SS. Bias
adjustment defines the likelihood rather than the optimizer, so this made
retrospective(), jitter(), self_test(), profile(), run_mse(),
remove_F() and sample_rec() compare two different objectives: a Mohn's rho
or a jitter spread computed this way was not measuring what it reported. The
resolved settings are now recovered from the data_list, which covers every
caller. Only models fitted with a non-default bias_adjust_* are affected;
comp_offset was already carried correctly and is now pinned by a test.
The optimizer and sdreport fields -- newtonsteps, use_gradient,
rel_tol, nlminb_control, getJointPrecision and the rest -- still fall
back to their defaults in a refit. Those choose how the optimizer runs rather
than what it optimizes, so they do not change the model, but a source fit that
needed non-default optimizer settings will refit without them. TMBfilename is
the exception worth naming: it selects the compiled template, so a model fitted
against an alternate template refits against the bundled ceattle and is not
the same likelihood. Refitting such a model through any of these diagnostics is
not supported.
sim_mod() simulates with the model's own observation bias adjustment. It
hardcoded the -sigma^2/2 offset on the simulated index and catch, which
agreed with the estimator only because a refit used to force bias adjustment
back on. With the refit now keeping the source setting, a model fitted at
bias_adjust_obs = FALSE would be simulated with the offset and then fitted by
a likelihood expecting none -- a systematic bias in scale, and so in
catchability, that no number of simulations averages away. self_test() and
run_mse(simulate_data = TRUE) on such a model would report that artifact as
estimation error. The simulator now reads the flag the same way
residuals.Rceattle() already did. Models on the default bias adjustment are
unaffected.
A simulation whose unfished reference run fails keeps an OM_no_F entry.
run_mse() assigned the failure result with sim_list$OM_no_F <- NULL, which
removes a list element rather than setting it to NULL, so the simulation
came back without the name at all. is.null() tests were unaffected, but code
keying on names() saw something different from what the documentation
described. The element is now present and NULL, as stated.
A failed re-assessment now stops its simulation instead of being absorbed.
When an estimation-model refit failed, run_mse() could not assign to
em_use, so it still held the previous year's assessment; the guard that
followed tested is.null(em_use), which such a failure never produces. The
simulation carried on, stored that stale assessment under the failed year's
name, handed its catch advice to the next iteration, and was reported as
successful once a later iteration reset the failure flag. A simulation whose
assessment fails is now stopped and marked like one whose operating-model
refit fails. Any earlier run is worth re-checking: a failed assessment left no
trace in use_sim.
A simulation whose unfished reference run fails is no longer counted as
"did not collapse". remove_F() is a separate fit and can fail on its own.
The simulation is kept, since its assessments are the catch-advice record, but
OM_no_F is then NULL -- and sum(NULL < 1000) > 0 is FALSE in R, so
mse_summary() scored it as a non-collapse while keeping it in the
denominator. Because remove_F() fails preferentially at low stock sizes, that
biased the collapse metrics low, in the direction that flatters a harvest rule.
Such simulations are now excluded from the OM no F and SSB Collapse from F
metrics -- numerator and denominator both -- which report NA when none has an
unfished run; every other metric still uses them. Under msmMode > 0 a single
such simulation previously threw and took the whole summary with it.
run_mse(dir = ) accepts an absolute path. The save built its target as
file.path(getwd(), dir), which turns an absolute dir into
<cwd>/<abs path>, so the directory was never created and the run died with
"cannot open the connection". Relative paths were unaffected.
sampling_period is now honoured per fleet when more than one year advances
per assessment. run_mse() selected the newly sampled rows by testing the
year set and the fleet set separately, which is the same as matching
fleet-year pairs only while years_include spans a single year. With
assessment_period > 1 and fleets on different sampling periods, the two
tests together admitted every fleet in every year of the step: a fleet
surveyed every second year was given observations in its off years as well.
The estimation model was therefore fit to more survey and composition data
than the sampling design specifies, understating terminal-year uncertainty and
making a data-poor design perform like a data-rich one. Rows are now matched on
the fleet and year together. Runs with assessment_period = 1 -- the default
-- are unaffected; results move for any run combining a longer assessment
period with per-fleet sampling periods.
sample_rec(update_model = TRUE) works on models carrying catchability,
selectivity, or composition linkages. .refit_like() reconstructs the
build_catchability() / build_selectivity() / build_composition()
specifications from the source model, but sample_rec() still re-invoked
fit_mod() through its own hand-written copy of that block, and so omitted all
three. Because fit_mod() treats those arguments as the source of truth for
q_linkages / sel_linkages / comp_linkages, the rebuilt model had no
linkage table and therefore no beta_linkage parameters, which no longer
matched the parameters carried over from the source model. The inits check in
fit_mod() caught that and stopped with a length-mismatch error, so
sample_rec(update_model = TRUE) failed outright on any model using an
environmental linkage or a prior on catchability, on a selectivity parameter,
or on a Dirichlet-multinomial composition weight — it did not return quietly
wrong dynamics. sample_rec() now routes through .refit_like() like every
other refit.
Routing through .refit_like() also means sample_rec(update_model = TRUE)
now carries the source model's random_q and random_sel settings into the
rebuild, where the hand-written block left them at FALSE. For a model that
estimates time-varying catchability or selectivity as random effects, the
rebuilt object now keeps those random effects instead of dropping them.
The "invalid switch" errors for initMode and HCR now name the offending
value. They previously read Invalid 'initMode' specified: . with the value
dropped; they now echo what was passed (e.g. a since-renamed alias), alongside
the list of accepted names and integer codes.
The diet Dirichlet-multinomial weight is no longer estimated where there is
no diet likelihood. build_map() freed diet_comp_weights for any predator
with Diet_distribution = "DirichletMultinomial" whenever msmMode > 0, but
the model only fits the diet composition for predators with
suitMode > 0 -- under empirical suitability (suitMode = 0) the diet
proportions are taken as given. A multispecies run with empirical suitability
therefore carried one free, wholly uninformed overdispersion parameter per
predator, inflating the parameter count and risking a rank-deficient Hessian
under getsd = TRUE. The weight is now estimated only where the diet
composition is fit. Fits that did not use a DM diet are unchanged.
A composition-weighting prior on a fixed DM weight is now ignored instead of
adding a constant. A theta_comp / theta_caal / theta_diet prior from
build_composition(linkages = ) is prior-only, so when the map fixes the
weight it targets -- a single-species fit, empirical suitability, an "Off"
fleet, a fleet with no composition data, or a user-supplied map -- the prior
contributed a constant that shifted the reported jnll without moving any
estimate, making likelihoods non-comparable across configurations. Such priors
are now dropped with a message. The linkage rows themselves are kept, so
beta_linkage keeps its length and inits remain portable between fits that
share one compFun. linkage_spec() still errors up front on a prior that
can never apply to the data at hand (a non-DM Comp_distribution /
Diet_distribution).
Hake selectivity no longer warns about a valid Time_varying_sel = 0.
build_map_selectivity() warned "Time_varying_sel for fleet N is not
compatible ... Current value: Off" on every fleet that was not "IID" --
including "Off" (0), which its own message and data_check() both list as
valid for the "Hake" (Taylor et al. 2014) form. Because fit_mod() wraps
build_map() in suppressWarnings(), the spurious warning surfaced only in
the callers that build the map directly, notably run_mse() and
retrospective(), where it appeared once per iteration. The map itself was
always correct (no coefficient deviates are estimated for "Off"), so fits
are unchanged. The warning now fires only for a mode the "Hake" form cannot
represent, and names 'Off' rather than the non-existent 'None'.
The diagnostic refit paths preserve catchability / selectivity / composition
linkages. .refit_like() -- the engine behind run_mse(),
retrospective(), jitter(), self_test(), profile(), and remove_F() --
rebuilt the recruitment, M1, and growth specifications from the source model
but silently dropped the build_catchability() / build_selectivity() /
build_composition() linkages. On a model using any of them (e.g. a
Dirichlet-multinomial composition-weight prior), the reconstructed model
lacked the linkages' beta_linkage parameters while the warm-start values
still carried them, and TMB::MakeADFun() segfaulted on the length mismatch.
The three linkage constructors are now rebuilt from the stored data_list
fields (q_linkages / sel_linkages / comp_linkages), so these models run
through the MSE / retrospective / jitter loops.
run_mse() extends the selectivity-deviation parameters at the model's own
sex dimension. The projection-year extension of log_sel_slp_dev /
sel_inf_dev / sel_coff_dev hardcoded the sex dimension to 2, which for a
single-sex model both recycled the fitted values into a phantom second sex and
produced a warm-start whose length disagreed with the parameter template. Now
taken from the fitted arrays (max_sex).
fit_mod() raises a clear error instead of crashing on mismatched
inits. When the supplied inits omit a parameter the model declares, or
any parameter's length disagrees with the model implied by data_list
(a dropped linkage, or a warm start not extended to a later endyr),
fit_mod() now stops with a message naming the parameter and its lengths,
rather than letting TMB::MakeADFun() segfault in getParameterOrder().
The diagnostic refit paths preserve the random_q / random_sel switches.
These are now stored on data_list (like random_rec) and forwarded by
.refit_like(), so a random-catchability or random-selectivity model keeps its
random-effect structure across MSE / retrospective / jitter refits instead of
silently reverting to fixed effects.
build_params() no longer pads init_dev behind a string comparison. The
-999 padding of the unused init_dev columns was partly gated on
initMode > 0, evaluated against the canonical initMode string that
switch_check() has already resolved — "FreeParams" > "0" is TRUE, so the
gate was always open. The padding is unconditional now, which is what the C++
requires (every mode reads only columns 1:(nages - 1)). No fit changes:
build_params() is bit-identical for all six modes on BS2017SS and
BS2017MS, from both string and integer input. This was the only place whose
behavior depended on the lexicographic value of a switch alias, so an alias
beginning with a digit can no longer flip it.
Composition weights now warm-start from inits like every other
parameter. fit_mod() re-read comp_weights, caal_weights and
diet_comp_weights from their fleet_control columns on every fit, even when
inits was supplied, so a weight handed to a refit was discarded. These are
estimated parameters -- the Dirichlet-multinomial likelihood fits them, which
is why they can carry a prior -- and their columns are starting values, read
when a model is built from scratch. A refit now keeps the estimate it was
given, which is what makes iterative reweighting possible and what every other
parameter already did.
Which models this moves depends on whether the weight is estimated. Under a
multinomial likelihood it is a fixed multiplier that never leaves its column
value, so those models -- including the bundled examples -- are unaffected, and
an MSE over them is bit-identical. Under a Dirichlet-multinomial it is
estimated and can move a long way: fitting BS2017SS with DM composition puts
the weights between -0.6 and 12.7 (log scale), starting from the default column
value of 1. Results therefore move for any DM model under every
run_mse(), retrospective(), jitter(), self_test() and profile()
refit, each of which previously discarded the fitted estimate and restarted
from the column.
Editing a Comp_weights column and re-fitting from an existing fit no longer
has an effect. Either build the model afresh (inits = NULL), or set the
parameter the fit actually reads -- inits$comp_weights[flt] <- w (likewise
caal_weights / diet_comp_weights) -- which is what reweight_comps() does.
fit_mod() now warns when supplied inits disagree with the columns, so an
edit that would once have taken effect silently is no longer silent.
A simulation that does not run to completion now returns only a marker.
run_mse() previously returned the partially advanced operating and estimation
models alongside use_sim = FALSE. Those models describe a state the
simulation never reached, and nothing downstream filtered on use_sim, so they
could be averaged into performance metrics as though the simulation had
finished. A failed simulation is now list(use_sim = FALSE, failure = ...)
with no models attached, and mse_summary() drops such simulations up front
with a warning naming them, or errors if none completed. Code that reached into
mse$Sim_n$OM, $EM or $OM_no_F without checking use_sim should now check
it -- filter first with
mse <- mse[vapply(mse, function(x) isTRUE(x$use_sim), logical(1))].
Help pages, code comments, and vignettes were rewritten for clarity. No code or model behavior changed.
Four help pages that disagreed with the code were corrected. jitter()
now states that starting values are perturbed around the model's initial
(pre-fit) parameters, not the fitted values; the build_params() and
TMBphase() @return descriptions were fixed (they had described a map object
and standard errors respectively); and a mislabeled comment now correctly
attributes the time-varying survey catchability SD.
Diet_distribution accepts string aliases. Like Comp_distribution and
CAAL_distribution, the per-predator diet composition likelihood switch now takes
"Multinomial" (0) or "DirichletMultinomial" (1) in addition to the integer code
-- previously it accepted only the integer. The diet likelihood has no AFSC variant,
so "MultinomialAFSC" / -1 is rejected with a clear message.
Clearer spec-tree printing for data objects and fits. print()/summary() of an
Rceattle_data object or a fitted model now shows each switch by its string alias
(e.g. srr_fun = mean, M1_model = fixed, msmMode = MSVPA) rather than its raw
integer code, so the outline reads the same even if a code is renumbered. The fleet
section is laid out as aligned, │-separated columns with sel: / q: labels and a
[shared: ...] tag on mirrored selectivity/catchability blocks.
alpha_wt_len, beta_wt_len, and
Selectivity_dimension now print only when growth is estimated (growth_model > 0);
CAAL_distribution / CAAL_weights only when the model has CAAL data; and
Sel_norm_bin_upper only when a fleet normalizes selectivity at a specific bin. The
default value is still applied silently in every case -- this only stops the messages
nagging about inputs the configuration never consumes (e.g. weight-length parameters
in a single-species age-based model).linkage_spec() fills in by from the process it is attached to. Omitting
by now defaults to the base stratum of the process -- ~ fleet for
catchability, selectivity, and the fleet composition weights (theta_comp /
theta_caal), and ~ species for recruitment, M, growth, and the diet weight
(theta_diet) -- so a base-case linkage no longer needs an explicit
by = ~ fleet / by = ~ species. An explicit by (a formula, or NULL for a
single shared coefficient) is always kept as given, so code that passes by
explicitly -- as the bundled examples and real assessments do -- is unchanged.
A linkage that omitted by on a fleet-keyed process does change: it now spans
every fleet (with a separate coefficient each) rather than collapsing to fleet 1.
In particular a ~ 1 selectivity/catchability prior now applies per fleet, so
name the target fleet(s) via linkage_spec(fleet = ...) -- and note it will error
on a model with mirrored selectivity fleets unless a fleet is named. Restrict a
covariate or prior to the fleets you mean with the fleet argument. When by is
omitted on a catchability/selectivity linkage, fit_mod() now prints a one-time
message noting that it spans every eligible fleet, so the per-fleet expansion is
not a surprise.q
linkage only makes sense on a fleet whose catchability is estimated. Fleets that
hold q fixed or solve it analytically were already rejected; a fleet with no
index data (so Catchability is NA and there is no q to estimate) was not, so
an omitted-by linkage silently attached meaningless q coefficients to, e.g.,
fishery fleets. Such fleets are now rejected with the same clear error, naming them
and pointing to linkage_spec(fleet = ...) to restrict the linkage.right_floor selectivity linkage now applies its offset on the logit scale.
For a double-normal fleet, sel_inf(1) parameterises the right-tail floor as
logit(right_floor), but a log-link linkage offset was applied by multiplying the
already-transformed probability, which could push right_floor above 1 and corrupt the
descending limb. The offset now acts on the logit (additive and multiplicative), like the
peak and the other inflection slots, so right_floor stays in [0, 1] for any offset.
A fleet with no right_floor linkage is bit-identical (the four reference models are
unchanged).log_sigma_linkage and log_obs_sd_linkage (the ~ (1 | Year) / rw() / ar1()
and Rogers-2024 QAR1 SDs) and index_q_log_sd (the legacy Catchability = "AR1" q
observation SD). Left free, each was estimated jointly with its own penalized deviates and
could collapse toward 0, so a phased fit silently converged to the random-effect-off
solution -- the estimated time-variation vanished -- instead of the correct optimum. All
three are now held during phasing like every other process SD. Fits without these random
effects are unaffected (the four reference models stay bit-identical).src/TMB/ceattle.cpp (was ceattle_v01_11.cpp), and its
compiled DLL is ceattle. Internal rename only -- the model and results are
unchanged (the four reference models stay bit-identical), and fit_mod() loads the
renamed DLL automatically. A fit built with a custom fit_control(TMBfilename = ...)
is unaffected. Caveat: an object fitted by an earlier version stores
TMBfilename = "ceattle_v01_11", so re-running osa_residuals() on such a saved fit
needs it re-fit under this version first (the old DLL name is no longer built).?build_selectivity prior example into @examples (it previously rendered
a broken \verb{}); corrected the ?build_hcr PFMC notation to use the normal quantile
qnorm() rather than Phi (which denotes the CDF); and regrouped the 4.10.0 linkage
additions under New features. Minor vignette / README fixes.random_sel = TRUE), integrating a flat field gives a NaN marginal objective
and the fit could not start without an external warm start. Phasing the deviates
as penalized effects first builds up realistic non-zero values, so the final
Laplace begins near the mode. Gated on the presence of random effects, so fits
without any are unchanged (the four reference models remain bit-identical).OSA residuals now support every survey-index likelihood family.
osa_residuals() previously produced one-step-ahead residuals only for the
lognormal IID index (Index_distribution "Lognormal"). It now covers all
families: natural-scale "Normal" residualizes as an independent normal on the
natural scale, and the correlated covariance families "MVN" / "MVNORM" are
whitened by the lower Cholesky of the fleet's survey covariance
Sigma = L L', so the residuals are the multivariate-Gaussian one-step-ahead
innovations L^-1 (obs - q*pred) — the closed form TMB::oneStepPredict()
reproduces for a Gaussian block (Thygesen et al. 2017). This supersedes the
5.0.2 exclusion of these fleets. The ordinary model fit is unchanged (the
whitening applies only to the post-hoc OSA observation vector).
String aliases for the fit_mod() mode switches. estimateMode and
msmMode now accept readable names alongside their integer codes:
estimateMode = "Estimate" (0), "Hindcast" (1), "Projection" (2),
"DebugBuild" (3), "DebugOptimize" (4); msmMode = "SingleSpecies" (0),
"MSVPA" (1), "TypeIIIMSVPA" (2). The integer codes still work, and an
unrecognized string errors with the valid options listed.
osa_residuals() / build_osa_data() previously laid MVN-covariance
(Index_distribution "MVN"/"MVNORM") and natural-scale "Normal" survey
observations into the one-step-ahead observation vector as if they were
independent lognormals, even though the C++ likelihood for those families does
not read the OSA observation vector — producing invalid residuals with no
warning. Those fleets are now excluded from the OSA residuals with a warning
naming them; lognormal (IID) index fleets in the same model are unaffected.
(Covariance-aware OSA residuals for the MVN families are a separate follow-up.)Comp_accum_young / Comp_accum_old (AFSC
ac_yng/ac_old), osa_residuals() now residualizes the folded composition
that was actually fit -- one residual per fitted (folded) bin per sex block,
rather than refusing the combination. build_osa_data() applies the same
per-sex-block young/old fold to the OSA observation vector, so the one-step-ahead
decomposition matches the fitted composition likelihood bin-for-bin. Fleets
without accumulation are unchanged.Also covers the unreleased 4.14.0 development increment, whose entries were folded into this section.
mse_summary() now returns a per-entity list instead of one stacked
data.frame. The result is list(species, fleet, total, meta): species
(one row per species, keyed by Species) holds the conservation/status
metrics; fleet (one row per fishery fleet) holds average catch / catch IAV
/ P(Closed); total is the across-fleet totals; meta carries run
provenance. This removes the NA padding and the ambiguity of the old shape,
where species, fleet, and "All" rows were stacked in one frame and a column
like Average Catch meant different things per row. The metric values are
unchanged. Update summ$"<metric>" call sites to summ$species$"<metric>"
or summ$fleet$"<metric>".ConstantFSPR harvest control rule (HCR = 4) applying Fmult
twice, which set the projected F to Ftarget * Fmult^2. It is now
Ftarget * Fmult. Only affects projections with Fmult != 1 (the default
Fmult = 1 is unchanged).?build_hcr: the SESSF
(Tier 1) and NPFMC (Tier 3) fishing-mortality ramps and the PFMC 40-10
buffer (the normal quantile function, not the CDF).plot_logindex() as a deprecated shim for plot_index(log = TRUE).
It forwards to plot_index() with a .Deprecated() notice.build_growth(sd_plus_group=)). For
estimated growth, choose how the oldest age class's standard deviation of
length-at-age is set: "WHAM" (default) pins it to the upper anchor
exp(sd_Linf) (the WHAM SDAA convention, unchanged from prior releases);
"SS3" instead interpolates it by length like any interior age. Per-species
(scalar or length-nspp), round-trips through save_config()/load_config(),
and affects only growth_model > 0 fits. Existing models are bit-identical
under the default.The NA-handling diagnostic in check_caal_data() now reports "CAAL data
have NAs ..." instead of mislabeling them as "Composition data" (the
message was copy-pasted from check_composition_data()). Message text only;
no change to the data or fit.
model_average(uncertainty = TRUE) no longer opens a plot device partway
through the computation. A stray plot_ssb() call fired before the averaged
sdrep$sd was populated, so it drew stale, pre-averaging confidence
intervals as an unrequested side effect; the averaged object it returns is
unchanged.
Save / load a run configuration (save_config() / load_config()). A full
run configuration -- the [model_config()] structure plus the estimation controls
and fit_control() bundle -- round-trips to a documented, git-diffable YAML file
(each field carries its doc string as a comment; fields at their default are
omitted so two runs diff to only their real differences). Apply a saved config
with fit_mod(data_list, config = load_config("run.yaml")); it fills only the
arguments the caller did not pass (an explicit argument always wins), and every
fit now records the run configuration it used (fit$run_config, also reachable
via run_config(fit)). Follows SAM's saveConf/loadConf and the
config-separate-from-weights idiom; adds a yaml dependency.
Composition tail accumulation (AFSC ac_yng / ac_old). Two new
fleet_control columns, Comp_accum_young and Comp_accum_old, fold the
young and old tails of a fleet's age/length composition into a boundary bin
before the composition likelihood (per sex block for joint-sex comps), for
every composition family including the default MultinomialAFSC. They are
1-based bin ordinals on the fleet's composition dimension; leaving them unset
(or young = 1 / old at the last bin) applies no accumulation, so every
existing model is bit-identical. data_check() rejects out-of-range or
inverted (young > old) bins, and OSA residuals are not available for a fleet
with active accumulation (the residuals are built on the un-accumulated bins).
self_test() no longer errors with object 'getsd' not found. Its
per-simulation refit closure referenced a getsd value that the function --
unlike retrospective(), jitter(), and profile() -- never defined, so
every simulation died on both the sequential and the parallel-cluster path.
self_test() now takes a getsd argument (default NULL, inheriting the
input model's setting) like its sibling functions.Intuitive fleet_control / data column names. The workbook and data-list
columns were renamed to say what they hold, each keeping a back-compat alias
that is upgraded on read, so existing data lists, .rda, workbooks, and
scripts keep working (with a one-time deprecation message) and fit
identically: Q_index/Q_init/Q_sd_prior →
Catchability_index/Catchability_init/Catchability_prior_sd;
Comp_loglike/CAAL_loglike/Index_loglike →
Comp_distribution/CAAL_distribution/Index_distribution;
Sel_norm_bin1/Sel_norm_bin2 → Sel_norm_bin/Sel_norm_bin_upper;
weight1_Numbers2 → Observation_units; proj_F_prop → Proj_F_proportion;
the control scalar sigma_rec_prior → sigma_rec; the bioenergetics
Diet_loglike → Diet_distribution; and the post-fit output column
Est_weights_mcallister → Comp_weights_mcallister (the old name is still
written for downstream readers).
write_template(). A new export that writes a minimal, structurally
complete single-species starter workbook on the canonical column names; it
round-trips through read_data() and builds under fit_mod(estimateMode = 3).
Single-source workbook schema. The column dictionary now lives once in the
package (R/0-column_schema.R) and drives switch_check() defaults,
write_data() object order, the embedded meta_data documentation sheet, and
the roxygen field dictionary, which are kept in sync by guard tests. ~16
previously used-but-undocumented columns are now documented.
read_data() names the offending sheet or cell instead of failing
cryptically. A required sheet (control, fleet_control) is named when
absent, and optional sheets are skipped, so a minimal single-species workbook
reads cleanly. A non-numeric cell in the control or bioenergetics rows is named
rather than silently becoming NA. rearrange_data() likewise fails clearly
on a malformed fleet_control rather than via a cryptic dplyr error.
Removed the dead accumulation-age feature. Accumulation_age_lower/upper
were only range-validated and never applied to composition data; they are
dropped (with a soft-deprecation message if an old workbook carries them).
initMode = "OffsetEquilibrium" (5). A new initial-age-structure mode: an
F = 0 equilibrium seeded by the first-year recruitment
(R_init * exp(rec_dev[year 1])) decayed by residual natural mortality M1,
with initial deviates turned off and no init-dev penalty. Unlike
initMode = "Equilibrium", which sits at the initial equilibrium recruitment
R_init, the initial numbers track the year-1 recruitment deviation, matching
the AFSC GOA pollock (Cole Monnahan) convention — the first-year cohort and
the initial numbers-at-age share one deviation. The name refers to that
offset; the equilibrium itself is unfished. Note the deviation enters as a
single scalar on every age, so it moves the level of the initial numbers-at-age
without changing their proportions-at-age. Every other mode is unchanged
(golden references bit-identical).
Priors on base selectivity parameters through the linkage grammar. A
selectivity linkage with an intercept-only formula and a priors entry now
places a prior on the base logistic parameter that carries the level — the
ascending/descending slope (slp_asc / slp_desc, on log_sel_slp, log
scale) or inflection (inf_asc / inf_desc, on sel_inf, natural scale),
e.g. build_selectivity(linkages = list(inf_asc = linkage_spec(~ 1, priors = list(\(Intercept)` = normal(0, 3))))). Use lognormal()for the log-scale slopes andnormal()for the natural-scale inflections. This mirrors the prior-onlybuild_composition()path and reproduces the AFSC GOA pollock selectivity priors. (Previously such an intercept prior was silently inert for selectivity — thesel` process was missing from the prior re-targeting.)
Per-predator suitability reference years (suit_styr / suit_endyr).
These fit_mod() arguments (and the underlying data_list fields) now accept
a vector of length nspp so each predator can average its suitability over a
different set of years — e.g. a California Current model with hake 1980–2019,
arrowtooth 2013–2018, and sablefish 2005–2008. A single scalar is recycled to
every predator, exactly reproducing the previous global-window behavior
(BS2017MS and the golden references are unchanged to numerical tolerance).
Internally suit_styr / suit_endyr became per-predator DATA_IVECTORs and
the suitability-averaging and stomach-content prediction loops index them by
predator.
plot_diet_comp2() and plot_diet_comp1(). plot_diet_comp2() adds
aggregation-aware diet-composition diagnostics (line plots when prey- or
predator-age is aggregated, dodged bars when both are, bubble grids when fully
disaggregated), built on residuals(source = "diet"). plot_diet_comp1() is
an alias of plot_diet_comp() (the bubble/grid diagnostic).
read_data() tolerates trailing empty age columns in NByageFixed. Older
writers pad the fixed numbers-at-age sheet to a wider age range than
max(nages); the all-NA trailing columns are now dropped on read instead of
tripping the data_check() column-count validation.
Parallel workers now run the in-session package. run_mse(),
retrospective(), and jitter() use a FORK cluster on non-Windows platforms,
so workers run the parent's loaded namespace via copy-on-write. This fixes
silently running a stale installed package on the workers during
pkgload::load_all() development sessions, and removes the per-worker startup
cost. Parallel and serial runs stay bit-identical for a given seed; Windows
keeps the previous PSOCK path as a fallback.
Random-effect linkages (~ (1 | group)). A linkage formula may now carry
an IID random effect, so a parameter can vary year to year (or over any
grouping) as a set of deviations damped by an estimated
N(0, sigma) density, rather than fixed covariate slopes:
fit_mod(...,
qFun = build_catchability(linkages = list(
q = linkage_spec(~ (1 | Year), by = ~ fleet))))
Each level of the grouping variable gets one deviation (beta_linkage_re);
each distinct group estimates its own log-SD (log_sigma_linkage). The
density is reported in the new jnll_comp row "Linkage random effects".
The deviation SD is routed through the same linkage_spec() arguments as
every other parameter: init = list(sigma = v) fixes it at an input
value (reproducing the legacy Time_varying_*_sd_prior fixed input), and
priors = list(sigma = lognormal(...)) places a prior on it and
estimates it — the first prior on a deviation SD anywhere in the model.
data_requirements() — see which inputs a model configuration needs. A new
exported reader reports, for a given model spec, which top-level data inputs are
Required, Optional (used if supplied, otherwise default-filled by
clean_data()), or Ignored (not consulted because the feature is switched
off) — the same conditions data_check() enforces at fit time, surfaced up
front instead of buried in the validator:
data_requirements(msmMode = 1) # multispecies: diet/ration/bioenergetics Required
data_requirements(BS2017SS, msmMode = 0) # preview a data object as single-species
It accepts either an existing (possibly partial) data list or the convenience
switch arguments; an explicit switch argument overrides the data list's stored
value (matching fit_mod() precedence). Internally, data_check()'s conditional
presence-requirement gates were refactored to consume one declarative
requirement table, so the reader and the validator can never drift apart.
build_data() — assemble a data list in R. A code-first constructor
complementing read_data(): supply only the blocks a model uses and the
optional blocks a single-species model does not need are default-filled by
clean_data(). Three combinable entry points cover the workflows real
assessments use:
build_data(base = BS2017SS, projyr = 2060) # copy-and-edit a dataset
build_data(file = "model.xlsx", fleet_control = fc) # read a workbook, override
build_data(nspp = 1, styr = 1977, endyr = 2023, # assemble from blocks
fleet_control = fc, catch_data = catch, index_data = survey)
Overrides are checked against the recognized schema, so a typo (maturty) is
caught at construction with a suggestion rather than surfacing later in a fit;
legacy names (fsh_biom, srv_biom, wt, pmature, Pyrs) are mapped to
their canonical equivalents. Validation is deferred to data_check() at fit
time (one source of truth); build_data() runs only a light presence
pre-check so a missing required block is reported early. The result is the
same bare list read_data() returns and round-trips through write_data()
unchanged — a build_data(base = X) object fits bit-identically to X.
model_config() — a model configuration that travels with the data. The
model-structure arguments of fit_mod() (msmMode, initMode, the HCR and
the build_*() process specifications) can now be bundled into a slot on the
data list, so a data object records how it is meant to be fit:
dat <- build_data(base = BS2017MS, model_config = model_config(msmMode = 1))
fit_mod(dat) # fits as multispecies without passing msmMode
fit_mod()'s signature and defaults are unchanged; when a data list carries a
model_config, fit_mod() reads each field only for arguments the caller did
not pass (detected with missing()), and an explicitly-passed argument
always overrides the slot. With no slot present the behavior is byte-identical
(a BS2017SS fit is bit-identical). A call that passes an argument — even at
its default — overrides the slot, so omit the argument to let the configuration
take effect. The slot is code-side structure, not a workbook sheet, so it does
not persist through a write_data()/read_data() round-trip (a warning fires).
Spec-tree print() / summary() for data objects and fits. A
build_data() object now carries the class "Rceattle_data" and prints as an
indented specification tree — dimensions → fleets (with their selectivity /
catchability forms and mirroring) → configured processes → active linkages →
any attached model_config() — instead of dumping the ~40-element list. The
same tree is shown by print() on a fitted model above its fit statistics, so
"read 600 lines of switch tables" becomes "read the printout". The class is a
thin tag: every consumer still treats the object as a plain list, so it does
not change a fit.
Random-walk linkages (rw(1 | group)). A deviation that follows a random
walk — N(0, sigma) on successive first differences, the first deviate
pinned so the walk's level stays with the base parameter — reproducing the
Dorn-style Time_varying_* = "RandomWalk" process through the grammar. The
grouping variable must be numeric (a real elapsed-time lag).
AR1 linkages (ar1(1 | group)). A stationary first-order autoregressive
deviation, SCALE(AR1(rho), sigma) with sigma the marginal SD and rho
the correlation (the glmmTMB convention, and the same form as the Rogers et
al. (2024) QAR1 catchability process). The correlation is routed through the
same grammar as sigma: init = list(rho = 0.7) fixes it,
priors = list(rho = normal(0, 0.3)) places a prior on it, else it is
estimated free. Reduces to the IID density at rho = 0.
State-space environmental covariate (Rogers et al. 2024 QAR1). An
ar1(1 | Year) term can be turned into a measured latent covariate via
observe / obs_sd: the AR1 latent is observed as an env_data column with
a fixed measurement SD, and enters the linked parameter through an estimated
effect size. This reproduces the Rogers-2024 QAR1 catchability model
(Estimate_q = 6) through the grammar:
build_catchability(linkages = list(
q = linkage_spec(~ ar1(1 | Year), by = ~ fleet, fleet = 1,
observe = "QcovPol", obs_sd = 0.1)))
The AR1 process, the covariate observation, and the effect size are all
carried by the one term; observe requires an ar1 term and a positive
fixed obs_sd.
Priors on Dirichlet-multinomial data weights (build_composition()). The
DM likelihood self-tunes each composition dataset's effective sample size, but
that weight can be poorly identified when a fleet has few comp years. A new
build_composition() linkage attaches a prior to the DM weight, keeping the
implied effective sample size in a believable range without reverting to
Francis / McAllister–Ianelli hand-tuning:
fit_mod(...,
compFun = build_composition(linkages = list(
theta_comp = linkage_spec(~ 1, by = ~ fleet, fleet = c(1, 2),
priors = list(`(Intercept)` = gamma(2, 0.5))))))
Keys theta_comp / theta_caal / theta_diet cover age/length composition,
conditional age-at-length, and predator diet. The process is prior-only (the
weight is a scalar, not year-varying): a covariate, an init, or an
est_phase on a composition spec, or a linkage on a fleet/predator not fit
with the DirichletMultinomial likelihood, errors up front.
Formula-linkage effect sizes are now reported. The estimated linkage
coefficients (beta_linkage), the random-effect deviations (beta_linkage_re),
and the Rogers-2024 QAR1 effect size (beta_linkage_obs) appear in
fit$quantities, and the coefficients / effect size are ADREPORT'd so they
carry a standard error in the sdreport. Previously the QAR1 effect size — the
quantity the model exists to estimate — was buried in the raw parameter vector
with no readable exposure or uncertainty.
QAR1 covariates need not span the whole series. A state-space observe
covariate (e.g. a survey index that began mid-series) is now handled without
hand-padding: env_data is auto-extended to start at styr with NA for
missing years, the latent AR1 spans all hindcast years, and the observation is
applied only where the covariate is present (un-observed years are masked
out — no fabricated observations, unlike the legacy mean-fill). Fixed-effect
covariates must still be finite over the model range: a missing year in a
fixed-effect (non-observe) covariate now errors clearly rather than silently
producing an NaN.
QAR1 observation SD can be fixed or estimated. The Rogers-2024 observe /
obs_sd state-space covariate holds the measurement SD fixed by default;
pass linkage_spec(..., obs_sd_est = TRUE) to estimate it (one per observed
group, started from obs_sd), as the reference Estimate_q = 6 / GOApollock
model does. Estimation is opt-in because a freely-estimated obs_sd collapses
toward 0 on a smooth covariate (the effect-size / obs_sd identifiability
degeneracy) — keep it fixed unless the covariate is informative (a prior on
obs_sd to regularise this is future work).
Selectivity penalties can be given as standard deviations. The cryptic
penalty weights in Sel_curve_pen1/2/3 can instead be supplied as SDs via
Sel_shape_sd (+ Sel_shape_dir "Decreasing"/"Increasing" for the
directional sign), Sel_curvature_sd, and Sel_devmag_sd. Each penalty is a
Gaussian SSQ, so switch_check() converts weight = 1/(2*sd^2). Legacy
Sel_curve_pen values are never overwritten (existing models are bit-identical);
a fleet supplying the SD columns fits equivalently. Sel_shape_sd /
Sel_devmag_sd apply to NonParametric (2/9) and LogisticPM (11);
Sel_curvature_sd is NonParametric-only (LogisticPM does not use
Sel_curve_pen2). Setting an SD column on a form that doesn't use that slot as a
weight (e.g. 2D/3D-AR1, which reuse Sel_curve_pen for logit-scale correlations),
or a non-positive SD, errors clearly.
Readable string aliases for integer-coded switches. Following the CIE
review's "rename options to improve interpretability" (e.g. constant → not
estimated), integer-only switches now also accept self-explanatory strings,
resolved by switch_check() (or build_srr()) to the same integer codes (so
fits are identical). New aliases follow a consistent convention — the
not-estimated value is "Fixed", estimated is "Estimated":
estDynamics: "Estimated" (0) / "Fixed" (1) / "FixedScaled" (2) /
"FixedScaledByAge" (3);Estimate_index_sd / Estimate_catch_sd: "Fixed" (0) / "Estimated" (1) /
"Analytical" (2);srr_est_mode (via build_srr()): "Fixed" (0) / "Estimated" (1) /
"LognormalPrior" (2) / "BetaPrior" (3);suitMode: the documented string map ("Empirical", "GammaWeight", …) is
now actually applied (previously defined but never wired).M1_re = 6 (separable age × year 2D-AR1 on M) now estimates its correlations.
A gate bug in build_map() (if(M1_re_model == 5) inside a block reachable only
when the mode is 3 or 6) plus a reference to an undefined index left mode 6's age
and year AR1 correlations unmapped, so the separable AR1 silently collapsed to IID
(identical to M1_re = 3). Mode 6 now frees both rho hyperparameters, so it is a
genuine separable 2D-AR1. This changes results only for models using M1_re = 6
(none of the bundled examples do); golden references are unaffected.
The deprecated est_M1 name is now recognized everywhere M1_model is.
The natural-mortality estimation switch was renamed est_M1 → M1_model, but
the data dictionary still documented est_M1 and no alias existed, so a data
list carrying est_M1 was silently ignored — it fell through to the M1_model
default with no warning. est_M1 is now folded into M1_model in fit_mod()
(before the build_M1() reconciliation), switch_check(), and combine_data(),
with a deprecation message. As with M1_model, the value set on the data list
is a default that an explicit build_M1(M1_model = ...) argument overrides — but
a data-list value that differs from the build_M1() setting now warns rather
than being dropped silently. The recommended way to request M1 estimation remains
fit_mod(..., M1Fun = build_M1(M1_model = ...)). The dictionary now documents
M1_model.
A catchability linkage on a non-estimated q now errors. A q linkage
(environmental or random-effect) on a fleet whose Catchability is "Fixed"
or an analytical form ("Analytical" / "AnalyticalArith") is rejected up
front, naming the fleet. Previously only the analytical forms were caught, so
a linkage on a "Fixed" fleet silently turned a fixed catchability
time-varying — contrary to the assessor's setting. Set Catchability to
"Estimated" / "Estimated-with-prior" to link q.
This is the same statistical model as the existing Time_varying_q/_sel
deviate processes, expressed through the linkage grammar.
Selectivity_block is now optional; Q_block is deprecated and ignored. The
per-observation time-block columns in index_data/catch_data are read only for
Time_varying_{sel,q} = "Block"; every other configuration (including the
random-effect linkages) ignores them. clean_data() now default-fills a missing
Selectivity_block with 1 (a single block), so you need only supply it for
Block-mode fleets. Q_block was never read (q time-blocking reuses
Selectivity_block) — supplying it now warns, and it has been dropped from all
bundled example datasets (as has the unused Selectivity_block). Existing models
are bit-identical.
Time_varying_q, Time_varying_sel, and M1_re are soft-deprecated in favor
of random-effect linkages. fit_mod() now warns (naming the fleets/species and
the grammar equivalent) when a model uses these legacy time-variation switches,
pointing at build_catchability() / build_selectivity() / build_M1() with
(1 | Year) / rw(1 | Year) / ar1(1 | Year) — which additionally allow a prior
on, or free estimation of, the deviation SD. The legacy switches still fit with
their exact numerics (they keep their own C++ path); this is only a nudge. The
warning is not raised where no grammar equivalent exists yet: the environmental /
Rogers-AR1 catchability modes (which overload Time_varying_q to name env
columns), the non-parametric selectivity forms, and the separable M1_re = 6
(age × year).
fleet_control columns Q_prior → Q_init, Index_sd_prior → Index_sd,
Catch_sd_prior → Catch_sd. These are start/input values, not priors (the
prior on q lives in Q_sd_prior), so the names misled. As with the other
renames, the old names are accepted and upgraded in place by switch_check() /
read_data(), and the bundled datasets were regenerated — existing scripts keep
fitting identically.
fleet_control columns Time_varying_q_sd_prior / Time_varying_sel_sd_prior
renamed to Time_varying_q_sd / Time_varying_sel_sd. They hold the input
value of the time-varying deviate SD, not a prior on it (no density is placed
on the SD), so the _prior suffix was misleading. The old names are still
accepted — switch_check() and read_data() upgrade them in place with a
one-time message — so existing data lists, saved xlsx files, and the bundled
example datasets keep fitting identically. Update scripts to the new names at
your convenience.
Environmental linkages on catchability and selectivity. The
formula-driven linkage grammar (linkage_spec()) now extends to survey
catchability and to the parametric selectivity forms, both indexed by a new
fleet stratum (by = ~ fleet):
fit_mod(...,
qFun = build_catchability(linkages = list(
q = linkage_spec(~ temp, by = ~ fleet))),
selFun = build_selectivity(linkages = list(
inf_asc = linkage_spec(~ cold_pool, by = ~ fleet))))
Selectivity keys are the shared parameter slots slp_asc/slp_desc/
inf_asc/inf_desc, with DoubleNormal aliases sigma_asc/sigma_desc/
peak/right_floor. Supported forms: Logistic, DoubleLogistic,
DescendingLogistic, DoubleNormal, LogisticPM. Every parameter accepts
link = "log" (multiplicative) or link = "identity" (additive). A linkage
on an unsupported form or the non-parametric coff errors at fit time
naming the fleet, rather than being estimated to no effect.
build_catchability() and build_selectivity() exported, mirroring
build_growth() / build_M1() / build_srr(), each carrying a linkages
argument for its process.
Linkage formulas are now parsed with the reformulas package (shared by
lme4 and glmmTMB), so (1 | Year) / ar1(Year + 0 | fleet) syntax is
recognized and an unknown covariance-structure wrapper errors instead of
silently degrading to unstructured.
Catchability = "Environmental" (Estimate_q = 5) is deprecated. It
still fits with its existing numerics but emits a warning pointing at
build_catchability(linkages = ...), which names covariates by formula and
carries priors, bounds, and an estimation phase.fleet_control$Sel_avgsel_pen).
Non-parametric (type 9 / NonParametricPM) fleets can now carry the AMAK/ebswp
base-level regulariser weight * (log(mean(exp(base coefficients))))^2 — ADMB's
10 * square(avgsel_*) — which mildly pins the overall level of the base
selectivity coefficients that the per-year mean-centering otherwise leaves free.
The per-fleet weight defaults to 0 (off), so existing models and the BS2017SS
golden reference are unchanged; set Sel_avgsel_pen = 10 to match AMAK. The
penalty is accumulated once per shared-selectivity block (on the lead fleet).Non-parametric (AMAK "pm", type 9 / NonParametricPM) selectivity deviate
penalties. Two corrections so the type-9 selectivity and its deviate penalty
reproduce ADMB/AMAK exactly when the fleet excludes a first bin (e.g. the
acoustic-survey age-1) and/or starts after the model start year:
Bin_first_selected) are now held at 0 before each
year's mean-centering instead of being carried through the random walk.
Previously their log-selectivity accumulated the per-year centering offset
and drifted, inflating the curvature / random-walk penalty on a bin that is
zeroed out of the fit anyway.Sel_start_year, the curve is now rebuilt from the
base coefficients each year rather than iterating the running random walk
over the (data-free) pre-survey years. Iterating instead converged the
excluded first bin to a different fixed point and perturbed the start-year
base selectivity, inflating the deviate penalty on the first change year.
A fleet that starts at styr with no excluded bins is unaffected (the reset
reduces to the original start-year behaviour); the BS2017SS golden reference is
unchanged.hessian_conditioning diagnostic now always names the flat direction. When
the Hessian's least-identified direction was spread diffusely over many
coefficients (no single coefficient above the reporting threshold), the message
read loads on: . with nothing after it. The check now aggregates the
eigenvector's squared loadings by parameter block and reports the block(s) making
up the direction with their percentage share (e.g. loads on: rec_dev (69%) + ln_srv_sel (31%)), falling back to par.fixed names when cov.fixed carries no
dimnames.
Mohn's rho was computed from the wrong row for forecast years beyond the
terminal year. retrospective() accumulated relative error into
mohns[ind, ] while reading from mohns[j, ]; the two indices coincide only
at forecast year 0, so retrospective bias was wrong for every additional
forecast year. The observation counter on the adjacent line was correctly
indexed, which masked the problem.
Projection-year conditional age-at-length was missing from every MSE.
run_mse() built proj_caal and then appended a different object
(proj_comp) to a non-existent proj_caal field instead of appending the
CAAL rows to caal_data. When the composition branch had not run,
proj_comp did not exist at all.
sim_mod() reproduced the previous observation's draw instead of
simulating. The CAAL branch tested Comp_loglike (rather than
CAAL_loglike) against integer codes, and neither the composition nor the
CAAL branch handled "MultinomialAFSC" — the default Comp_loglike. In both
cases no branch fired, leaving the simulation variable holding its value from
the previous row. Unrecognised likelihood codes now raise an error rather
than falling through.
sample_rec() and retrospective forecasts produced NaN recruitment
deviations. Both took log() of the mean of log(R) - log(R_hat), a
log-scale quantity centred near zero whose mean is routinely negative. The
trend adjustment is now applied additively on the log scale, matching the
sibling sampling branch.
combine_data() discarded merged environmental data (it assigned the
merge to its input rather than to the returned object) and appended two
NA-named elements by iterating past the end of its column-name vector.
read_data() overwrote a user-supplied fleet_control$Month with zero.
The guard was inverted relative to its own message, so per-fleet survey
months set in the workbook were silently reset.
build_hcr_map() disabled SPR reference points too eagerly. It turned off
Ftarget/Flimit when any fishery for a species had zero proj_F_prop,
though its comment and message both describe the all-zero case.
set_phases() declared log_M1 twice, misaligning the positional
phase-to-parameter pairing in TMBphase() for every subsequent parameter.
fit_mod() could fail with object 'opt' not found for
estimateMode = 2 combined with HCR = "NoFishing".
Time_varying_q environmental-index lists were destroyed. For fleets with
Catchability = "Environmental", a comma-separated list of env_data column
indices (e.g. "1,3") was coerced with as.integer() and became NA, both in
fleet_control itself and in the index_varying_q vector passed to TMB.
process_residuals() tests that vector against a mode code, so the NA
silently poisoned a branch rather than failing.
Natural-mortality environmental linkages used the wrong coefficients.
Assigning a scalar to the vector beta_M1_tmp broadcast it, so the dot
product collapsed to beta_last * sum(env) rather than sum_j env_j * beta_j.
Dormant in practice because M1_beta is mapped out by default.
Observation years earlier than styr produced a large negative array
index. Two consecutive if statements (rather than if/else if) applied
both year transformations in sequence. Five sites.
Selectivity max-normalisation branched on an AD type, freezing the
location of the maximum at its initial-parameter position for the whole
optimisation. If the peak moved during fitting, the curve was normalised by
the wrong bin and the gradient with respect to the true peak was absent. Now
folded with max2(), matching the neighbouring single-bin normalisation.
plot_form() errored on every call. It plots the Kinzey & Punt (2009)
functional responses, whose parameters (logH_1, logH_1a, logH_1b,
logH_2, logH_3, H_4) are commented out in the TMB template,
build_params(), build_map() and build_bounds(), and whose modes
(msmMode 3-9) data_check() blocks. params$logH_1 was therefore always
NULL and the first line failed with "non-numeric argument to mathematical
function". The function remains exported but now raises a clear message
naming the supported alternatives; its body is retained, commented, so it can
be revived with the C++ if these formulations are validated.
Predation suitability could become infinite. The zero-guard tested
diet_prop_sum, but the division is by suma_suit + other_food_diet_prop,
which can reach zero or go negative independently.
estimateMode = 3 now returns the real objective and gradient. It
previously shared mode 4's placeholder objective (dummy^2), so obj$fn()
returned zero and every gradient element was identically zero — a build-only
object appeared healthy while carrying no information. Mode 3 is the
build-without-optimizing entry point (the analogue of WHAM's
fit_wham(do.fit = FALSE) and SAM's sam.fit(run = FALSE)) and can now be
used to inspect a model before committing to a fit. estimateMode = 4 is
unchanged and still returns the placeholder: it maps out every hindcast
parameter, so it is a plumbing smoke test rather than a likelihood.Observed-proportion columns are now found by name rather than by counting
from a fixed position. comp_data and caal_data begin with identifying
columns (fleet, species, sex, year, sample size) followed by the observed
proportion at each age or length bin. sim_mod() located those proportions
by a fixed offset ([, 9:ncol(x)], [, 7:ncol(x)]), so adding or reordering
an identifying column would have written simulated values into the wrong
columns without any error. Both tables now resolve the proportion columns by
name, as rearrange_data() already did.
Added an internal parameter dictionary (R/0-parameter_dictionary.R)
mapping each of the 43 TMB parameters to its natural-scale name, its process
group, a one-sentence meaning, and its dimensions — so diagnostics and error
messages can describe log_sel_slp_dev or index_q_rho in terms a reader
can act on. A test enforces that the dictionary and the parameter list stay
in exact correspondence.
The non-parametric selectivity curvature penalty is computed in one pass rather than recomputing the full second difference for each age (O(n) instead of O(n^2)). Numerically identical.
fleet_control$Index_loglike
gains "Normal": the index residual obs - q*pred is normal with an absolute
observation SD (the index_data$Log_sd column is read as a natural-scale SD, not
a log-scale CV), i.e. 0.5*(obs - q*pred)^2 / sd^2. This matches the AMAK/ebswp
avo_like / cpue_like survey likelihoods (which use an absolute sigma) so those
indices can be reproduced exactly, rather than approximated by the default
lognormal. Fixed alongside a latent crash: the covariance (MVN/MVNORM) block was
gated on Index_loglike >= 1, which now also matched "Normal" and applied
MVNORM() to a 1x1 dummy Sigma; it is now restricted to "MVN" / "MVNORM".fleet_control column
Index_loglike ("Lognormal", the default, or "MVN" / "MVNORM"). Supply
the covariance matrix (e.g. a VAST-derived Sigma) as a named element of the new
index_cov data list, keyed by Fleet_name. The likelihood uses TMB's native
density::MVNORM(Sigma) on the natural-scale residual r = obs - q*pred:
"MVN" reports the bare quadratic form 0.5 * r' Sigma^-1 r (the AMAK/ebswp
DoCovBTS bottom-trawl survey value, matching ADMB's reported likelihood),
while "MVNORM" reports the full normalized density
0.5 * (r' Sigma^-1 r + logdet(Sigma) + n*log(2*pi)) — the two give an
identical fit and differ only by a fixed constant. A companion catchability
option Catchability = "AnalyticalArith" gives the arithmetic-mean analytical q
(mean(obs)/mean(pred)) that the AMAK covariance survey uses (the existing
"Analytical" q remains the geometric mean). Defaults are fully back-compatible:
existing models get Index_loglike = "Lognormal" and are numerically unchanged.fleet_control$Selectivity options reproduce the ADMB AMAK ("pm") selectivity:
NonParametricPM (Ianelli coefficient selectivity with the decreasing,
curvature, and deviation-magnitude penalties, whose weights are set by
Sel_curve_pen1 / Sel_curve_pen2 / Sel_curve_pen3) and LogisticPM
(logistic + a free age-1 selectivity). Both take Time_varying_sel = "RandomWalk". Four new per-fleet fleet_control columns tune the shape
penalty: Sel_pen_first_bin and Sel_pen_last_bin (the bin range of the
adjacent-bin shape penalty, letting it span a narrower range than the first
selected bin), Sel_shape_mode ("Directional" one-sided decreasing/increasing,
the AMAK default, or "Smooth" two-sided curvature), and Sel_cap_bin
(hold the realized non-parametric curve flat at/after a bin). Each is given on
the fleet's own selectivity dimension — an age for age-based fleets, a 1-based
length-bin ordinal for length-based fleets — and is range-checked accordingly.
All default to the previous behavior when unset.plot_*() function now builds its figure with ggplot2 (colourblind-safe
palettes — the Okabe-Ito qualitative palette for series identity and viridis
for ordered magnitude such as year — and theme_classic) and returns the
ggplot object, so
plots print when called interactively and can be further customised
(plot_biomass(fit) + ggplot2::ggtitle(...)) or saved with ggplot2::ggsave().
Pass file = "stem" to also write the figure to stem_<plot>.png. Plot
conventions follow r4ss / WHAM / SAM (line + 95% ribbon time series; observed
points + error bars + predicted line for index/catch fits; year-coloured
at-age curves for selectivity and mortality surfaces).plot_index() gains a log argument for the log-scale survey-index fit.plot_biomass(), plot_ssb(), plot_recruitment(),
and the other plot_timeseries() wrappers) again honour user-supplied
line_col, lwd, and lty: pass a colour, line width, and/or line type per
model to override the defaults
(e.g. plot_biomass(list(m1, m2), line_col = c("black", "red"), lty = c(1, 2))).
lwd keeps the base-graphics convention where the default (3) renders as a
standard-weight line.plot_stock_recruit() adds a 95% data ellipse of the SSB–recruitment cloud
(add_ci, default TRUE).tests/testthat/ (see
the new tests/testthat/README.md), runs fully under continuous integration,
and has automated code-coverage reporting.plot_*() functions now return a ggplot object instead of drawing as a
base-graphics side effect (returning NULL). Scripts that only called them
for the side effect still work (the object prints); scripts that depended on
the base-graphics device state or on a NULL return may need updating.plot_logindex() has been removed; use plot_index(..., log = TRUE).gplots and oce dependencies have been dropped (no longer used)."RandomWalk", not
"IID". The Ianelli non-parametric form (Selectivity = "NonParametric")
previously fired its time-varying coefficient deviations on Time_varying_sel = "IID"; it now requires "RandomWalk" (matching the random-walk structure the
penalty implements) and rejects "IID" with an error at build_map(). A model
using NonParametric + IID must switch to RandomWalk.Selectivity_index and selectivity type (a mirrored curve),
the selectivity shape/deviation penalty is accumulated only on the lead fleet
rather than once per fleet, matching ADMB. This changes the objective only for
models that both mirror a selectivity and put a penalty on it (non-parametric
or time-varying); models with a unique selectivity index per fleet, and all
bundled examples, are numerically unaffected.Catchability = "AnalyticalArith" left an unused free catchability parameter,
making the Hessian singular. The arithmetic-mean analytical q solves q from the
data (like the geometric "Analytical"), so its index_log_q is never used —
but build_map() excluded only "Analytical" from estimation, so the
AnalyticalArith fleet's index_log_q was still freed. That parameter never
entered the objective, leaving a zero-gradient flat direction that prevented
sdreport() from inverting the Hessian (pdHess = FALSE). It is now mapped
out, and such models converge with an invertible Hessian.Q_index, not once per estimated parameter. Fleets sharing a
Q_index estimate one catchability and one deviate vector, but the model
looped over every fleet, so a mirrored pair applied the Q_prior twice to the
same parameter — tightening an intended prior SD of 0.2 to 0.2/sqrt(2) — and
penalized the shared index_q_dev vector once per fleet for the IID, random
walk and AR1 forms. A new flt_q_lead (the catchability analogue of
flt_sel_lead) accumulates them on one fleet per group. Models whose fleets
all have distinct Q_index are unchanged.Catchability = "PowerEquation" is now rejected as not yet implemented. It
was accepted as a valid switch, but the power coefficient (index_q_pow) is
not built as a parameter and the model does not apply it, so the fleet
silently got a plain estimated q. data_check() now errors, matching how
length-based suitMode values are handled.flt_sel_lead could put the selectivity penalty on an Off fleet. The
lead was the first fleet in a Selectivity_index group by row order. When that
fleet was Fleet_type = "Off" the model's flt_type > 0 gate then skipped
the penalty for the whole group, leaving the shared selectivity unpenalized.
The lead is now the first estimated fleet in the group, matching the map
donor.Off fleet stopped estimating selectivity and
catchability. adjust_map_shared_params() copied the first sharing fleet's
map slice onto the rest. When that fleet was Fleet_type = "Off" its slice is
all NA, so every fleet sharing the index silently had its selectivity /
catchability parameters fixed at their starting values. The donor is now the
first estimated fleet in the group, and the copy is skipped when the group
has none. Groups led by an estimated fleet are unchanged.Sel_start_year was not shared across mirrored fleets,
making the fit depend on fleet_control row order. The default derived from
the data is already the earliest first-observation year across each
Selectivity_index group, but a value the user sets directly was used
per fleet. Since fleets sharing an index share one deviation block,
adjust_map_shared_params() then overwrote the mirrored fleet's mask with the
lead fleet's, so whichever fleet appeared first governed the group: when that
fleet started later, a sharing fleet with earlier data silently lost those
deviations (12 years in a 1982/1994 pair). Sel_start_year now resolves to the
group minimum for both the map mask and the model's penalty anchor, however
it was set. data_check() warns when a mirrored group has differing
Sel_start_year, and build_map() warns when Bin_first_selected or
N_sel_bins differ within a group (those are likewise taken from the lead
fleet). Unmirrored fleets and derived defaults are unchanged.fleet_control$Fleet_code is now required to equal the row number. It is
used directly as the fleet slot of the per-fleet parameter and map arrays,
which are built in fleet_control row order; a mismatch silently attached
parameters to the wrong fleet. data_check() now rejects it, and the
remaining places that read a fleet_control column by Fleet_code instead of
row index were corrected.minage, which is wrong for length-based fleets. Sel_norm_bin1,
Sel_norm_bin2, and the shape-penalty range/cap columns subtracted
minage to get the 0-based template index. That is correct for an age-based
fleet (the value is an age), but a length-based fleet's value is a 1-based
length-bin ordinal and must be offset by 1 — so those columns silently pointed
at the wrong length bin whenever minage != 1. The offset is now chosen per
fleet from Selectivity_dimension. Age-based fits, and length-based fits with
minage == 1 (where the two offsets coincide), are unchanged.NonParametric (type 2), NonParametricPM (type 9), and LogisticPM
(type 11) forms were hard-coded to the number of ages, so a length-based fleet
(Selectivity_dimension = "Length") with more length bins than ages left the
upper length bins unpenalized (and read the wrong array). The penalties now run
over the fleet's own selectivity dimension (nlengths for length-based,
nages for age-based), so every parametric and non-parametric selectivity form
works on both age and length. Age-based fits are numerically unchanged.index_cov (MVN/MVNORM) Sigma is positionally keyed to a
fleet's fitted survey observations, so any workflow that changes that set —
a retrospective() peel, an endyr / styr subset, or a run_mse()
assessment step that appends survey observations — left the Sigma at its
original dimension and tripped rearrange_data()'s dimension check
("N x N but the fleet has M fitted survey observations"). clean_data() now
tags each Sigma with its fitted years the first time it is seen and, on every
subsequent pass, re-keys it to the current fitted set: retained years keep
their full covariance block, and new (future/simulated) years are added as an
independent diagonal block with variance (Observation * Log_sd)^2. Because
every re-fit routes through clean_data(), retrospective, MSE, and jitter now
all work with covariance-survey models; fresh fits and non-MVN fleets are
numerically unchanged.build_map() never consulted fleet_control$Sel_start_year, so a fleet
with time-varying ("RandomWalk") selectivity had deviations estimated across
every hindcast year — including years before its first observation. Those
deviations are informed by no data and constrained by no penalty (every
selectivity penalty in the objective is anchored at Sel_start_year), leaving
unidentified flat directions that stall the optimizer. In the EBS pollock model
this left ~54 free pre-survey deviations on a bottom-trawl survey starting in
1982 and ~240 on an acoustic survey starting in 1994 — a total parameter count
of 1483 against 1225 for the equivalent ADMB (AMAK) model, which never declares
them. Deviations before Sel_start_year are now fixed at 0, giving 1249. The
two selectivity parameterizations differ in where the base curve lives and are
handled accordingly: LogisticPM (and other curve-based forms) estimate a
separate base, so deviations are fixed through the start year; the
non-parametric random walk maps its mean (sel_coff) off and lets the start-year
deviation carry the base, so only the deviations strictly before it are fixed.Sel_start_year now defaults to the fleet's first year of data rather than
styr. It is an optional fleet_control column, so the fix above only took
effect for users who knew to set it — a model with a late-starting survey would
silently carry unidentified deviations. The default is derived from
catch_data / index_data / comp_data / caal_data, consistent with how
switch_check() already auto-"Off"s fleets with no observations. Fleets
sharing a Selectivity_index share one selectivity curve, so the start year is
the earliest first-observation year across the whole group: a fleet whose own
data starts late but which mirrors an earlier fleet's curve (e.g. an AVO index
starting in 2006 mirroring an acoustic survey starting in 1994) must not drop
the deviations the mirrored fleet's data informs. Set the column explicitly to
override. Only models with time-varying selectivity on a fleet whose data starts
after styr are affected; for those, the previous behaviour is recovered by
setting Sel_start_year = styr.LogisticPM selectivity started with an unusable age-1 selectivity.
build_params() initializes sel_inf[2] to 10, which is correct for its
usual meaning (the descending-limb inflection age), but LogisticPM
(type 11) repurposes that slot as the free first-bin (age-1) log-selectivity.
The shared default therefore started age-1 selectivity at exp(10) = 22026
(for reference, the ADMB AMAK "pm" estimate is sel_age_one_bts = -3.19, i.e.
0.04). When age-1 was selected (Bin_first_selected = 1) this swamped the
survey-index prediction and the optimizer diverged with a gradient blow-up on
sel_inf; when it was not selected the bad value was silently masked by the
zeroed first bin. sel_inf[2] now defaults to 0 (age-1 selectivity = 1) for
LogisticPM fleets.plot_maturity() read a non-existent pmature field and errored on real
fits; it now reads data_list$maturity.plot_ration() failed for single-sex models (a dropped array dimension); the
sex dimension is now indexed explicitly.plot_stock_recruit() drew the mean-recruitment reference line a factor of
1e6 too high (recruitment points are in millions but the line was not scaled),
which under ggplot's free y-scale collapsed the SSB–recruitment cloud to the
axis; the line is now scaled to match the points.plot_index(), plot_catch(), and plot_indexresidual() drew
prediction-only rows (Year < 0, excluded from the likelihood) as if they
were fitted observations; these rows are now omitted.plot_mortality(M2 = TRUE) labelled the y-axis "M1 + M2" while plotting M2;
the label now reads "M2".osa_residuals() now includes fleet in the one-step-ahead conditioning order
(source, year, fleet, then bin), making the sequence fully deterministic
when several fleets report in the same year. Previously the within-year,
multi-fleet order fell to the incidental row order of the observation table.
This does not change results for fixed-effects fits (random_rec = FALSE),
where OSA residuals are order-invariant, and ascending fleet order matches
WHAM's within-stage sequencing so the WHAM cross-check is unaffected; for
random-effects fits it makes individual residuals reproducible (the overall
N(0,1) properties were already order-invariant).osa_residuals() now builds the composition / conditional-age-at-length /
diet one-step-ahead observation data on demand, so OSA residuals can be
computed from any fit. Previously the composition OSA data had to be assembled
during the fit via fit_control(osa = TRUE); that pre-build is no longer
required (the reconstruction reads only the *_ctl / *_obs arrays already
carried by the fitted object, and is identical to an osa = TRUE fit).
Aggregate index/catch OSA residuals are unaffected.
Breaking: the osa argument to fit_control() has been removed; it is no
longer needed (see above), and passing it now raises an "unused argument"
error. The $osa element is no longer stored on fitted Rceattle objects.
fit_control() gains bias_adjust_obs and bias_adjust_proc (both default
TRUE) to toggle the lognormal bias correction (-sigma^2/2) applied to the
observation (index / catch) and process (recruitment, initial abundance, and
the Beverton-Holt steepness prior) likelihoods, respectively. The defaults give
the standard mean-unbiased behaviour (E[R] = R0); set a flag to FALSE to
drop the correction for that likelihood group. The values are passed to the TMB
template as DATA_SCALARs.
Added one-step-ahead (OSA) residuals for model validation via the new
exported osa_residuals() (Thygesen et al. 2017; Trijoulet et al. 2023).
Unlike Pearson residuals, OSA residuals are iid standard normal under a
correctly specified model even for correlated composition data. All fitted
observation types are supported: survey indices, fishery catch, age/length
composition, conditional age-at-length, and predator diet (stomach-content)
composition. Composition data are residualized with the conditional binomial
/ beta-binomial decomposition (multinomial / Dirichlet-multinomial), and the
MultinomialAFSC likelihood is residualized under the full multinomial. The
fitted objective is unchanged: a new obsvec/keep/osa_mode machinery
feeds TMB::oneStepPredict() while leaving normal fitting bit-for-bit
identical. The oneStepPredict() call is split by observation type so the
continuous (lognormal) index/catch series and the composition series can be
residualized with the correct settings; discrete = TRUE treats composition
as discrete (randomized quantile residuals; Dunn and Smyth 1996) while the
aggregate series stay continuous.
fit_control() gains comp_offset, the small proportion offset added to the
observed and predicted age/length composition, conditional-age-at-length, and
predator diet (stomach-content) bins before the multinomial /
Dirichlet-multinomial likelihood (and to the matching OSA observation vector,
so fitting and OSA residuals use the same offset). It applies to every
composition likelihood, including the "Martin's" (comp_ll_type = -1) form. It
defaults to 1e-5 (the historical CEATTLE value, which avoids log(0) for
empty bins); set comp_offset = 0 for a standard WHAM-style multinomial. The
value is stored on data_list$comp_offset (filled by switch_check()), so
internal re-fits (projections, retrospective(), jitter(), run_mse())
inherit it; fit_control(comp_offset = ...) overrides the stored value.
osa_residuals() gains a parallel argument (default TRUE) that computes
the per-observation one-step-ahead loop with parallel::mclapply() -- a
near-linear speedup for models with random effects, where each observation
triggers a Laplace re-evaluation (set options(mc.cores = ) to choose cores;
serial fallback on Windows). The discrete randomized-quantile path stays serial
so it remains reproducible given seed.
Added osa_diagnostics() for the Stewart and Monnahan (2025) statistical
diagnostics -- the standard deviation of the normalized residuals (SDNR) and
the lower/upper tail statistics, each with its standard-normal null interval
-- and a plot() method for rceattle_osa objects, styled after the
NOAA-AFSC afscOSA package. The plot draws up to two figures: an aggregate
Q-Q figure for the index/catch series (which have no age/length bin, so no
bubbles), and a composition figure pairing the Q-Q plot with signed
OSA-residual and Pearson-residual bubble plots, with age-based bins (age
composition and conditional age-at-length) in the left column and length-based
bins in the right column, each on its own bin axis. Panel headers use the
fleet name from fleet_control. The plot() method takes source and
species arguments to subset the data shown (mirroring residuals()), and
combine = FALSE to draw the age and length composition as separate figures.
osa_residuals() now also carries a fleet_name column and (for composition)
the matching Pearson residuals.
Added process_residuals() for SAM-style process residuals on the model's
random-effect deviations (recruitment, initial abundance, and catchability),
validating the process model as a complement to the observation residuals.
residuals() gains type = "osa" and type = "process"; plot_comp() and
plot_indexresidual() gain a residual_type = "osa" option that draws the
OSA diagnostics through the familiar plotting functions.
plot_comp() was re-implemented in ggplot2 for a consistent look with the OSA
plots: composition Pearson-residual bubbles plus observed-vs-fitted annual and
aggregated composition figures. The observed area and fitted line now span only
the observed bins (they no longer extend past the first / last bin), zero
observed proportions are retained (only NA is dropped), joint-sex (Sex == 3)
data are drawn on a single age/length axis with females above and males below
zero, and a species argument subsets the species shown.
unweighted_jnll_comp instead of the
CAAL slot. This affects the reported diagnostic matrix only; the fitted
objective was unaffected.projection_uncertainty moved from fit_mod() into fit_control(),
consolidating it with the other optimizer / reporting controls. Passing it
directly to fit_mod() still works but emits a deprecation warning and
forwards the value into fit_control() -- the same backward-compatible path
used by the other former fit_mod() control arguments (phase, getsd,
...).residuals() now follows the stats::residuals.glm() convention: type
selects the residual kind -- "response" (default), "pearson", "osa",
or "process" -- and a source argument selects the data source(s)
("index", "catch", "comp", "caal", or "all", the default), so by
default residuals for every applicable source are returned. A species
argument subsets to particular species. Pearson residuals are now available
for the aggregate index/catch series (standardized by the realized observation
log-SD) and for predator diet via source = "diet" (returned on its own in a
predator/prey schema); plot_diet_comp() now draws its diet residuals from
this single residuals() path. type selects the residual kind only; data
sources are selected with source.source argument is shared across residuals(), osa_residuals(), and
plot() (it replaces the earlier types argument of osa_residuals()),
accepting "index", "catch", "comp", "caal", "diet", and "all", so
the three entry points select data sources with one consistent vocabulary.estimateMode, suitMode, selectivity, catchability,
harvest-control-rule, M1_model, and bioenergetics (Ceq) codes now agree
across model-options-and-functionality, single-vs-multispecies,
projections-and-reference-points, data-without-excel, and
model-parameterizations.suitMode = 1/3/5) are not
yet enabled; use a weight-based mode (2/4/6) for parametric suitability.diet_data (removing the
legacy UobsWtAge / UobsAgeWt / stom_prop_data names).residuals() examples in the README to the type / source
convention (e.g. residuals(fit, type = "pearson", source = "comp")).vignettes/articles/developer-guide.Rmd) documenting the fit pipeline, the
switch system, the TMB module layout, and recipes for adding parameters, data
inputs, switch options, and likelihood components. Consolidates and updates
the GitHub wiki developer notes.convergence_diagnostics() function. It runs a battery of checks
covering the optimizer gradient, Hessian positive-definiteness and
conditioning, parameters on bounds, phasing, and parameter estimability, and
returns a structured "Rceattle_convergence" object whose status reflects
the worst severity found ("OK", "NOTE", "WARN", or "FAIL").fit_mod() now runs the convergence battery automatically and attaches the
result as fit$convergence; convergence_diagnostics() can also be called
directly to re-run it on any fit.-Warray-bounds false
positive from a -Wno-array-bounds compiler flag to a source-level
#pragma GCC diagnostic ignored "-Warray-bounds", so
R CMD check --as-cran no longer warns about non-portable,
warning-suppressing compilation flags. Real diagnostics still surface.The pre-fit pipeline files in R/ were reorganised so they are easier to
navigate. None of these changes alter model output.
fit_mod() runs its stages
as clean_data() -> data_check() -> build_params() -> build_map() ->
build_bounds() -> rearrange_data() -> fit -> rename_output(), so the
files were renumbered to match (data_check is now 1-, build_params
2-, build_map 3-, build_parameter_bounds 4-). A pipeline map was
added to the top of fit_mod().R/0-switches.R: the
string<->integer maps (formerly 0-constants.R) plus switch_check(),
revert_switches(), validate_switches(), and convert_switches(), with a
header documenting the order in which they run.build_hcr_map() moved into R/0-build_hcr.R
alongside build_hcr() (the separate 2-build_hcr_map.R was removed).rearrange_dat() is renamed rearrange_data(). The old name still works
as a deprecated alias (emits a one-time .Deprecated() warning) and will be
removed in a future release.check_composition_data(), check_caal_data(), calc_mcall_ianelli(), and
calc_mcall_ianelli_diet() are no longer exported; they are internal
helpers called only from within the package.Rceattle::: self-references in build_bounds() (a package should
not use ::: for its own objects).profile.Rceattle() gained ... for S3 consistency with the
stats::profile generic.profile_param() was renamed profile() and turned into an S3 class.
sd_L1 and sd_Linf parameters. This allows users to specify priors, initial values, and bounds for growth SD endpoints using the same formula-driven interface used for mean growth parameters.log_* to their natural-scale counterparts (K / L1 / Linf / m; M1; R0 / alpha / beta). Internal parameters remain on the log scale."(Intercept)" parameters are applied on their natural scale.link = "log" (the new default) applies the offset multiplicatively on the natural scale (additive on the log scale); link = "identity" applies it additively on the natural scale.build_growth() gains a growth_age_L1 argument (scalar or length-nspp vector) for the age at which mean length equals L1. Matches SS3's Growth_Age_for_L1. Default NA inherits data_list$growth_age_L1 if set, else falls back to max(0.5, minage[sp]) so minage = 0 models pick up an SS3-consistent half-year anchor while minage >= 1 models stay backwards-compatible.self_test() simulates nsim datasets from a fitted model and re-fits the model to each simulated dataset, returning the list of refits. Runs in parallel by default (PSOCK cluster, capped at 2 cores under R CMD check) with a per-simulation seed + i so results are reproducible under both sequential and parallel execution.profile_param() generalises the legacy profile_rsigma() example helper to any parameter slot in Rceattle$estimated_params. Supports arbitrary N-D cross-profiles via expand.grid() over a list of per-cell value grids (e.g. cross-profiling log_M1 across sex, or R_log_sd across multiple species). Natural-scale aliases "sigmaR" / "R_sd", "M1", and "R0" / "alpha" / "beta" apply the implicit log() transform and (for the rec_pars aliases) auto-fill the column, so slots only needs the species index. slots defaults to species 1 with a warning. Fits run in parallel on the same PSOCK harness as jitter() / retrospective() / self_test().retrospective() and jitter(): Both diagnostics now run their independent peels / starts on a PSOCK cluster (same approach as run_mse()). New cores argument on each (default parallel::detectCores() - 6, capped at 2 when _R_CHECK_LIMIT_CORES_ is set); pass cores = 1 to force sequential execution.as.data.frame.Rceattle(): The tidy long-format frame now carries a se column alongside value / lwr / upr, populated from the TMB sdreport for any ADREPORT'd quantity. Set to NA for non-ADREPORT'd quantities and for fits produced with getsd = FALSE.build_srr() where the Bmsy_lim penalty was incorrectly disabled for current Ricker implementations due to an index mismatch.last_par returned wrong vector: Fixed fit_mod() so the value stored on the returned fit is the optimizer's last parameter vector rather than a stale prior reference, removing the need for the surrounding try() guards in downstream callers.minage = 0: Fixed a segfault in growth.hpp when minage = 0 by guarding current_age, age_L1, and the cohort-boundary age_L1_ceil against the zero-age anchor. Also corrected length-at-age at minage = 0 so the closed-form anchor at L1 is honored on both the within-year and cohort recursion paths.calculate_weight() now computes each bin's midpoint as (lengths[ln] + lengths[ln+1]) / 2 (with the plus-group extended by half the final interior width) rather than lengths[ln] + (lengths[1] - lengths[0]) / 2. The previous formula assumed uniform bin widths and silently mis-located the midpoint for non-uniform length bins; for uniform bins (the common case) the two are algebraically identical, so existing fits are unchanged.growth.hpp now pin the oldest age class's length_sd to exp(growth_log_sd(sp, sex, 1)) (SD at Linf), matching WHAM (SDAA plus group = SD_len(1)), instead of the length-based interpolation between SD(L1) and SD(Linf) used previously. This makes the plus-group convention consistent across von Bertalanffy and Richards growth and across both builders; expect a small numerical change in length_sd at the oldest age relative to prior versions.fit_mod() bounds ordering: fit_mod() now indexes parameter bounds by name rather than positional order when assembling L / U for nlminb. Previously, when map$mapFactor and bounds$lower were not in identical order, parameters could be paired with another parameter's bounds, producing silently wrong constraints. start_par is now also subset by name with drop = FALSE.mse_summary() Tier-3 Flimit: The internal flimit_tier3_fun() returned Flimit (its argument) instead of the depletion-adjusted tier3_flimit it had just computed, so the Tier-3 (HCR = 5) branch of P(F > Flimit) reduced to the base-Flimit check. Now returns the adjusted vector.mse_summary() HCR coercion: HCR is now normalized to its integer code before downstream comparisons (HCR == 5, etc.). build_hcr() accepts either an integer or a string alias (e.g. "NPFMC"); mse_summary() previously assumed integer form and silently produced wrong status flags when fits carried the string form.mse_summary() OM status at assessment years: P(F > Flimit) and P(SSB < SSBlimit) are now reported for the OM evaluated at the same assessment years as the EM (previously only the EM's perceived status was returned), and the SSB-limit threshold dispatch is consolidated in one helper so the Tier-3 / Category-1 / dynamic-vs-static cases stay aligned across the EM and OM paths.clean_data() inactive-fleet handling: The auto-Off branch no longer nulls out proj_F_prop and Catchability on fleets it flips to "Off". The downstream TMB code already ignores those columns for off fleets, and clearing them lost information users needed when re-enabling a fleet.biom_yrs index vector to block_yrs in the selectivity and catchability block branches of build_map_*(). The previous name was a leftover from the index-data path and shadowed nothing, but read as if it referred to biomass-observation years.plot_index() / plot_catch() / plot_logindex() warnings: Wrapped the internal gplots::plotCI() calls in suppressWarnings() so plotting a fit no longer prints the recurring "In arrows(...): zero-length arrow is of indeterminate angle and so skipped" noise when CI half-widths are zero.data_check() now errors when growth_model == 0 (empirical weight-at-age) is combined with non-empty caal_data for a given species. The C++ growth matrix is not populated from the age-transition matrix in the empirical branch, so pred_CAAL collapses to ~0 and the multinomial NLL becomes uninformative.Selectivity and Fleet_type != "Off" now require at least one positive-sample comp_data or caal_data row. Either provide composition / CAAL data, mark the fleet as Selectivity = "Fixed" with emp_sel, or set Fleet_type = "Off".clean_data() automatically flips Fleet_type to "Off" for fleets that carry no catch or index observations, preventing the optimizer from drifting on unconstrained selectivity / catchability blocks.minage guard: data_check() errors when any species has minage < 0.vignette("environmental-linkages-and-priors") (and updated _pkgdown.yml) to cover the new linkage intercept behavior, link-function semantics, growth SD endpoints, and Double Normal selectivity.build_srr() / build_M1() / build_growth() (deprecation warnings, soft-deprecated arg docs, and the vignette() pointers in the model-options vignette) from the old environmental-linkages slug to the renamed environmental-linkages-and-priors vignette so the soft-deprecation warnings now resolve.sel_inf(1) is the right-tail floor (analogous to SS3 P6 / end_logit), not a fixed-by-map placeholder.srr_indices / M1_indices arguments and the
legacy env-driven integer codes (srr_fun %in% c(1, 3, 5),
M1_model %in% c(4, 5)) continue to work in 4.4.0 with a one-time
warning that points users at the linkage table. Removal has been
rescheduled from v4.2.0 to v4.5.0 to extend the migration window;
see the "Scheduled removal" section under 4.1.0 below for the
unchanged cleanup checklist.MakeADFun triggered by Beverton-Holt / Ricker
fits with a recruitment linkage. Section 6.9 (expected recruitment)
was indexing R0, alpha, and Beta with the function-scope yr
variable, which the preceding forecast loop left equal to nyrs --
one column past the end of the (nspp, nyrs) matrices. The year-0
block now uses an explicit first_yr = 0 constant. Mean-recruitment
fits (srr_fun = 0) happened to dodge the segfault because
calculate_recruitment() doesn't dereference alpha/Beta for
that case.alpha and Beta now apply the linkage offset on the log
scale (exp(rec_pars + offset)) to match the documented contract
and the log_R0 formula on the same line. Previously the offset
was added on the linear scale, which silently corrupted BH/Ricker
recruitment whenever a non-zero log_alpha or log_beta linkage
offset was active.R0/alpha/Beta column-count assertion at the entry of
section 6.9 so future stale-loop / sizing regressions surface as a
clean R-level error from TMB rather than an opaque segfault.Linkage rows whose design_col == "(Intercept)" no longer carry the
parameter level themselves. Instead:
rec_pars, log_M1, log_growth_pars)
remains estimable and holds the level. Phasing and the per-process
map machinery operate on the base parameter as they would without
any linkages.(Intercept) row's beta_linkage slot is fixed at 0 and
mapped NA. It exists in the table for bookkeeping plus as a hook
for init and priors.init = list((Intercept) = X) on the spec flows to the base
parameter -- it sets the base parameter's starting value rather
than the linkage row.(Intercept) row are re-targeted to the
base parameter in the slot 19 contribution; the prior density
evaluates against rec_pars(sp, 0) / log_M1(sp, sx, ag) /
log_growth_pars(sp, sx, par) rather than the (zero) linkage
value.For slope-only formulas (~ 0 + temp) the behaviour is unchanged:
the base parameter is still mapped NA at its build_params()
default, and the linkage row carries the year-by-year offset.
Year 0 of the recruitment block no longer bakes the year-0 covariate
contribution into R0. R0 is computed from rec_pars(sp, 0)
alone, and the linkage offset multiplies against R0 each year
(including year 0):
R(yr) = R0 * exp(rec_dev(yr) + linkage_offset(yr))
This makes the legacy srr_fun = 1 / 3 / 5 quirk (which double-counted
Temp[0]) obsolete. Users migrating from the legacy paths should now
get clean log-linear behaviour without surprise offsets.
init_supplied (logical) column on Rceattle_linkage_table
tracks whether the user explicitly supplied an init for that row.
Used by build_params() to decide whether to push a base-parameter
init.linkage_is_intercept IVECTOR in the TMB encoding (set from
design_col == "(Intercept)") used by the slot-19 prior dispatch
to evaluate intercept priors against the base parameter.tests-Dynamics/test-linkage-auto-map.R,
tests-Dynamics/test-recruitment-linkage.R, and
tests-Dynamics/test-growth-linkage-species.R were updated to assert
the new contract (base parameters estimable, intercept rows fixed at
0, slope-only offsets in the year-by-year tensor).
A new S3 method on as.data.frame() flattens derived population
quantities into a long data.frame with columns
year, species, sex, age, quantity, value, lwr, upr so that custom
plotting and post-processing don't have to walk the nested
quantities list or rely on the dimnames decisions in
rename_output(). Two shapes are supported and combined into one
frame:
Species-by-year (default which): biomass, ssb, R,
biomass_depletion, ssb_depletion, F_spp. Other species/year
series (B0, SB0, DynamicB0, DynamicSB0, DynamicSBF,
exploitable_biomass, proj_F, fT) are available by name; pass
which = "all" to get every known quantity present on the fit.
Species-by-sex-by-age-by-year: N_at_age, biomass_at_age,
Z_at_age, M_at_age, M1_at_age, M2_at_age, F_at_age,
consumption_at_age, B_eaten_as_prey, NByage0, NByageF. The
age column is biological age (offset by data_list$minage), and
cells padded out to max(nsex) / max(nages) for species with
fewer sexes or ages are dropped rather than returned as NA.
lwr / upr are populated from the TMB sdreport for any quantity
that was ADREPORT'd (currently biomass, ssb, R); other
quantities and fits produced with getsd = FALSE get NA for the
band. The ci_level argument (default 0.95) controls width.
Continuing the Phase A work from 4.2.0, three more classes of inputs
that were previously required as non-NULL can now be omitted, with
data_check() enforcing them only when the model actually needs them:
Phase B: bioenergetics scalars. Ceq, Cindex, Pvalue,
fday, CA, CB, Qc, Tco, Tcm, Tcl, CK1, CK4 may be
NULL in single-species mode. switch_check() fills them with
safe sentinels so TMB's length-nspp DATA_VECTOR requirements
are satisfied. When msmMode > 0 the scalars are required;
data_check() reports which ones are missing or wrong-length in
a single grouped error.
Phase C: env_data. May be NULL. clean_data() defaults it
to a Year-only data.frame(Year = styr:projyr) with zero indices.
Existing checks still error when a feature actually needs an
index (env-dependent catchability, temperature-dependent
consumption, env linkages, srr_indices, M1_indices).
Phase D: emp_sel. New requirement check: when any fleet has
Selectivity = "Fixed", emp_sel must be supplied. Other fleets
do not need it.
Tests. A new tests-Data-processing/test-optional-fields.R
file exercises 25 NULL / requirement scenarios across the four
phases.
Several fields in data_list that were previously required as non-NULL
data.frames are now truly optional. Users who do not need composition
data, conditional age-at-length, empirical selectivity, fixed
numbers-at-age, ration data, or diet data can omit them entirely;
clean_data() default-fills the missing fields with empty data.frames
that carry the metadata columns the downstream code expects, and
data_check() enforces the field only under the conditions where the
model actually needs it.
Phase A (this release): comp_data, caal_data, emp_sel,
NByageFixed, ration_data, diet_data may be NULL. Conditional
requirements are still enforced (caal_data when
any(growth_model > 0); NByageFixed when any(estDynamics > 0);
diet_data when msmMode > 0).
data_check() reorganisation. The validation function has been
reorganised into eight topical sections (top-level scalars; per-species
dimensions; biology; fleet control; observation tables; diet &
predation; environmental data; switches), with shared has_data() /
fc_num() helpers and consolidated duplicate guards. New checks were
added for year-scalar ordering, lognormal SDs, sample sizes,
probability ranges, observation values, fleet referential integrity,
selectivity bin bounds, predation cross-checks, duplicate observations,
and probability-matrix row sums. Several pre-existing dead branches and
matrix-$ access bugs were fixed at the same time.
transpose_fleet_control() removed. The deprecated long-format
fleet_control transposer has been removed from clean_data(),
read_data(), and the package namespace.
A new long-format linkage table lets users express how process
parameters depend on environmental covariates and on stratifying
factors (species, sex, age) through a single formula-driven helper,
linkage_spec(). Each row of the table corresponds to exactly one
estimated coefficient. fit_mod() pools every spec into a shared
design matrix X and a per-row parameter vector
beta_linkage; the TMB template iterates the table once and
accumulates per-process offsets on the linear predictor of the
underlying parameter.
New constructor: linkage_spec(). Captures
(formula, by, species, link, init, bounds, priors, est_phase)
for one process parameter. Anything model.matrix() understands
works: ~ 1, ~ temp + PDO, ~ poly(temp, 4), ~ I(temp^2),
~ splines::ns(temp, df = 4), ~ temp * PDO, etc.
Per-species formulas. Register multiple specs against the
same parameter via linkages = list(log_K = list(spec_a, spec_b))
with each spec's optional species = ... argument scoping it
to a subset of stocks. The pooler unions the design columns
across specs so there's no duplication when species share
covariates.
Priors. First-class via prior_normal(),
prior_lognormal(), prior_gamma(), prior_beta(). The same
constructors are available unprefixed (normal() /
lognormal() / ...) only inside the priors = ... argument
via a private NSE data mask, so user code stays close to
mathematical notation without masking
base::gamma() / base::beta() at the package level. Priors
can be a single value applied to every species, or a named list
keyed by species id (and shortly, by (species, sex)).
Bounds. Per-row lower / upper flow into
build_bounds()$lower$beta_linkage /
build_bounds()$upper$beta_linkage.
Growth (von Bertalanffy / Richards) is the first process
fully wired to the new pipeline. build_growth() gains a
linkages argument and a string-named fun
("empirical" / "vonBertalanffy" / "Richards"); integer
codes still work (fun = 1 is shorthand for
fun = "vonBertalanffy") so existing scripts don't need to be
rewritten apart from substituting fun = for growth_model =.
build_growth(
fun = "vonBertalanffy",
linkages = list(
log_K = linkage_spec(
formula = ~ temp,
by = ~ species + sex,
priors = list(temp = normal(0, 1))
)
)
)
TMB plumbing. New src/TMB/linkage.hpp accumulator;
ceattle_v01_11.cpp reads parallel DATA_IVECTOR(linkage_*)
inputs plus a DATA_MATRIX(linkage_X) and writes a
growth_linkage_offset tensor that is added (additively, on the
log scale) to growth_parameters. Per-row prior densities
contribute to slot 19 of the joint NLL ("Linkage-table priors"
in fit$quantities$jnll_comp).
Documentation. New vignette
vignette("environmental-linkages", package = "Rceattle") walks
through the API, prior families, species-keyed priors,
per-species formulas, basis-expansion formulas, and the
underlying pipeline.
Natural mortality is the second process wired to the
pipeline. build_M1() gains a linkages argument keyed by
log_M1; the offset is added on the log scale to log_M1 inside
the M1_at_age compute. A row's age_bin == NA broadcasts the
offset across ages; specific values pin it to that age slice.
build_M1() also gains string-form acceptance for M1_model
and M1_re (parity with build_growth(fun)):
build_M1(M1_model = "sex_age_invariant", # or 1
M1_re = "ar1_age", # or 4
linkages = list(
log_M1 = linkage_spec(formula = ~ temp, by = ~ species)
))
Growth and M can be linked in the same fit; their rows share the
same global linkage table and the same beta_linkage
parameter vector.
Per-(species, sex) priors. In addition to scalar and
species-keyed priors, each priors[[col]] value may be a
two-level nested list keyed first by species id then by sex id:
priors = list(
temp = list(
`1` = list(`1` = normal(0, 0.1),
`2` = normal(0, 0.2)), # sp 1 by sex
`2` = normal(0, 0.5) # sp 2, both sexes
)
)
Missing keys at either level resolve to "no prior" for that
stratum. The validator checks the structure recursively and
emits actionable error messages keyed by
priors$<col>$<species>[$<sex>] paths.
Default by = ~ species. linkage_spec() now defaults the
by argument to ~ species, so each linkage produces one
coefficient per species without the user having to spell it out.
Pass ~ species + sex for per-(species, sex) coefficients, or
by = NULL to share a single coefficient across every
species/sex (the prior default). This matches the typical
multispecies assessment use case where each stock has its own
environmental sensitivity.
Recruitment is the third process wired to the pipeline.
build_srr() gains a linkages argument keyed by log_R0,
log_alpha, or log_beta; the offset is added on the log scale
to the corresponding parameter at every recruitment compute
call site (hindcast, BRPs, projections, expected R). log_R0
is meaningful for any srr_fun; log_alpha and log_beta
only do work for SRRs that consume them
(srr_fun in c(2, 3, 4, 5) -- Beverton-Holt and Ricker).
build_srr(
srr_fun = 2,
linkages = list(
log_alpha = linkage_spec(formula = ~ temp,
priors = list(temp = normal(0, 0.5)))
)
)
Growth, M, and recruitment can be linked in the same fit; their
rows share the same global linkage table and the same
beta_linkage parameter vector. End-to-end tests in
tests/testthat/tests-Dynamics/test-linkage-auto-map.R verify that
the base parameter (e.g., log_growth_pars, log_M1, rec_pars)
is automatically mapped out (set to NA) when a linkage is active
for that parameter, allowing the linkage intercept to define the base value.
tests/testthat/tests-Dynamics/test-recruitment-linkage.R
cover the analytical relations
R = R0 * exp(beta * temp[yr]) (mean R) and the
growth + M + recruitment composition.
Soft deprecation in build_M1(). The legacy column-index
argument M1_indices and the env-driven structural integer
codes M1_model %in% c(4, 5) are subsumed by the new
linkages = list(log_M1 = ...) argument. Both still work for one
release cycle, but emit a one-time warning that points users at
the equivalent linkage-table call. The string aliases
"env_sex_invariant" / "env_sex_specific" (added briefly on
the dev branch) are removed; they were never released. The cpp's
M1_beta / M1_mult code path stays in place for now -- both
paths add additively to log_M1 on the log scale -- but do not
use both for the same coefficient or you will double-count.
Roadmap. Recruitment is next on the same pipeline; then
random-effects pooling on re_group for hierarchical
shrinkage. The legacy M1_indices / M1_model = 4|5 paths
retire when recruitment migrates.
Schedule update (v4.4.0): The removal originally targeted for v4.2.0 has been pushed to v4.5.0 to give downstream users a longer migration window for the natural-scale linkage API rolled out in v4.4.0. The soft-deprecation warnings continue to point users at the equivalent linkage-table call. The cleanup checklist below is unchanged.
The soft-deprecated API surfaces below remain functional and emit one-time warnings pointing users at the linkage table. They will be removed entirely in 4.5.0. To migrate, replace:
| Legacy | New |
|---------------------------------------|----------------------------------------------------------|
| build_srr(srr_indices = ...) | build_srr(linkages = list(log_R0 = linkage_spec(...))) |
| build_srr(srr_fun = 1) | build_srr(srr_fun = 0) + linkage on log_R0 |
| build_srr(srr_fun = 3) | build_srr(srr_fun = 2) + linkage on log_alpha |
| build_srr(srr_fun = 5) | build_srr(srr_fun = 4) + linkage on log_alpha |
| build_M1(M1_indices = ...) | build_M1(linkages = list(log_M1 = linkage_spec(...))) |
| build_M1(M1_model = 4) | build_M1(M1_model = 1) + linkage on log_M1 |
| build_M1(M1_model = 5) | build_M1(M1_model = 2) + linkage on log_M1 |
Cpp cleanup checklist (search for LEGACY in
src/TMB/ceattle_v01_11.cpp):
PARAMETER_MATRIX(beta_rec_pars), PARAMETER_ARRAY(M1_beta),
and the scratch vectors srr_mult, beta_rec_tmp, env_rec_tmp,
M1_mult, beta_M1_tmp, env_M1_tmp.srr_env_mult blocks (hindcast, BRPs, dynamic
BRPs, projection, R_hat) and the M1_mult.sum() term inside the
M1_at_age compute.Type(0.0) for srr_env_mult at each
calculate_recruitment() call site (or drop the parameter from
the function signature in recruitment.hpp if no caller still
needs it).R-side cleanup:
srr_indices and M1_indices arguments from
build_srr() and build_M1().srr_fun %in% c(1, 3, 5) and
M1_model %in% c(4, 5) as unknown integer values in
.coerce_srr_fun() and .coerce_M1_arg() (drop the
.SRR_DEPRECATED_FUNS / .M1_DEPRECATED_MODELS constants).suppressWarnings() wrappers around internal
build_srr() / build_M1() re-callers in sim_mod(),
retrospective(), jitter(), run_mse(), project_no_F().New fit_control() constructor bundles the optimizer / sdreport /
phasing knobs that previously cluttered fit_mod()'s signature
(phase, getsd, bias.correct, use_gradient, rel_tol,
loopnum, newtonsteps, getJointPrecision, getReportCovariance,
verbose, TMBfilename, nlminb_control). Pass the result via the
new fit_control argument:
fit <- fit_mod(
data_list = BS2017SS,
msmMode = 0,
fit_control = fit_control(phase = TRUE, getsd = FALSE, loopnum = 1)
)
fit_mod()'s visible argument list shrinks from ~33 to ~22 args, so
calls now read as model spec rather than a pile of optimizer flags.
fit_mod() emits a deprecation warning if any of the legacy
control args are passed directly and forwards them into
fit_control for the duration of the deprecation window. Truly
unknown arguments still error with Unused arguments to fit_mod(): ... (no silent drops).
Internal callers (run_mse(), retrospective(), jitter(),
sim_mod(), project_no_F()) now wrap their control args in
fit_control(...) rather than passing them positionally.
S3 methods on the "Rceattle" class so a fit behaves like an R
model object: plot(), coef(), vcov(), logLik(),
residuals(). With df set on logLik, [stats::AIC()] also
works without a dedicated method. nobs() is intentionally
not defined: counting "observations" in a stock-assessment
likelihood (composition cells, indices, catches, priors) is not
well-defined, so [stats::BIC()] does not work — use AIC or
domain-specific information criteria.
plot.Rceattle() is a thin dispatcher: plot(fit, what = "biomass")
/ "ssb" / "recruitment" / "depletion" / "index" / "catch" /
"selectivity" / "mortality" / "data". ... is forwarded to
the underlying plot_*() function.
residuals.Rceattle(type = ...) returns a long-format data frame
with rows from one or more of the four fitted data sources:
"index" and "catch" (log-scale by default; switch with
scale = "natural"), "comp" (Pearson on fitted proportions, with
the Age0_Length1 flag preserved), and "caal" (Pearson on
fitted proportions, with both the conditioning Length and the
age Bin). type = "all" returns all four stacked.
README now has a self-contained Getting started block that fits a bundled model and exercises every new S3 method, so first-time users on the pkgdown site / CRAN no longer have to bounce to the GitHub wiki to see a working example.
Vignette 8 ("Model parameterizations") is being expanded to fill in coverage gaps surfaced during the 4.0.2 release audit:
M1_re = 0..6).srr_est_mode = 3).Catchability = 4 (Power),
5 (Environmental), and 6 (AR1).Selectivity = 6 (2DAR1) and
7 (3DAR1).growth_model = 1 von Bertalanffy /
2 Richards).initMode = 0..4).data_check() now stops with a clear message if a user requests
msmMode = 3:9 (Kinzey & Punt 2009 functional responses --
Holling I/II/III, predator interference, predator preemption,
Hassell-Varley, Ecosim). Those code paths are unvalidated against
the current parameter set and should not be used for advice.plot_timeseries, plot_selectivity,
plot_mortality, plot_maturity, plot_b_eaten, plot_b_eaten_prop,
plot_m_at_age, plot_m2_at_age_prop, plot_f, plot_ration,
plot_index, plot_catch, plot_logindex, plot_indexresidual,
plot_comp, plot_selectivity_vs_maturity, plot_stock_recruit)
now restore graphics par() on exit instead of mutating the
user's device permanently.T / F shortcuts with TRUE / FALSE throughout the
package source (~60 occurrences)..data$Bin / .data$Length inside tidyr::pivot_wider
arguments with quoted strings (tidyselect 1.2.0 deprecation).examples/Georges_bank_example.R now calls plot_mortality()
instead of the long-removed plot_mort()._pkgdown.yml "Get started" / overview navbar links now point at
the actual generated articles/Rceattle-overview.html (was
Rceattle-overview-4_17_2025.html, which 404'd).README.md example links now reference the correct
underscore-separated filenames (Fit_2018_GOA_multi-species_model.R
etc.) -- the previous space-encoded URLs 404'd on GitHub.\seealso{} block in ?Rceattle-package by
consolidating R/Rceattle.R into R/Rceattle-package.R. Both
files declared _PACKAGE, so roxygen2 emitted the auto-generated
links twice.graphics::box to package imports (cleared the lone
R CMD check NOTE for plot_data).Finit in the
denominator for fished initial-equilibrium modes
(initMode = 3 or 4), correcting a bias in the plus-group
N-at-age when Finit > 0.inst/CITATION and ?Rceattle-package.initMode accepts integer codes or string aliases.HCR accepts integer codes or string aliases.tests/testthat/test-subdir-folders.R no longer calls
testthat::test_dir() for each subdirectory. Nested test_dir()
inside test_check() triggered an "evaluation nested too deeply:
infinite recursion" abort inside rlang's trace deparser whenever
any test failed, masking the real failure. Subdirectory test files
are now discovered with list.files() and pulled in via
source() so they register against the outer reporter directly.tests/testthat.R now wraps library() calls in
suppressPackageStartupMessages() so transient build-version
notices (e.g. "package 'dplyr' was built under R version 4.5.2")
do not get captured as test warnings whose backtraces then crash
rlang's expr_deparse at end-of-run.tests/testthat/test-Dynamics/test-initial-dynamics evaluates the different starting conditions.run_mse() and check_mse() now use parallel::parLapply on a
PSOCK cluster instead of foreach::foreach(...) %dopar%.
%dopar% path triggered rlang::expr_deparse infinite
recursion under nested test_that backtraces because each
foreach invocation captures call frames that recurse during
error formatting. PSOCK workers are clean R processes with no
captured promise chains, so the issue does not occur.run_mse() gains a cores argument (default NULL picks
parallel::detectCores() - 6); both functions cap at 2 cores
when _R_CHECK_LIMIT_CORES_ is set so they comply with CRAN's
R CMD check limit.foreach and doParallel removed from Imports:.TMBhelper moved from Imports: to Suggests:. Rceattle now uses an
internal .fit_tmb() wrapper that delegates to TMBhelper::fit_tmb()
when the (optional, GitHub-only) package is installed and otherwise
falls back to a stats::nlminb() + TMB::sdreport() path. This
removes the largest install-friction barrier for new users.dplyr moved from Depends: to Imports:. The package no longer
attaches dplyr to the user's search path on load (so it no longer
masks stats::filter / stats::lag).kableExtra dropped from Suggests:; vignettes now use
knitr::kable() for table rendering.quarto dropped as a vignette engine; all vignettes are now .Rmd.run_mse() no longer has om = ms_run, em = ss_run defaults.
Both arguments are now required and validated as objects of class
"Rceattle" before the MSE loop runs. Calling run_mse() with no
arguments previously produced a confusing "object 'ms_run' not found"
error; it now stops with a clear message.print.Rceattle() and summary.Rceattle(). Auto-printing a fit
inside knitr / RStudio / R Markdown previously dumped tens of MB of
nested data and could trigger deep recursion errors during vignette
rendering..Rbuildignore (excludes examples/, R/dev/,
src/TMB/Dev/, .Rhistory, .claude/, .DS_Store, build tarballs,
.Rcheck directories)..gitignore to catch all *.o / *.so / *.dll and
*.Rcheck/ directories.-Wfixed-enum-extension warning by
scoping the diagnostic pragma around #include <TMB.hpp> rather
than via global Makevars flags.The 4.0.1 development cycle reorganized several data_list columns
and fit_mod / build_* arguments. Models or data files saved
against earlier 4.x revisions may need updating; see the renames
below. Compiled from
inst/Running_list_of_updates.qmd plus the dev branch commit log.
Pyrs -> ration_data (the old name is still accepted on read,
but is silently renamed).UobsWtAge -> stom_prop_data.fsh_biom -> catch_data.srv_biom -> index_data.Nselages -> N_sel_bins (in fleet_control).Sel_norm_bin1 / Sel_norm_bin2 <- Age_max_selected /
Age_max_selected_upper (selectivity normalization bins).Age_first_selected -> Bin_first_selected (in fleet_control).sel -> sel_at_age (model report).fleet_control now carries a Month column (month of observation
for indices / fisheries).build_M1: M1_prior_mean -> M_prior,
M1_prior_sd -> M_prior_sd.build_srr: srr_prior_mean -> srr_prior;
R_hat_endyr replaced by srr_hat_styr / srr_hat_endyr.fit_mod: suit_meanyr replaced by suit_styr / suit_endyr.initMode semantics revised: 0 = free-parameter N-at-age,
1 = unfished equilibrium with no devs, 2 (default) = unfished
equilibrium with initial devs, 3 = fished equilibrium with initial
devs. Type 4 ("non-equilibrium scaled") added later.fleet_control$Comp_loglike = 1 (or "DirichletMultinomial").CAAL_loglike /
CAAL_weights controls in fleet_control. CAAL data also flow
through sim_mod() for simulation testing.Diet_loglike switch on the bioenergetics control sheet selects
between multinomial (0) and Dirichlet-multinomial (1) for diet
composition.Selectivity = "Hake" or 5),
after Taylor et al.2DAR1 (= 6) and 3DAR1 (= 7) selectivity
parameterizations, after Cheng et al. (2024).Catchability = 6 ("AR1"): annual AR1 catchability deviates fit
to an environmental index, after Rogers et al. (2024) for the GOA
pollock model. Environmental q-link (Catchability = 5) also
exposed.build_growth() and the growthFun
argument to fit_mod(). alpha_wt_len / beta_wt_len added to
the data control sheet. Length-based suitability (suitMode = 1 /
2 / 3 / 4 / 5 / 6) wired through to use the estimated
growth model. Comparison with WHAM growth implemented under
tests/comparison/.suitMode per
predator).suit_styr / suit_endyr) instead of "mean year".srr_est_mode = 3.M_prior /
M_prior_sd priors carried through build_M1().remove_F() function returns a fitted model with F set to 0 --
used internally for dynamic reference point calculation.DynamicHCR = TRUE in build_hcr() to switch from static to
dynamic SB0 reference points.HCR = 1): maximize joint catch across
species, optionally constrained to keep depletion above Plimit.HCR = 6) using Pstar /
Sigma uncertainty buffer.HCR = 7).HCRorder controls the order in
which species F is solved (e.g. predators before prey) inside
build_hcr().run_mse() now writes per-simulation .rds files when dir is
specified, for streaming-friendly long runs. load_mse() reads
those back.check_mse() validates which OM/EM simulations converged.mse_summary() produces a per-fleet performance-metric table
(mean catch, IAV, P(closed), MSE on SSB, P(F > Flimit),
P(SSB < SSBlimit), terminal depletion, ...).cap (catch cap), catch_mult (catch
multiplier), rec_trend (linear projected recruitment trend),
fut_sample (future sampling effort), per-fleet
assessment_period / sampling_period, regenerate_past (refit
EM to OM-simulated past data), and timeout/try-error handling
per simulation.Recruitment_and_fixed_F_projections.R and Simulation_testing.R
examples added.jitter() function to perturb starting values and re-fit, for
global-vs-local-minimum diagnostics.retrospective() peels with optional nyrs_forecast.model_average() for averaging derived quantities across multiple
fitted models, with optional bootstrap uncertainty.compare_sim() and sim_mod() for parametric simulation testing.McAllister-Ianelli-reweighting.R example for composition
reweighting.Selectivity = "Fixed" (= 0) for empirically supplied selectivity
blocks via the emp_sel data sheet.TMBfilename argument to fit_mod() to point at an alternate
.cpp during development.fleet_control. Selectivity
normalization is now controlled via Age_max_selected (i.e.
Sel_norm_bin1) on a per-fleet basis instead of always
normalizing by the maximum-selectivity age.NA values inside the valid age/length range of composition data
are now coerced to 0 with a warning (previously silently dropped
or errored).Selectivity_dimension and N_sel_bins).nages at data_check() time.