(Advanced)Mathematical definition

Last updated: 2026-07-11

This note defines the observed-to-expected ratio used by stg_pair_enrichment() and separates the scientific estimator from the finite value used only for heatmap display.

1. Statistical unit and notation

Let \(r\) index an independent ROI or biological replicate, \(c\) index a condition, and \(i,j,k\) index cell types. Within ROI \(r\), define:

Symbol	Definition
\(S_{ir}\)	number of singlets assigned to type (i)
\(S_r=\sum_kS_{kr}\)	total singlet count
\(p_{ir}=S_{ir}/S_r\)	empirical singlet proportion
\(D_r\)	total number of certain heterotypic mixed units
\(O_{ijr}\)	observed count of the undirected pair \(\{i,j\}\)

The braces \(\{i,j\}\) indicate an unordered pair: \(\{i,j\}=\{j,i\}\). Therefore, each biological pair appears once in the triangular heatmap.

2. Conditional heterotypic null model

Consider two independent draws \(X,Y\) from the ROI singlet composition. Before conditioning,

\[P(X=i,Y=j)=p_{ir}p_{jr}.\]

For \(i\ne j\), the unordered event \(\{i,j\}\) contains two mutually exclusive ordered outcomes, \((i,j)\) and \((j,i)\). Thus,

\[P(\{i,j\})=2p_{ir}p_{jr}.\]

The probability that two draws have different types is

\[H_r=P(X\ne Y) =1-P(X=Y) =1-\sum_kp_{kr}^2.\]

Because the analyzed RCTD class consists of heterotypic doublets, the relevant sample space is \(X\ne Y\). The conditional pair probability is therefore

\[q_{ijr} =P(\{i,j\}\mid X\ne Y) =\frac{2p_{ir}p_{jr}}{1-\sum_kp_{kr}^2}, \qquad i<j.\]

The probabilities sum to one over all unordered heterotypic pairs:

\[\sum_{i<j}q_{ijr} = \frac{\sum_{i<j}2p_{ir}p_{jr}} {1-\sum_kp_{kr}^2} =1.\]

The expected count in ROI \(r\) is

\[E_{ijr}=D_rq_{ijr}.\]

This null model asks whether the doublet-pair composition differs from random heterotypic pairing under the local singlet composition. It does not assume equal cell-type abundance.

3. Pair-specific ROI eligibility

Rare singlet populations yield unstable \(p_{ir}\) and very small expected counts. For minimum count \(m=100\), define

\[I_{ijr} =\mathbf 1(S_{ir}\ge m,\;S_{jr}\ge m).\]

ROI \(r\) contributes to pair \(\{i,j\}\) only when \(I_{ijr}=1\). The eligible set for condition \(c\) is

\[\mathcal R_{ijc} =\{r:\operatorname{condition}(r)=c,\ I_{ijr}=1\}.\]

Crucially, \(H_r=1-\sum_kp_{kr}^2\) is computed from every singlet type in the ROI before applying \(I_{ijr}\). Recalculating the denominator after removing rare types would change the null sample space and over-normalize the remaining pairs.

4. Condition-level aggregation

Observed and expected counts are summed before their ratio is calculated:

\[O_{ijc} =\sum_{r\in\mathcal R_{ijc}}O_{ijr}, \qquad E_{ijc} =\sum_{r\in\mathcal R_{ijc}}E_{ijr}.\]

This is a ratio of pooled counts, not the arithmetic mean of ROI-level \(O/E\) values. It weights each ROI through its expected information content and avoids undefined ROI-level ratios when \(E\) is tiny.

Let

\[N_{ijc} =\sum_{r\in\mathcal R_{ijc}}D_r.\]

The adaptive expected-count threshold is

\[E_{\min,ijc} =\max(E_0,\alpha N_{ijc}) =\max(5,10^{-4}N_{ijc}).\]

The estimate is considered reportable when

\[|\mathcal R_{ijc}|\ge K \quad\text{and}\quad E_{ijc}\ge E_{\min,ijc},\]

where the package default is \(K=1\). Increasing \(K\) is appropriate when the study design requires evidence from multiple biological replicates.

5. Primary enrichment effect size

For an estimable pair, define

\[R_{O/E,ijc} =\frac{O_{ijc}}{E_{ijc}},\]

and its log-scale representation

\[L_{ijc} =\log_2R_{O/E,ijc} =\log_2\left(\frac{O_{ijc}}{E_{ijc}}\right).\]

Interpretation:

\(R_{O/E}>1\), or \(L>0\): enriched relative to the null.
\(R_{O/E}=1\), or \(L=0\): consistent with the null expectation.
\(0<R_{O/E}<1\), or \(L<0\): depleted relative to the null.

No pseudocount is added to positive \(O\), and \(E\) is never modified. This preserves the exact effect size and prevents a fixed constant from dominating pairs with small \(E\).

6. Observed zeros and heatmap display

If an estimable pair has \(O_{ijc}=0\), then

\[R_{O/E,ijc}=0, \qquad L_{ijc}=-\infty.\]

An infinite value cannot be mapped to a finite color scale. For visualization only, stGrads defines

\[L^{\mathrm{display}}_{ijc} =\log_2\left(\frac{\delta}{E_{ijc}}\right), \qquad \delta=0.5.\]

The output simultaneously records zero_observed = TRUE, and the triangular heatmap labels the tile “Zero observed”. This display value must not be reported as the exact biological effect size.

Non-estimable pairs have log2_oe = NA and are not passed to geom_tile(), so their locations remain blank rather than appearing as a third biological state.

7. Sensitivity estimate

For comparison with earlier analyses, the package retains

\[R^{\mathrm{sensitivity}}_{ijc} =\frac{O_{ijc}+0.5}{E_{ijc}+0.5}.\]

This quantity is stored in oe_sensitivity and log2_oe_sensitivity. It is not used by the default heatmap and should be described explicitly as a sensitivity analysis if reported.

8. Reproducible implementation

enrichment <- stg_pair_enrichment(
  metadata,
  sample_col = "MatchKey",
  condition_col = "Condition",
  min_singlet_cells = 100,
  min_expected = 5,
  min_expected_fraction = 1e-4,
  min_eligible_roi = 1,
  zero_display_pseudocount = 0.5
)

condition_oe <- enrichment$condition
filter_parameters <- enrichment$parameters

stg_plot_pair_triangle(
  condition_oe,
  value_col = "log2_oe",
  show_counts = TRUE,
  zero_label = "Zero\nobserved"
)

Important output fields:

Column	Meaning
`observed`	aggregated (\(O_{ijc}\))
`expected`	aggregated (\(E_{ijc}\))
`oe_raw`	primary unregularized (\(O/E\)) effect size
`log2_oe`	exact \(log_2(O/E)\), including \(-\infty\) for observed zero
`log2_oe_display`	finite value used only for heatmap coloring
`zero_observed`	indicator that the exact observed count is zero
`estimable`	whether abundance, replicate, and expected-count rules pass
`exclusion_reason`	reason a pair is not estimable