Scoop, explained: profiling attacks against higher-order masking

Train a neural network against a masked implementation and you often watch nothing happen. The loss sits at the value of a blind guess for epoch after epoch, then drops all at once. The field calls this the plateau, and it gets longer as the masking gets stronger. Scoop is an optimizer we designed to make it shorter. This post explains where the plateau comes from, what Scoop changes, and what held up afterwards. One headline result did not, and I’ll get to it.

Masking in one picture

Masking is the standard defence against power analysis. Instead of computing on a secret value $\sym{s}{s}$ directly, the implementation splits it into $\sym{n}{n} + 1$ random shares and only ever handles the shares:

Each share is uniformly random, and any $n$ of them together are independent of $s$. So a power trace that leaks one share, or any $n$ shares, says nothing about the secret. To learn $s$ from the leakage you have to combine all $n + 1$ shares at once, which makes $\sym{n}{n}$ the masking order: an attack that combines fewer samples, or uses lower statistical moments, sees noise.

The attacks here are profiling attacks. You train a model on a copy of the device you control, then use it on the target. The hard setting, and the one Scoop is about, is when you don’t know the masks even on your own copy. The model gets traces and the secret, never the shares, and has to discover the recombination by itself.

The plateau

Training a model means minimising its loss over the profiling traces, almost always with stochastic gradient descent or a variant like Adam. With the negative log-likelihood loss, a model is correct once its loss drops below the entropy of the secret, $\sym{entropy}{\mathbb{H}[s]}$: that’s the loss of a model that outputs the uniform distribution. The gap is the perceived information:

The plateau is the stretch at the start of training where the loss sits at $\sym{entropy}{\mathbb{H}[s]}$ and the perceived information stays at zero. Masure et al. (TCHES 2023) conjectured that its length grows exponentially with the masking order. Some second-order masked datasets are still out of reach of deep learning.

That matters for evaluation, not just for patience. A long plateau doesn’t mean the device is secure. It means you can’t tell how secure it is until the plateau ends. If that takes weeks, an evaluation that stops earlier reports a security level the device doesn’t have.

The existing explanation came from a theorem of Shalev-Shwartz et al. about learning parity-like functions, which is what recombining shares amounts to. Informally: as the number of terms grows, the gradient points in nearly the same direction whatever the function you’re trying to learn. Here the “function” is the device’s leakage model, and the number of terms plays the role of the masking order:

The left side is how much the gradient changes from one leakage model $\sym{u}{u}$ to another. It shrinks exponentially with $\sym{n}{n}$, so at high order the gradient carries almost no information about what the model should learn. Masure et al. didn’t go further than that, which left the question of why.

A model small enough to look at

To see what the loss surface actually looks like, the paper shrinks the problem until it fits on a plot. Each trace is just the shares, each multiplied by a random sign:

The signs $\sym{alpha}{\alpha_i}$ are the leakage model, the thing the network has to discover. Even a one-layer network on this input has $2(n+1)$ weights, too many to draw. So the paper uses a scheme-aware model from Masure et al.: one tiny predictor per share, each with a single weight $\sym{theta}{\theta_i}$, whose outputs are combined by a convolution product $\sym{conv}{*}$, the operation that matches XOR on probability vectors:

With one weight per share, the loss has a closed form (the paper derives it with a computer-algebra system), a single global minimum at $\sym{theta}{\theta_i} = \sym{alpha}{\alpha_i}$, and for two shares it’s a surface you can look at. The caveat, which the paper states: scheme-aware models are known to scale worse than ordinary networks, so lessons from this model may not transfer exactly.

Two shares (first-order masking). The secret is one bit, so any loss below 1 bit means the model is right. The region of correct models is the quadrant matching the signs, here $\theta_0 < 0$, $\theta_1 > 0$ for $\alpha_0 = -1$, $\alpha_1 = 1$. The loss gets below 0.5 inside the plotted window, and from the initialisation region near the origin the gradient flow heads straight for that quadrant.

Three shares (second-order). The loss is now a function of three weights, so the paper plots a slice with $\theta_2$ fixed at its best value. Two things change. The surface is flatter: the loss doesn’t get as low and the slopes towards it are gentler, which is to say the gradients are smaller. And the origin becomes a saddle point: the gradient there is near zero and the flow is drawn towards it, but it is neither a minimum nor a maximum. Most paths detour past it before turning towards a minimum.

Now look at where training starts. PyTorch’s default initialisation draws each weight uniformly in a small interval around zero, and plain gradient descent moves by a step proportional to the gradient:

Put those together and you get the plateau. Training starts near the origin, which is near the saddle, and the longer the traces, the closer it starts. It starts on a surface whose gradients shrink with every extra share, so its steps shrink too. It has to crawl out of the flattest part of the landscape with the smallest steps.

The paper scaled the analysis up to six shares. The expected squared gradient norm falls exponentially with the number of shares, and the variance of the gradient across inputs falls at the same rate as the Shalev-Shwartz bound.

The obvious fixes don’t work. A larger learning rate compensates for small gradients until it exceeds what the local curvature allows, and then training diverges. Learning-rate schedules have been tried in this field without a principled study, and still plateau. A wider initialisation would start further from the saddle, but normalised inputs need weights centred on zero with a small spread, or the network saturates.

demo://loss-landscape● running in your browser

2 shares3 shares

--p

ℓ_1+εℓ₂

1.10 · ℓ_1+ε, ε = 0.1 (Scoop’s)

axes ∇ψ(θ)axes θ

0-22-0.10.1

0-22-0.10.1

θ₀ (axis ∇ψ)θ₁ (axis ∇ψ)

run optimisersnew init

• gradient descenteq. 2.4.2plateau end: step 220
• NewtonHutchinson diagonalplateau end: step 56
• mirror descentℓp potentialplateau end: step 359
• Scoopmirror + Newtonplateau end: step 95

0.45 bits1.25— 𝓛 = 1 bit: guessing

1.090.41𝓛 (bits)step →

Static render of the default setting: 2 shares, p = 1.10. Plateau ends (largest drop in loss): Newton 56 · Scoop 95 · gradient descent 220 · mirror descent 359 steps. With JavaScript on, it re-runs live.

Use the curvature

If the problem is small steps on flat ground, the classic answer is Newton’s method. Instead of stepping along the gradient, it divides the gradient by the curvature, the Hessian: large steps where the surface is flat, small ones where it bends sharply. On convex problems it’s the optimal way to reach a minimum.

Deep learning can’t compute the full Hessian, so practical methods use an estimate $\sym{hhat}{\hat{h}_t}$ of it, together with a learning rate:

On the three-share toy model this works as hoped. The loss itself barely moves (the secret is one bit), so the paper tracks how much it drops per step, $\sym{dL}{\delta_{\mathcal{L}}} = \mathcal{L}(\theta^{(t)}) - \mathcal{L}(\theta^{(t+1)})$, and takes the end of the plateau to be where that drop peaks. Gradient descent gets there after 380 iterations, Newton’s method after 50: close to ten times faster.

Getting from a toy model to a real network takes three fixes.

Keep it heading downhill. Away from convex problems, Newton’s method can converge to a maximum as happily as to a minimum. The Hessian estimate must stay positive definite. Quasi-Newton methods like L-BFGS guarantee that, but cost too much memory and compute for this use. The alternative is to shift the estimate’s eigenvalues until they’re all positive. That makes it badly conditioned, since the smallest eigenvalue ends up tiny, so the update is also clipped.

Only estimate the diagonal. A full Hessian is quadratic in the number of parameters. A diagonal approximation is cheap, and Newton’s method still converges with an inexact Hessian. One existing diagonal estimator, Gauss–Newton–Bartlett, used by the Sophia optimizer, drops one term of the Hessian to stay positive. That bias is argued to be small in general, but the paper is wary of assuming it in side-channel analysis, where general deep-learning habits have transferred poorly.

Instead, Scoop uses Hutchinson’s estimator. Draw a random vector $\sym{z}{z}$ with zero-mean, unit-variance entries, and compute

On average this is exactly the Hessian’s diagonal, and it never forms the Hessian: the Hessian-vector product on the right is just the gradient of a dot product, one extra backward pass, linear in the number of parameters. Rademacher entries ($\pm 1$ with equal odds) give the lowest variance.

The catch is that Hutchinson’s estimator converges slowly. The paper’s own contribution here is a biased version: with a finite number of samples $\sym{m}{m}$, you get a smaller expected error if the entries of $\sym{z}{z}$ have a variance slightly below 1. The optimal variance is

where the overlined matrix is the Hessian with its diagonal zeroed. As $\sym{m}{m}$ grows this tends to 1, the unbiased case. Computing it exactly needs the Hessian you’re trying to estimate, so Scoop uses a fixed heuristic: entries of $\pm 0.9$.

Prefer sparse weights

The second idea is specific to side-channel analysis. A trace has thousands of samples, and only a few of them depend on the secret. An ideal model should rely on a handful of features, so most of its first-layer weights should be zero, and that has been observed in practice. Since the first layer holds most of the weights when inputs are this long, the paper aims for sparsity across the whole model.

The usual way to get sparsity is to add a penalty, like an $\ell_1$ norm, to the loss. That’s explicit regularisation. The alternative is to pick an optimizer that drifts towards sparse solutions by construction: implicit regularisation.

Gradient descent has a hidden geometry. Each step is the solution of a small problem: follow the gradient, but don’t move far, with “far” measured as Euclidean distance. Mirror descent swaps that distance for a Bregman divergence $\sym{breg}{D_\psi}$ built from a convex potential $\sym{psi}{\psi}$:

Solving it gives an update that runs in a different space. Map the weights through $\sym{gradpsi}{\nabla\psi}$ (the dual space), take an ordinary gradient step there, and map back:

Mirror descent is known to act as an implicit $\psi$-regulariser, and the result extends to the stochastic version for some model families. The natural potential for sparsity would be the $\ell_1$ norm, but the update needs $\nabla\psi$, and $\ell_1$ has no gradient at zero. So Scoop follows Azizan et al. and uses the $\ell_{1+\sym{eps}{\epsilon}}$ norm with $\sym{eps}{\epsilon} = 0.1$.

Here’s the intuition for why that favours sparse weights. For a single weight, the map into the dual space is $\sym{gradpsi}{(1+\epsilon)|\theta|^{\epsilon}\operatorname{sign}(\theta)}$, and the map back raises to the power $1/\sym{eps}{\epsilon} = 10$. A weight’s dual value has to grow quite large before the weight itself is anything but tiny: small dual values are crushed towards zero on the way back. Only weights that keep receiving gradient in the same direction escape.

Scoop

Scoop puts the two ideas together: mirror descent with the $\ell_{1+\epsilon}$ potential, where the gradient step in the dual space is preconditioned by the inverse of a diagonal Hessian estimate. The name is a stretched acronym of exactly that, SeCond-Order precOnditioned sParse stochastic mirror descent. One iteration:

Line by line: estimate the Hessian diagonal with the biased Hutchinson trick; keep running averages of it and of the gradient (momenta $\sym{beta}{\beta_2}$ and $\sym{beta}{\beta_1}$); map the weights into the dual space, subtract the clipped, curvature-scaled step ($\sym{c}{c}$ is the clipping function); map back. Everything is element-wise, so the cost stays linear in the number of parameters. It is more expensive than Adam, but not by much.

Results

Simulated masking

The cleanest test is a noise-free simulation that contains every combination of share values, which turns the attack into a pure optimisation problem: an 8-bit secret under Boolean masking of order $n$, Hamming-weight leakage of an AES S-box output. Three architectures (an MLP, a VGG-style CNN and a Transformer) are each trained with Adam and with Scoop for up to $10^5$ epochs. The plateau length is the number of epochs until the validation loss reaches $\mathbb{H}[s] - 0.05$, averaged over 100 runs. Plateau reduction with Scoop, compared to Adam:

Read the first column first: without masking, Scoop makes the plateau longer. The gain starts with masking and grows with the order. For the CNN at order 3 and above, the plateau is about five times shorter, for roughly 5% more training time on the same hardware. The Transformer could only be run up to order 2 on our GPU; at order 2, Adam never found the secret within the $10^5$-epoch budget, while Scoop did in under $10^3$ epochs.

And the sparsity shows up: an MLP trained against fourth-order masking ends with its weights more concentrated around zero under Scoop than under Adam.

ASCADv1

ASCADv1 is the field’s standard benchmark: power traces of a software AES with first-order Boolean masking. The usual extracted version targets the third S-box byte, which has no first-order leakage. With a VGG-style CNN and an MLP, each tuned for a few hours, against the best results we knew of:

$N_a$ is the number of attack traces needed to recover the key byte. Fewer is better, and 73 was below every published result we knew of. The GPU times run on different hardware, so read them as orders of magnitude.

Which half of Scoop does the work? Same CNN, five optimizers:

Each ingredient alone beats Adam, and mirror descent alone is already strong. Combined with the GNB Hessian it does worse than on its own; combined with the biased Hutchinson estimator it gives the best attack. The paper is careful here: it would take stronger evidence to say the biased estimator reliably beats the unbiased one.

The plateau result is clearest on a single MLP. Trained with Scoop, its training curves show almost no plateau. The same architecture with the same seed, trained with Adam, plateaus for 38 epochs.

Scoop also makes hyperparameter search cheaper. The paper sampled over 150 random MLP architectures and 100 random CNNs from a wide grid, and trained each one with both optimizers:

With Scoop, about twice as many random architectures turn into working models. Weighing success rate against tuning time, the paper estimates the cost of finding a working model drops by 18% for MLPs and 64% for CNNs, while attack performance, measured as perceived information, improves by 287% and 26%.

These ASCADv1 models have since been checked for the failure described below. The CNN’s input gradients line up with where the masking shares are known to leak, and its key ranking barely changes when the label prior is divided out. They learned the leakage.

ASCADv2, and what we got wrong

ASCADv2 is much harder: affine masking, which behaves like second-order masking, plus shuffling of the order in which the S-box lookups run. Traces are 15,000 samples long. The only attack before ours used a weakness of the affine scheme, a multiplicative mask shared across the whole state, and switched the shuffling off. The Scoop paper reported the first attack without those assumptions: a single-hidden-layer MLP with about 231 million parameters, recovering the key byte after 150,000 attack traces. Adam, Sophia and mirror descent alone, given the same setup, all failed.

That attack was a false positive. Zeroing the model’s first layer, which cuts it off from the traces entirely, left the key recovery working. The targeted byte isn’t uniformly distributed in ASCADv2, and the model had learned that distribution rather than the leakage. The standard metrics reward that. The full story, and the checks that catch it, are in Is it really broken?.

So the comparison with Adam on ASCADv2 says nothing either way. The simulations and ASCADv1, where the models demonstrably use the traces, are what support Scoop.

What’s still open

Scoop shortens the plateau; it doesn’t remove it. The paper’s own explanation pinned the plateau on gradient-based training: shrinking gradients and a saddle point. Later work puts that in doubt. A Bayesian network trained with no gradient at all plateaus too, and its plateau grows with the masking order at the same rate as Adam’s. That suggests the cause lies in masking itself, and that Scoop helps by making the optimiser cope better, not by removing the cause. There’s a short note on it: Is the gradient to blame for the plateau?

A smaller lead from the paper: $\epsilon$, fixed at 0.1 here, could be tuned or scheduled during training.

Code

Scoop is a PyTorch optimizer, a fork of Sophia, and drops into an ordinary training loop. The one change is that the backward pass keeps its graph, so the Hessian-vector product can differentiate through it, and every few mini-batches you refresh the Hessian estimate:

from scoop.scoop import Scoop

optimizer = Scoop(model.parameters(), lr=1e-4)

for step, (x, y) in enumerate(loader):
    optimizer.zero_grad()
    loss = F.nll_loss(model(x), y) / math.log(2)  # loss in bits
    loss.backward(create_graph=True)              # keep the graph for the Hessian-vector product
    if step % hessian_every == hessian_every - 1:
        optimizer.hutchinson_hessian()            # refresh the diagonal Hessian estimate
    optimizer.step()

The README lists the other knobs: the momenta (default 0.965 and 0.99), $\ell_2$ weight decay, the choice between the classic and biased Hutchinson estimators, and how many Hutchinson samples to draw. It suggests putting them in your tuning grid. Setting $\epsilon \geq 1$ turns off the push towards sparsity if your problem doesn’t want it.

Code: [code link: placeholder] · Paper: ePrint 2025/498