[edudc]

Hi, after a long time trying to use DeepTrack2 for fitting real holography images I found there was a bug in the scattered field. I checked against another python package called HoloPy and with this PR fixes they agree.

TL;DR: the Mie coefficients and S1/S2 are fine (they match HoloPy to 1e-8). The bugs are in the step that turns S1/S2 into a field on the image plane.

Bugs

1. The sign of the scattered field is flipped in both modes

• The code has -1j / (k R). The textbook formula (Bohren & Huffman eq. 4.75) is exp(ikR) / (-ikR) * S, and 1/(-i) = +i.
• The consequence was that for a bead denser than water DeepTrack2 advanced the phase instead of delaying it. For a 228 nm, 1.59 n bead in focus, the current code gives -0.15 rad. HoloPy with the same parameters gives +1.37 rad. And the estimate 2π·Δn·d/λ gives ~1.4 rad.
• The holograms come out with inverted fringes.

2. Hybrid mode is missing a factor -2π / ((k dx)² cos θ)

• S(θ) is per unit solid angle, but the FFT grid is flat in (kx, ky).
• Without the (k dx)² term, the scattered strength depends on the simulation pixel size.
• Without 1/cos θ, high angles are under-weighted. Fitting a field generated with HoloPy of a 228 nm sphere gives 190 nm with a good residual because it tries to compensate the missing light at high angles with a smaller bead.

3. The pupil's defocus phase also advances the illumination (suggested by @cmanzo)

• K dz sqrt(1 - (NA/n)² ρ²) at ρ = 0 is K dz, so every propagation step multiplies the plane wave by exp(i K dz).
• Brightfield propagates the illumination through the volume slices and back by z_max, leaving exp(-i K z_min). Its phase depends on where the volume sits, but MieSphere fields are added at the focus without it.
• A small dt.Sphere far away in the corner changes a MieSphere's image by up to 5.4, on an image ranging 0.56–1.33 (633 nm)
• Darkfield (|field - 1|²) has a background of up to 2.0 instead of 0.

Fixes

• scatterers.py: flipped the sign in geometric mode and added the missing factor in hybrid.
• test_scatterers.py:
  • A weak sphere is a thin phase object, so its scattered field summed over the plane must equal +i k0 Δn V.
  • It checks the sign for both modes. For hybrid it also checks magnitude and phase, at two pixel sizes.
  • Current code gives -0.18 (and -0.04 at half the pixel size); with the fix it gives 1.01
• optics.py: sqrt(...) - 1 in _pupil_numpy and _pupil_torch, propagate relative to the illumination, as the Mie fields already are.
• test_optics.py::test_Brightfield_illumination_phase_independent_of_volume (numpy + torch).
• test_dlcc.py::test_3_B: I regenerated the gold values, because the stored numbers were with the old sign. The old values are roughly the new ones mirrored around 1 (0.85 ↔ 1.17), which is what the sign flip does. Please double-check you're OK with this one.

With the fix, hybrid mode matches HoloPy:

• complex scale 1.000, phase 0, residual 0.2–1.3% (3–6 um behind the bead, NA 0.9 and 1.2);
• hologram Pearson +1.000;
• 228 nm field generated with HoloPy is fitted by DeepTrack2 as 228 nm.

Heads up

• This changes every simulated Mie image. Geometric flips contrast; hybrid changes in strength and shape. Probably worth a line in the release notes.
• iSCAT: phase_shift_correction multiplies by exp(ikz + iπ/2). Was that tuned on top of the old sign? With this fix, the iSCAT contrast shifts by π. I didn't touch it because I don't have an iSCAT reference to check against.