Skip to content

Fix BCI2000RawIO scaling: decimal gains were parsed as 0 and the offset was applied with the wrong convention - #1932

Open
aaronkanzer wants to merge 1 commit into
NeuralEnsemble:masterfrom
aaronkanzer:fix/bci2000-scaling
Open

aaronkanzer wants to merge 1 commit into
NeuralEnsemble:masterfrom
aaronkanzer:fix/bci2000-scaling

Conversation

@aaronkanzer

Copy link
Copy Markdown

BCI2000RawIO has three problems that together make the scaled signals wrong for every channel.

  1. On NumPy 2.5, parse_header() raises TypeError: data type 'a64' not understood. The state definitions use the "a64" dtype alias, which was deprecated in NumPy 2.0 and removed in 2.5.
  2. SourceChGain values like 0.01617 are parsed as 0. Header values are matched with r"(-*\d+)(\w*)", which stops at the decimal point. The string fallback in _parse_header has the same problem.
  3. SourceChOffset is passed straight through as neo's offset. BCI2000 defines uV = (raw - SourceChOffset) * SourceChGain, while neo computes raw * gain + offset.

Repro with bci2000/eeg1_1.dat from ephy_testing_data. Its header has SourceChGain= 64 0.01617 0.01591 ... and SourceChOffset= 64 43 55 .... Run with neo 0.14.6 and NumPy 2.2 (on NumPy 2.5 it fails at parse_header()):

from neo.rawio.bci2000rawio import BCI2000RawIO
r = BCI2000RawIO(filename="eeg1_1.dat"); r.parse_header()
print(r.header["signal_channels"]["gain"][:3])
raw = r.get_analogsignal_chunk(0, 0, 0, 3, 0, channel_indexes=[0, 1, 2])
print(r.rescale_signal_raw_to_float(raw, dtype="float64", stream_index=0, channel_indexes=[0, 1, 2]))

Observed:

[0. 0. 0.]
[[43. 55. 69.]
 [43. 55. 69.]
 [43. 55. 69.]]

Every channel is a constant equal to its raw offset.

Expected: gains [0.01617 0.01591 0.01595], and channel 0 starting [-16.2185, 1.3745, -9.2331] uV. With the fix, the whole file matches BCI2000's own FileReader.py (the BCI2kReader package) to within 1.5e-5 uV.

Fix:

  • Use "S64" instead of "a64".
  • Parse float header values and the gain and offset strings with a pattern that keeps the decimal part. Integer parameters keep the old pattern.
  • Store offset = -SourceChOffset * SourceChGain.

The rawio test class also defined entities_to_download twice and never set entities_to_test, so the compliance tests never ran on these files. The test now lists the three files and adds a scaling check against values from BCI2kReader.

@zm711

zm711 commented Oct 24, 2026

Copy link
Copy Markdown
Contributor

@h-mayorquin could you review this? I don't think about regex at all so I trust your review of this.

As far as NumPy 2.5 we haven't officially started support that yet. I'm working on that now.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants