Skip to content

fix(config): decode escapes in unquoted config values - #2297

Open
JingHao-Leon wants to merge 1 commit into
gitpython-developers:mainfrom
JingHao-Leon:fix/config-unquoted-escape-parsing
Open

JingHao-Leon wants to merge 1 commit into
gitpython-developers:mainfrom
JingHao-Leon:fix/config-unquoted-escape-parsing

Conversation

@JingHao-Leon

@JingHao-Leon JingHao-Leon commented Oct 19, 2026 •

Copy link
Copy Markdown

This fix and its description were drafted with AI assistance (Codex) and are agent-authored; no statement here is made on the account owner's behalf about review or approval.

Description

GitConfigParser leaves backslash escapes literal in unquoted config values, so it disagrees with Git on the same file. Quoted values are decoded correctly; unquoted ones are not.

config git config --get a.b GitPython before
b = first\nsecond first<LF>second first\nsecond
b = first\tsecond first<TAB>second first\tsecond
b = foo\\bar foo\bar foo\\bar
b = ab\"cd ab"cd ab\"cd

The value also decoded differently depending only on whether it was wrapped, which is the inconsistency that identifies the cause: b = first\nsecond stayed literal, while b = first\nsecond \ continued onto a second line was decoded.

This also made read-modify-write corrupt config files. The parsed value held no control character, so the writer's CR/LF/NUL guard never triggered, and saving an unrelated key changed what Git read:

original:   [a]\n\tb = x\\ny\n
git reads:  x<LF>y
GitPython:  x\ny          (literal backslash, no control char)
rewritten:  [a]\n\tb = "x\\\\ny\\\n"\n
git reads:  x\\ny         (value changed meaning)

After the fix GitPython reads x<LF>y like Git does, and the existing writer guard rejects the control character instead of corrupting the file.

Root cause

git/config.py:624 — parse_value() was called only when continued was set:

        continued = False
        while True:
            if not is_line_continuation(optval):
                break
            ...
            continued = True
        if continued:
            optval = parse_value(optval)

parse_value() is what decodes escapes, strips unquoted comments and trims whitespace. Gating it on continued made the decode depend on line wrapping, so a single-line unquoted value skipped it while a continued one got it.

Why the decode is conditional on the escapes being known

Git decodes the standard escape sequences in unquoted values as well, but it is not accurate to say it treats unquoted values exactly as it treats quoted ones: for an escape it does not recognise, git rejects the line outright.

$ printf '[a]\n\tb = C:\\Users\n' > c
$ GIT_CONFIG_GLOBAL=c GIT_CONFIG_SYSTEM=/dev/null git config --get a.b
fatal: bad config line 2 in file c

So path = C:\Users\me is not a line git can read at all. GitPython is deliberately more tolerant and keeps such backslashes literal, which is what lets a Windows path survive a round-trip, and existing tests depend on it.

Decoding unconditionally regresses that tolerance. In C:\Users\RUNNER~1\AppData\Local\Temp\test_foo, the \t of \Temp and \test became tabs while the \U of \Users stayed literal, so the path no longer matched. That is not hypothetical: it turned four Windows jobs red on the first push of this branch —

test_config.py::TestBase::test_conditional_includes_from_git_dir
test_config.py::TestBase::test_conditional_includes_from_branch_name
test_config.py::TestBase::test_conditional_includes_remote_url
test_config.py::TestBase::test_multiple_include_paths_with_same_key

I did not touch those tests to match the new behaviour, since that would only hide the regression. Instead parse_value() now runs only when every backslash in the value introduces a known escape:

  • first\nsecond, a\tb, foo\\bar — all backslashes known, so decoded as git does.
  • C:\Users\...\Temp\test_foo — contains the unknown \U, so the whole value stays literal and the path is preserved.

The escape table moved to a module-level UNQUOTED_ESCAPES so the decoder and the new escapes_are_known() guard cannot drift apart. A trailing backslash is treated as known, since git drops it rather than treating it as an unknown escape; without that, a value ending in \ after a continuation would stop being comment-stripped.

One existing expectation moves with this: test_backslash_line_continuation asserted k = val\\ followed by a newline reads as val\\. Git yields val\, since the second backslash escapes the first and there is no continuation, so that case now asserts the value Git actually produces.

Validation

All figures are from this branch against git 2.50.1 (Apple Git-155). Every expected value in the new fixture was first read out of real git config --get rather than assumed.

New regression test, red before the fix:

$ PYTHONPATH="$PWD" python -m pytest test/test_config.py::TestBase::test_config_with_unquoted_escapes -q
E       AssertionError: 'first\\nsecond' != 'first\nsecond'
1 failed in 0.21s

Config suite after the fix:

$ PYTHONPATH="$PWD" python -m pytest test/test_config.py -q -p no:randomly --no-cov
54 passed, 2 skipped, 29 subtests passed in 1.64s

(before the change: 53 passed, 2 skipped — the extra test is the new one.)

Full suite, same tree, before and after:

before:  870 passed, 76 skipped, 913 errors, 53 subtests passed  (260.81s)
after:   871 passed, 76 skipped, 913 errors, 53 subtests passed  (253.28s)

The +1 is the new test; skipped and error counts are identical. The 913 errors are environmental and not related to this change: the sandbox clone is blobless, so test/lib/helper.py::TestBase.setUpClass cannot materialise historical blobs for its two dependency fixtures and git gc aborts. Those same 913 errors are present on unmodified main, and the lazy-fetch path was disabled on both runs so neither hung.

Lint, using the rule set from pyproject.toml (E, W, F, B, C4, TC004):

$ python -m ruff check --config /tmp/ruff_real.toml git/config.py test/test_config.py
All checks passed!
$ python -m ruff format --check --config /tmp/ruff_real.toml git/config.py test/test_config.py
2 files already formatted

What I could not verify locally

I am on macOS, so the Windows path handling that drove the design above is not reproducible here — POSIX paths contain no backslashes. I checked the mechanism by feeding the exact Windows temp paths through GitConfigParser directly on this machine and confirming they now round-trip unchanged:

C:\Users\RUNNER~1\AppData\Local\Temp\test_conditional_includes_from_git_dirjyqm_zla\config2
C:\Users\RUNNER~1\AppData\Local\Temp\test_multiple_include_paths_with_same_keyabc\config2
C:\Users\RUNNER~1\AppData\Local\Temp\test_conditional_includes_remote_urlxyz\config2

All three round-trip exactly. That exercises the same escapes_are_known() branch the four failing tests depend on, but it is not a substitute for running the Windows jobs, so the Windows CI result on this push is the real check.

No changelog entry: CONTRIBUTING.md does not ask for one, doc/source/changes.rst has no unreleased section, and the comparable config fixes (#2213, #2245, #2227, #2211) did not touch it.

Fixes #2296

@JingHao-Leon
JingHao-Leon marked this pull request as draft October 19, 2026 11:07
Git decodes the standard escape sequences in unquoted config values, but
`GitConfigParser._read` only called `parse_value()` when a value continued
onto a following line. A single-line unquoted value therefore kept its
escapes literal, so `b = first\nsecond` read back as `first\nsecond`
rather than `first<LF>second`.

The same value also decoded differently depending only on whether it was
wrapped: `b = first\nsecond \` continued onto a second line already went
through `parse_value()` and was decoded.

The missing decode made writes unsafe. The parsed value held no control
character, so the writer's CR/LF/NUL guard did not trigger, and saving an
unrelated key emitted a value that git then read differently than before.
With the escape decoded, the existing guard rejects the value instead of
corrupting the file.

A value is decoded only when every backslash in it introduces a known
escape. Git rejects an unknown escape outright, so GitPython never had to
handle one; it tolerates them instead, which is what keeps a Windows path
such as `C:\Users\me` usable. Decoding unconditionally would have broken
that: the `\t` of `\Temp\test_foo` would become a tab while the `\U` of
`\Users` stayed literal, mangling the path. Guarding on the whole value
decodes `a\tb` and `foo\\bar` as git does and leaves the path alone.

An existing case in `test_backslash_line_continuation` asserted the
undecoded spelling `val\\` for `k = val\\` followed by a newline. Git
yields `val\` there, since the second backslash escapes the first, so
that expectation is corrected to match.

Checked against git 2.50.1 (Apple Git-155).

Co-Authored-By: Codex <codex@openai.com>
@JingHao-Leon
JingHao-Leon force-pushed the fix/config-unquoted-escape-parsing branch from fd7027c to 983e182 Compare October 19, 2026 11:16
@JingHao-Leon
JingHao-Leon marked this pull request as ready for review October 19, 2026 12:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

GitConfigParser does not decode backslash escapes in unquoted config values

1 participant