packfile: fix corruption due to stale delta base cache entries
The delta base cache is a process-global hashmap that is keyed by the
address of the `struct packed_git` plus the offset of the base object
within that pack. Entries part of the cache are never removed when a
pack is closed, and neither when the pack is subsequently freed. As a
consequence, the cache may contain stale entries.
For a long time, the worst consequence of this leaking cache was that we
held on to memory that we could've released. But the reason for this was
that we didn't even free the packfiles, either. That has changed in
6f1e9394e2 (object: fix leaking packfiles when closing object store,
2024-08-08), where we plugged that leak.
Now that we free them, a new packfile may be allocated using the exact
same address as a previously allocated one. And if the new packfile has
both the same address and a similar layout, it may happen that a
preexisting entry from a previously-allocated in the delta base cache
would have the exact same key.
All of this sounds very theoretical, but we can actually trigger this
bug somewhat reliably! When doing a merge in a repository with lots of
submodules that have similar-looking packfiles we end up opening and
then closing the object databases of each of the submodules in sequence.
Because of the above mentioned commit we would close and free each of
the packfiles part of the respective databases, but we wouldn't evict
their delta base entries from the cache.
When using glibc, one of the packfiles will eventually get the exact
same address, and that will then cause Git to read the wrong entry from
the cache. Git detects this and aborts with an error:
$ git merge branch-b
error: Could not read 584ef938be4a749bfa13f68d5ac5545bc029e529
error: could not parse commit 584ef938be4a749bfa13f68d5ac5545bc029e529
error: failed to merge submodule G (repository corrupt)
Now in this case we're lucky that Git detects this error because we try
to read a commit from a different submodule via an object database that
doesn't have it. But potentially, in an even more contrived scenario, we
might even silently yield wrong data from the cache.
Fix this bug by evicting cache entries that belong to a specific pack
when closing it.
Note that the added test reliably reproduces the above bug on my machine
that uses NixOS at c59305bab206 (cosmic-applets: add missing runtime
dependency (#566040), 2026-10-01) with glibc 2.44-25. But as we rely on
specific allocation behaviour of glibc it is very likely that the test
will not work on other platforms.
Reported-by: Guillaume Chauvel <guillaume.chauvel@gmail.com>
Helped-by: Philippe Blain <levraiphilippeblain@gmail.com>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
diff --git a/packfile.c b/packfile.c
index af1b837..365c54c 100644
--- a/packfile.c
+++ b/packfile.c
@@ -1253,6 +1253,18 @@ void clear_delta_base_cache(void)
}
}
+static void delta_base_cache_evict_entry(struct packed_git *p)
+{
+ struct list_head *lru, *tmp;
+
+ list_for_each_safe(lru, tmp, &delta_base_cache_lru) {
+ struct delta_base_cache_entry *entry =
+ list_entry(lru, struct delta_base_cache_entry, lru);
+ if (entry->key.p == p)
+ release_delta_base_cache(entry);
+ }
+}
+
void close_pack(struct packed_git *p)
{
close_pack_windows(p);
@@ -1261,6 +1273,7 @@ void close_pack(struct packed_git *p)
close_pack_revindex(p);
close_pack_mtimes(p);
oidset_clear(&p->bad_objects);
+ delta_base_cache_evict_entry(p);
}
static void add_delta_base_cache(struct packed_git *p, off_t base_offset,
diff --git a/t/t6437-submodule-merge.sh b/t/t6437-submodule-merge.sh
index 1546d5f..0ee3684 100755
--- a/t/t6437-submodule-merge.sh
+++ b/t/t6437-submodule-merge.sh
@@ -514,4 +514,51 @@
)
'
+test_expect_success 'merge with many packed submodules reports conflicts' '
+ test_config_global protocol.file.allow always &&
+
+ # Create 16 submodules with two divergent branches each.
+ submodules="A B C D E F G H I J K L M N O P" &&
+ for name in $submodules
+ do
+ git init source-$name &&
+ test_commit -C source-$name $name-main &&
+ git -C source-$name switch --create branch-a main &&
+ git -C source-$name commit --allow-empty --message $name-branch-a &&
+ git -C source-$name switch --create branch-b main &&
+ git -C source-$name commit --allow-empty --message $name-branch-b || return 1
+ done &&
+
+ # Create the superproject and add all submodules.
+ git init many-packed &&
+ for name in $submodules
+ do
+ git -C many-packed submodule add --branch main "file://$PWD/source-$name" $name || return 1
+ done &&
+ git -C many-packed commit --message main &&
+
+ # Create two divergent commits in the superproject that update all
+ # submodules to the divergent branches.
+ for branch in branch-a branch-b
+ do
+ git -C many-packed switch -c $branch main &&
+ for name in $submodules
+ do
+ git -C many-packed/$name switch $branch || return 1
+ done &&
+ git -C many-packed add $submodules &&
+ git -C many-packed commit --message $branch || return 1
+ done &&
+
+ # Clone the superproject to ensure that everything is well-packed and
+ # then merge the two branches, creating conflicts for every submodule.
+ git clone many-packed many-packed-clone &&
+ git -C many-packed-clone submodule update --init &&
+ git -C many-packed-clone switch branch-a &&
+ test_expect_code 1 git -C many-packed-clone -c advice.submoduleMergeConflict=false merge branch-b >out 2>err &&
+ grep "^CONFLICT (submodule)" out >conflicts &&
+ test_line_count = 16 conflicts &&
+ test_must_be_empty err
+'
+
test_done
