fetch: write commit-graph using updated refs only
When fetch.writeCommitGraph was introduced in 50f26bd035 (fetch: add
fetch.writeCommitGraph config setting, 2019-09-02), the stated goal
was to stay updated with the latest commits after fetching new
objects. The implementation used write_commit_graph_reachable()
because it was the only API available, but two things have changed
since then:
1. write_commit_graph() was added, and it accepts an explicit set of
commits as seeds, enabling more targeted commit-graph updates.
2. The ref-scanning callback add_ref_to_set() became more expensive
in 630cd5194e (commit-graph.c: peel refs in 'add_ref_to_set',
2020-07-22) when it started to validate the refs against the odb
for correctness. On a repository with many refs, this makes the
full reachable scan unnecessarily costly for a targeted fetch.
Optimize the commit-graph write by using only the newly updated refs
as seeds instead of scanning all refs after every fetch. To keep
this change small, skip the optimization for multi-remote fetches
(since that would require propagating the set of refs across process
boundaries).
This relies on the commit-graph write being additive, keeping the
commits that are already in the graph. fetch already operates in
this mode (COMMIT_GRAPH_WRITE_SPLIT) and now that becomes
required for correctness. Without that mode, the write would
replace the commit-graph and lose other commits.
After fetch_one() returns, call prepare_commit_graph() (which is
made non-static by this commit) to determine the graph-write mode:
- If no commit-graph exists yet, fall back to the full reachable
scan so the first graph creation covers all refs.
- If a commit-graph exists and the fetch updated at least one ref,
write incrementally using only the new refs as seeds.
- If a commit-graph exists but the fetch is a no-op, skip the
commit-graph write entirely.
- For the multi-remote path (fetch --all), where child processes
do the actual fetching, fall back to the full reachable scan.
Full commit-graph coverage of all refs remains the responsibility
of "git maintenance", "git gc" and "git commit-graph write".
Regular Git operations may trigger "git maintenance run --auto",
which periodically rebuilds the commit-graph from all reachable
refs.
The effect was measured on a synthetic setup: git.git with 200K
extra packed refs (~206K total), a local file:// remote, an existing
split commit-graph and a warm page cache. The times below are the
median of 9 runs of the trace2 region fetch/write-commit-graph:
scenario before after
no-op fetch 380 ms (skipped)
1 ref updated 357 ms 9.3 ms
10 refs updated 359 ms 8.9 ms
Signed-off-by: Kristofer Karlsson <krka@spotify.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Git is a fast, scalable, distributed revision control system with an unusually rich command set that provides both high-level operations and full access to internals.
Git is an Open Source project covered by the GNU General Public License version 2 (some parts of it are under different licenses, compatible with the GPLv2). It was originally written by Linus Torvalds with help of a group of hackers around the net.
Please read the file INSTALL for installation instructions.
Many Git online resources are accessible from https://git-scm.com/ including full documentation and Git related tools.
See Documentation/gittutorial.adoc to get started, then see Documentation/giteveryday.adoc for a useful minimum set of commands, and Documentation/git-<commandname>.adoc for documentation of each command. If git has been correctly installed, then the tutorial can also be read with man gittutorial or git help tutorial, and the documentation of each command with man git-<commandname> or git help <commandname>.
CVS users may also want to read Documentation/gitcvs-migration.adoc (man gitcvs-migration or git help cvs-migration if git is installed).
The user discussion and development of Git take place on the Git mailing list -- everyone is welcome to post bug reports, feature requests, comments and patches to git@vger.kernel.org (read Documentation/SubmittingPatches for instructions on patch submission and Documentation/CodingGuidelines).
Those wishing to help with error message, usage and informational message string translations (localization l10) should see po/README.md (a po file is a Portable Object file that holds the translations).
To subscribe to the list, send an email to git+subscribe@vger.kernel.org (see https://subspace.kernel.org/subscribing.html for details). The mailing list archives are available at https://lore.kernel.org/git/, https://marc.info/?l=git and other archival sites.
Issues which are security relevant should be disclosed privately to the Git Security mailing list git-security@googlegroups.com.
The maintainer frequently sends the “What's cooking” reports that list the current status of various development topics to the mailing list. The discussion following them give a good reference for project status, development direction and remaining tasks.
The name “git” was given by Linus Torvalds when he wrote the very first version. He described the tool as “the stupid content tracker” and the name as (depending on your mood):