tree 3bcd7e6d904cb858824f1f806f911615359f730e
parent 72e75ffee70f2e898ae687734aa109f559b39e5c
author Kristofer Karlsson <krka@spotify.com> 1791382977 +0000
committer Junio C Hamano <gitster@pobox.com> 1791391848 -0700

fetch: write commit-graph using updated refs only

When fetch.writeCommitGraph was introduced in 50f26bd035 (fetch: add
fetch.writeCommitGraph config setting, 2019-09-02), the stated goal
was to stay updated with the latest commits after fetching new
objects.  The implementation used write_commit_graph_reachable()
because it was the only API available, but two things have changed
since then:

 1. write_commit_graph() was added, and it accepts an explicit set of
    commits as seeds, enabling more targeted commit-graph updates.

 2. The ref-scanning callback add_ref_to_set() became more expensive
    in 630cd5194e (commit-graph.c: peel refs in 'add_ref_to_set',
    2020-07-22) when it started to validate the refs against the odb
    for correctness.  On a repository with many refs, this makes the
    full reachable scan unnecessarily costly for a targeted fetch.

Optimize the commit-graph write by using only the newly updated refs
as seeds instead of scanning all refs after every fetch.  To keep
this change small, skip the optimization for multi-remote fetches
(since that would require propagating the set of refs across process
boundaries).

This relies on the commit-graph write being additive, keeping the
commits that are already in the graph.  fetch already operates in
this mode (COMMIT_GRAPH_WRITE_SPLIT) and now that becomes
required for correctness.  Without that mode, the write would
replace the commit-graph and lose other commits.

After fetch_one() returns, call prepare_commit_graph() (which is
made non-static by this commit) to determine the graph-write mode:

 - If no commit-graph exists yet, fall back to the full reachable
   scan so the first graph creation covers all refs.

 - If a commit-graph exists and the fetch updated at least one ref,
   write incrementally using only the new refs as seeds.

 - If a commit-graph exists but the fetch is a no-op, skip the
   commit-graph write entirely.

 - For the multi-remote path (fetch --all), where child processes
   do the actual fetching, fall back to the full reachable scan.

Full commit-graph coverage of all refs remains the responsibility
of "git maintenance", "git gc" and "git commit-graph write".
Regular Git operations may trigger "git maintenance run --auto",
which periodically rebuilds the commit-graph from all reachable
refs.

The effect was measured on a synthetic setup: git.git with 200K
extra packed refs (~206K total), a local file:// remote, an existing
split commit-graph and a warm page cache.  The times below are the
median of 9 runs of the trace2 region fetch/write-commit-graph:

    scenario          before    after
    no-op fetch       380 ms    (skipped)
    1 ref updated     357 ms    9.3 ms
    10 refs updated   359 ms    8.9 ms

Signed-off-by: Kristofer Karlsson <krka@spotify.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>