| Packfile URIs |
| ============= |
| |
| This feature allows servers to serve part of their packfile response as URIs. |
| This allows server designs that improve scalability in bandwidth and CPU usage |
| (for example, by serving some data through a CDN), and (in the future) provides |
| some measure of resumability to clients. |
| |
| This feature is available only in protocol version 2. |
| |
| Protocol |
| -------- |
| |
| The server advertises the `packfile-uris` capability. |
| |
| If the client then communicates which protocols (HTTPS, etc.) it supports with |
| a `packfile-uris` argument, the server MAY send a `packfile-uris` section |
| directly before the `packfile` section (right after `wanted-refs` if it is |
| sent) containing URIs of any of the given protocols. The URIs point to |
| packfiles that use only features that the client has declared that it supports |
| (e.g. ofs-delta and thin-pack). See linkgit:gitprotocol-v2[5] for the documentation of |
| this section. |
| |
| Clients should then download and index all the given URIs (in addition to |
| downloading and indexing the packfile given in the `packfile` section of the |
| response) before performing the connectivity check. |
| |
| Server design |
| ------------- |
| |
| The server can be trivially made compatible with the proposed protocol by |
| having it advertise `packfile-uris`, tolerating the client sending |
| `packfile-uris`, and never sending any `packfile-uris` section. But we should |
| include some sort of non-trivial implementation in the Minimum Viable Product, |
| at least so that we can test the client. |
| |
| This is the implementation: a feature, marked experimental, that allows the |
| server to be configured by one or more `uploadpack.blobPackfileUri= |
| <object-hash> <pack-hash> <uri>` entries. Whenever the list of objects to be |
| sent is assembled, all such blobs are excluded, replaced with URIs. As noted |
| in "Future work" below, the server can evolve in the future to support |
| excluding other objects (or other implementations of servers could be made |
| that support excluding other objects) without needing a protocol change, so |
| clients should not expect that packfiles downloaded in this way only contain |
| single blobs. |
| |
| Client design |
| ------------- |
| |
| The client has a config variable `fetch.uriprotocols` that determines which |
| protocols the end user is willing to use. By default, this is empty. |
| |
| When the client downloads the given URIs, it should store them with "keep" |
| files, just like it does with the packfile in the `packfile` section. These |
| additional "keep" files can only be removed after the refs have been updated - |
| just like the "keep" file for the packfile in the `packfile` section. |
| |
| The division of work (initial fetch + additional URIs) introduces convenient |
| points for resumption of an interrupted clone - such resumption can be done |
| after the Minimum Viable Product (see "Future work"). |
| |
| Future work |
| ----------- |
| |
| The protocol design allows some evolution of the server and client without any |
| need for protocol changes, so only a small-scoped design is included here to |
| form the MVP. For example, the following can be done: |
| |
| * On the server, more sophisticated means of excluding objects (e.g. by |
| specifying a commit to represent that commit and all objects that it |
| references). |
| * On the client, resumption of clone. If a clone is interrupted, information |
| could be recorded in the repository's config and a "clone-resume" command |
| can resume the clone in progress. (Resumption of subsequent fetches is more |
| difficult because that must deal with the user wanting to use the repository |
| even after the fetch was interrupted.) |
| |
| There are some possible features that will require a change in protocol: |
| |
| * Additional HTTP headers (e.g. authentication) |
| * Byte range support |
| * Different file formats referenced by URIs (e.g. raw object) |
