)]}'
{
  "commit": "83558686ceaeb634b630d5177818d571cafafbf4",
  "tree": "afae7f9221fe3b09104b6e5791bed05e49f1c357",
  "parents": [
    "6b4cd2f82791b0c9726616832382e0af8d895d99"
  ],
  "author": {
    "name": "Jeff King",
    "email": "peff@peff.net",
    "time": "Fri Jul 15 06:43:47 2016 -0400"
  },
  "committer": {
    "name": "Junio C Hamano",
    "email": "gitster@pobox.com",
    "time": "Wed Jul 20 12:11:11 2016 -0700"
  },
  "message": "receive-pack: send keepalives during quiet periods\n\nAfter a client has sent us the complete pack, we may spend\nsome time processing the data and running hooks. If the\nclient asked us to be quiet, receive-pack won\u0027t send any\nprogress data during the index-pack or connectivity-check\nsteps. And hooks may or may not produce their own progress\noutput. In these cases, the network connection is totally\nsilent from both ends.\n\nGit itself doesn\u0027t care about this (it will wait forever),\nbut other parts of the system (e.g., firewalls,\nload-balancers, etc) might hang up the connection. So we\u0027d\nlike to send some sort of keepalive to let the network and\nthe client side know that we\u0027re still alive and processing.\n\nWe can use the same trick we did in 05e9515 (upload-pack:\nsend keepalive packets during pack computation, 2013-09-08).\nNamely, we will send an empty sideband data packet every `N`\nseconds that we do not relay any stderr data over the\nsideband channel. As with 05e9515, this means that we won\u0027t\nbother sending keepalives when there\u0027s actual progress data,\nbut will kick in when it has been disabled (or if there is a\nlull in the progress data).\n\nThe concept is simple, but the details are subtle enough\nthat they need discussing here.\n\nBefore the client sends us the pack, we don\u0027t want to do any\nkeepalives. We\u0027ll have sent our ref advertisement, and we\u0027re\nwaiting for them to send us the pack (and tell us that they\nsupport sidebands at all).\n\nWhile we\u0027re receiving the pack from the client (or waiting\nfor it to start), there\u0027s no need for keepalives; it\u0027s up to\nthem to keep the connection active by sending data.\nMoreover, it would be wrong for us to do so. When we are the\nserver in the smart-http protocol, we must treat our\nconnection as half-duplex. So any keepalives we send while\nreceiving the pack would potentially be buffered by the\nwebserver. Not only does this make them useless (since they\nwould not be delivered in a timely manner), but it could\nactually cause a deadlock if we fill up the buffer with\nkeepalives. (It wouldn\u0027t be wrong to send keepalives in this\nphase for a full-duplex connection like ssh; it\u0027s simply\npointless, as it is the client\u0027s responsibility to speak).\n\nAs soon as we\u0027ve gotten all of the pack data, then the\nclient is waiting for us to speak, and we should start\nkeepalives immediately. From here until the end of the\nconnection, we send one any time we are not otherwise\nsending data.\n\nBut there\u0027s a catch. Receive-pack doesn\u0027t know the moment\nwe\u0027ve gotten all the data. It passes the descriptor to\nindex-pack, who reads all of the data, and then starts\nresolving the deltas. We have to communicate that back.\n\nTo make this work, we instruct the sideband muxer to enable\nkeepalives in three phases:\n\n  1. In the beginning, not at all.\n\n  2. While reading from index-pack, wait for a signal\n     indicating end-of-input, and then start them.\n\n  3. Afterwards, always.\n\nThe signal from index-pack in phase 2 has to come over the\nstderr channel which the muxer is reading. We can\u0027t use an\nextra pipe because the portable run-command interface only\ngives us stderr and stdout.\n\nStdout is already used to pass the .keep filename back to\nreceive-pack. We could also send a signal there, but then we\nwould find out about it in the main thread. And the\nkeepalive needs to be done by the async muxer thread (since\nit\u0027s the one writing sideband data back to the client). And\nwe can\u0027t reliably signal the async thread from the main\nthread, because the async code sometimes uses threads and\nsometimes uses forked processes.\n\nTherefore the signal must come over the stderr channel,\nwhere it may be interspersed with other random\nhuman-readable messages from index-pack. This patch makes\nthe signal a single NUL byte.  This is easy to parse, should\nnot appear in any normal stderr output, and we don\u0027t have to\nworry about any timing issues (like seeing half the signal\nbytes in one read(), and half in a subsequent one).\n\nThis is a bit ugly, but it\u0027s simple to code and should work\nreliably.\n\nAnother option would be to stop using an async thread for\nmuxing entirely, and just poll() both stderr and stdout of\nindex-pack from the main thread. This would work for\nindex-pack (because we aren\u0027t doing anything useful in the\nmain thread while it runs anyway). But it would make the\nconnectivity check and the hook muxers much more\ncomplicated, as they need to simultaneously feed the\nsub-programs while reading their stderr.\n\nThe index-pack phase is the only one that needs this\nsignaling, so it could simply behave differently than the\nother two. That would mean having two separate\nimplementations of copy_to_sideband (and the keepalive\ncode), though. And it still doesn\u0027t get rid of the\nsignaling; it just means we can write a nicer message like\n\"END_OF_INPUT\" or something on stdout, since we don\u0027t have\nto worry about separating it from the stderr cruft.\n\nOne final note: this signaling trick is only done with\nindex-pack, not with unpack-objects. There\u0027s no point in\ndoing it for the latter, because by definition it only kicks\nin for a small number of objects, where keepalives are not\nas useful (and this conveniently lets us avoid duplicating\nthe implementation).\n\nSigned-off-by: Jeff King \u003cpeff@peff.net\u003e\nSigned-off-by: Junio C Hamano \u003cgitster@pobox.com\u003e\n",
  "tree_diff": [
    {
      "type": "modify",
      "old_id": "e455faeb2350fa2f10ee4869fdc2f364571876af",
      "old_mode": 33188,
      "old_path": "Documentation/config.txt",
      "new_id": "25c8e3659bbb2956c2a31047be82867f8b483c85",
      "new_mode": 33188,
      "new_path": "Documentation/config.txt"
    },
    {
      "type": "modify",
      "old_id": "1cba12063ad2bb7ec8e80b6deb2d5c7ad195cff4",
      "old_mode": 33188,
      "old_path": "builtin/index-pack.c",
      "new_id": "54f2cfbd6e4ebb58beeaae19cc3ef56e0882a0e9",
      "new_mode": 33188,
      "new_path": "builtin/index-pack.c"
    },
    {
      "type": "modify",
      "old_id": "7db16392798e00a7a4f4d4e09031fd0815de313b",
      "old_mode": 33188,
      "old_path": "builtin/receive-pack.c",
      "new_id": "e41f55f4f53e57dd294135d7799768d3aa235b18",
      "new_mode": 33188,
      "new_path": "builtin/receive-pack.c"
    }
  ]
}