Send feeds when generated HTML grows too long

This allows the GSA to still discover URLs that are past the point that
it actually indexes.

This code does suffer from an issue where a failure to write the HTML
prevents the feeds from being sent. The full solution would probably
entail stopping the generation of HTML and in a separate thread
continuing to loop through results.
6 files changed