D 2020-05-28T14:31:47.322
J icomment If\swe\suse\sbokbok\sas\sa\sbackend\saccess\sprotocol,\swe\scan\soverlap\srequests\sto\sthe\sbackend.\r\n\r\nGiven\sthat,\swe\scould\sparallelise\swhen\siterating\sover\scontents\sof\sa\sdirectory,\sand\seffectively\sextract\s(or\ssnapshot,\sfor\sthat\smatter)\sup\sto\sN\sthings\sat\sonce,\sfor\ssome\sconfigurable\sN.\r\n\r\nFor\sindirect\sblocks,\swe\scould\salso\sparallelise\sfetching\sthe\sdata\sblocks\s(probably\snot\sworth\sparallelising\sfetching\sindirect\sblocks\sthemselves),\salthough\swe'll\shave\sto\sput\sthem\sback\sinto\sorder\sif\sthey\sarrive\sout\sof\sorder.\r\n\r\nTo\sdo\sthis\snicely,\swe\sshould\swrite\sa\snice\sparallel-for-each\sfor\sthe\sdirectory\scase,\sand\sa\sparallel-for-map\sthat\sdoes\sa\sparallel\smap\sof\sclosure\s1\sover\ssome\slist,\sthen\sexecutes\sclosure\s2\son\seach\sresult\s*in\ssequence*,\staking\smaximum\sparallelism\sfrom\sa\sparameter.\sMake\ssure\sthat\snested\sparallel-*\sthings\suse\sthe\ssame\sparallelism\slimit\s(implement\sit\swith\sa\sglobal\smutex-protected\scounter\sor\ssomething)\sso\swe\sdon't\sgo\scrazy\swith\srecursions.
J login alaric
J mimetype text/x-fossil-wiki
K 97b608385736f7f946bc3b3b2a037b0b8945306e
U alaric
Z e9fa604e51aaf7d8593ed203b207378e
