ARGON
Check-in [e6c37344e9]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:wolfram: FIXME about lightweight job generation interface
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1:e6c37344e928750b5b6ae023e182135de9c3549b
User & Date: alaric 2012-07-26 08:36:41
Context
2012-11-30
11:51
Mainly NITROGEN - documenting the node lifecycle state machine. Updated other sections to refer properly to it. Removed the bootstrap code from the ARGON page as it's all been eaten up by NITROGEN. check-in: 309ad96ecf user: alaric tags: trunk
2012-07-26
08:36
wolfram: FIXME about lightweight job generation interface check-in: e6c37344e9 user: alaric tags: trunk
2012-07-23
16:31
Noted that one can schedule SINGLE jobs as well as job generators in HELIUM and WOLFRAM. check-in: 5cfd191570 user: alaric tags: trunk
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Changes to intro/wolfram.wiki.

327
328
329
330
331
332
333






334
335
336
337
338
339
340
all of the jobs have completed, the distributed job as a whole is
complete. The job generator system basically implements a lazy list of
jobs to do, allowing it to be computed on the fly. And for
parallelisable tasks that aren't generated in bulk, it will be
possible to submit a single job to be run, as a LITHIUM handler ID and
arguments, which may be distributed to a lightly-loaded node in the
cluster to be run if the local node is busy.







<h1>Distributed caching</h1>

WOLFRAM also provides a distributed cache mechanism.

There are several caches in the system. As well as one cache created
implicitly per node, which is stored only on that node, and a global







>
>
>
>
>
>







327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
all of the jobs have completed, the distributed job as a whole is
complete. The job generator system basically implements a lazy list of
jobs to do, allowing it to be computed on the fly. And for
parallelisable tasks that aren't generated in bulk, it will be
possible to submit a single job to be run, as a LITHIUM handler ID and
arguments, which may be distributed to a lightly-loaded node in the
cluster to be run if the local node is busy.

FIXME: Is it worth having a lighter-weight job generator interface
where you provide a local next-job-please closure, and remote nodes
call back via WOLFRAM to ask for new jobs? Actually distributing job
generation (causing the job generator state to be synchronised at the
TUNGSTEN level) might be too heavyweight.

<h1>Distributed caching</h1>

WOLFRAM also provides a distributed cache mechanism.

There are several caches in the system. As well as one cache created
implicitly per node, which is stored only on that node, and a global