Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
Comment: | Put in links to in-progress intro pages, so I remember they exist.
Elaborated somewhat on carbon and iodine. |
---|---|
Timelines: | family | ancestors | descendants | both | trunk |
Files: | files | file ages | folders |
SHA1: |
5b2e596027ca5fc978e716a282c001dd |
User & Date: | alaric 2012-12-06 13:29:15 |
Context
2012-12-06
| ||
20:23 | More elaboration of CARBON check-in: f22fe263a5 user: alaric tags: trunk | |
13:29 |
Put in links to in-progress intro pages, so I remember they exist.
Elaborated somewhat on carbon and iodine. check-in: 5b2e596027 user: alaric tags: trunk | |
2012-11-30
| ||
13:00 | Added more detail to NITROGEN about failure modes and special cases, sorted out the reference linking, and clarified what states real-time tasks and device drivers run in. check-in: 7ae135dc3e user: alaric tags: trunk | |
Changes
Changes to README.wiki.
︙ | ︙ | |||
211 212 213 214 215 216 217 | but because Mercury was the name of the Roman messenger god.</dd> <dt>[./intro/caesium.wiki|CAESIUM]</dt> <dd>Entities might also need to do things without being asked externally - so CAESIUM provides a distributed scheduler, invoking entity entry points using LITHIUM according to a schedule. The name is a nod to the use of the element Caesium inside atomic | | | | > | > > > | | | | | | > | > | 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 | but because Mercury was the name of the Roman messenger god.</dd> <dt>[./intro/caesium.wiki|CAESIUM]</dt> <dd>Entities might also need to do things without being asked externally - so CAESIUM provides a distributed scheduler, invoking entity entry points using LITHIUM according to a schedule. The name is a nod to the use of the element Caesium inside atomic clocks. [http://argon.org.uk/caesium.html|old site page]</dd> <dt>CARBON</dt> <dd>TUNGSTEN stores entity state as sets of tuples; CARBON adds an inference engine on top of that (like a PROLOG implementation) to allow high-level querying. CARBON can obtain data from in-memory temporary tuple sets, TUNGSTEN data (via WOLFRAM), or from tuple stores published via MERCURY. The publishing of tuple stores is such a performance-critical operation that special support for it is included in the implementation of MERCURY and TUNGSTEN to provide low-latency high-throughput access in common cases; this is ARGON's equivalent of the "sendfile" system call... The name is a play on the way that Carbon is used by organic systems to form complex structures, by analogy with the naming of IRON, but emphasising the complexity and versatility of the structures rather than brute strength. [http://www.snell-pym.org.uk/archives/2009/08/20/designing-a-global-knowledge-base/|conceptual design] [./intro/carbon.wiki|work in progress on the intro page]</dd> </dl> <h2>Userland</h2> <dl> <dt>AURUM</dt> <dd>HELIUM meters out memory and CPU time at run-time, within the context of a single thread; and TUNGSTEN meters out persistent storage space within the context of a single entity. However, a higher-level infrastructure is needed to allow management of those resources, and to manage other finite resources. AURUM is a set of software libraries for managing them, including distributed operation by communicating with "bank entities" via MERCURY. The name is, of course, a play on the use of gold as a medium of exchange. [http://argon.org.uk/aurum.html|old site page], [http://www.snell-pym.org.uk/archives/2011/05/13/aurum/|some blog thoughts], [./intro/aurum.wiki|work in progress on the intro page]</dd> <dt>FLUORINE</dt> <dd>All of these lovely entities communicating via MERCURY are lovely, but what about the rest of the world? FLUORINE is a toolkit for communicating with standard Internet protocols and data formats. The name is a play on the element Fluorine's extreme electronegativity, which allows it to bind aggressively to other molecules. [http://argon.org.uk/fluorine.html|old site page], [./intro/fluorine.wiki|work in progress on the intro page]</dd> <dt>IODINE</dt> <dd>This is a library of standard MERCURY/CARBON interfaces for entities to provide user-level actions, such as sending things to places. The name is a weak reference to the fact that many of the interfaces relate to user agents that represent human beings, and the letter "I" meaning personal identity. [http://argon.org.uk/iodine.html|old site page], [./intro/iodine.wiki|work in progress on the intro page]</dd> <dt>NEON</dt> <dd>Part of IODINE is the concept of a "user entity" that represents a human being in the world of ARGON, but how does a human user actually get the user entity to act on their behalf? NEON provides an interface whereby a human interface device such as a laptop computer, tablet, or mobile phone can let a user authenticate to a user agent, then allow the user agent to control the human interface hardware in order to present an interface to the user. This interface can be delegated in a controlled manner, thereby allowing other entities to present an interface (mediated by the user entity) to the user. The name is a reference to the use of neon in brightly coloured electrical signs, as graphical interfacing is part of its responsibilities. [http://argon.org.uk/neon.html|old site page], [http://www.snell-pym.org.uk/archives/2006/08/23/neon/|some blog thoughts], [http://www.snell-pym.org.uk/archives/2010/06/09/user-interfaces-for-event-streams/|some blog thoughts about events], [http://www.snell-pym.org.uk/archives/2009/12/16/personal-information-management/|some blog thoughts about personal information management], [./intro/iodine.wiki|work in progress on the intro page].</dd> </dl> <h1>Development roadmap</h1> ARGON has been designed with sufficient modularity to allow it to be constructed incrementally - in stages. Here's an approximate road map |
︙ | ︙ |
Changes to intro/aurum.wiki.
1 | Updated content will appear here shortly! | > > | 1 2 3 | <h1>WORK IN PROGRESS</h1> Updated content will appear here shortly! |
Changes to intro/carbon.wiki.
1 2 3 4 5 6 7 | While [./iron.wiki|IRON] is the model for individual particles of data flying about an ARGON system, CARBON is the larger-scale model of data en masse. CARBON is, at heart, defined in terms of IRON (a CARBON knowledge base can all be encoded in IRON), and CARBON is all about providing large-scale structure for bits or IRON data; but CARBON deals with issues of <em>scale</em> that IRON need not concern itself with. | > > | 1 2 3 4 5 6 7 8 9 | <h1>WORK IN PROGRESS</h1> While [./iron.wiki|IRON] is the model for individual particles of data flying about an ARGON system, CARBON is the larger-scale model of data en masse. CARBON is, at heart, defined in terms of IRON (a CARBON knowledge base can all be encoded in IRON), and CARBON is all about providing large-scale structure for bits or IRON data; but CARBON deals with issues of <em>scale</em> that IRON need not concern itself with. |
︙ | ︙ | |||
32 33 34 35 36 37 38 | <verbatim>!ns C </argon/carbon> [C:object: </example/foo> title: "A nice name" language: "en"]</verbatim> | | | | | 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 | <verbatim>!ns C </argon/carbon> [C:object: </example/foo> title: "A nice name" language: "en"]</verbatim> This is a relationship (<tt>/argon/carbon/object:title:language:</tt>) which can bind a title to any object, in this case being used to bind a title to an entity. To give an example of abstract objects, imagine building a Wikipedia-like database of useful facts about things at <tt>/wikipedia/</tt>. We might want to talk about love, which we might name <tt>/wikipedia/love</tt>. But there is no "love entity" that represents love in the world of ARGON; love isn't available as a software service (although you can rent a passable substitute, as they |
︙ | ︙ | |||
105 106 107 108 109 110 111 | or a string, but which also doesn't have a human-assigned global name. For instance, an e-commerce system will need a way to identify things like orders. The thing to do in this case is to still identify the object with a symbol, as it is not a concrete value; but to make a symbol up. This | | > | | | < < < < | | | | | 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 | or a string, but which also doesn't have a human-assigned global name. For instance, an e-commerce system will need a way to identify things like orders. The thing to do in this case is to still identify the object with a symbol, as it is not a concrete value; but to make a symbol up. This can be done by making a symbol relative to a single assigned namespace chosen for the purpose (eg, <tt>/com/mycompany/orders</tt>), named with a meaningful prefix followed by a unique number (and [./wolfram.wiki|WOLFRAM] provides us with Lamport timestamps which are guaranteed unique within the cluster). So the end result might be symbols like <tt>/com/mycompany/orders/id-2827391</tt>. Ideally, as orders are objects that one might perform operations upon (such as cancelling), the shop entity should give its orders entity IDs (as personae of itself, or by creating independent order entities) and assign the names to them. <h2>Compound objects</h2> Although anything can be described as a list of tuples making statements about it, that's not always the most efficient way of doing so. |
︙ | ︙ | |||
212 213 214 215 216 217 218 | vectors as honestly as it can, but we can still do lossy compression. If one follows the example of JPEG and breaks each plane of an image into 8x8 cells, then performs a discrete cosine transform on each and quantises them differentially to reserve more bits for more important image components, you end up with an array of 8x8 arrays of small integers for each plane. | | | | | | | | | | 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 | vectors as honestly as it can, but we can still do lossy compression. If one follows the example of JPEG and breaks each plane of an image into 8x8 cells, then performs a discrete cosine transform on each and quantises them differentially to reserve more bits for more important image components, you end up with an array of 8x8 arrays of small integers for each plane. We can then represent each plane as a vector made by taking the (0,0) element from each sub-array in turn, then the (0,1), then the (1,0), and so on in a JPEG-esque serpentine transposition that will tend to move all the significant information to the start of the vector, and make the trailing end of the vector largely zero. IRON will then have little difficulty in making a good job of losslessly compressing the result, thanks to the lossy encoding of the image. Compound objects allow for compact representation of things that would be messy with tuples; homogenous vectors are the best examples of those. But we also have compound objects that help with parallel updates. For instance, we might be interested in counting how many times some event has occurred. The naive solution is to have a tuple containing the count: <verbatim>[event: </example/foo> count: 57]</verbatim> When the event happens again, we look at the current count, add one, and tell CARBON: <verbatim>[C:not [event: </example/foo> count: 57]] [event: </example/foo> count: 58]</verbatim> The <tt>C:not</tt> tells CARBON to forget the old tuple, and then a new tuple is provided with the new count. However, if we have two such updates occuring in parallel, both of them might read 57 as the current value - and both would then write back 58 as the new count. The event has happened twice, but we've only gone up by one. And we won't even have a clue, as CARBON will not store <tt><nowiki>[event: </example/foo> count: 58]</nowiki></tt> twice - it merges identical tuples. However, if we declare: <verbatim>[C:counter: </example/foo-counter>]</verbatim> Then CARBON will treat that as an event counter object, storing a numeric counter and a buffer for pending events, initialised to zero |
︙ | ︙ | |||
284 285 286 287 288 289 290 | replicas are reachable: <verbatim>[C:counter-is-synchronized </example/foo-counter>]</verbatim> <h1>Temporary KBs</h1> Temporary knowledge bases can be created at will, and are stored | | > | 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 | replicas are reachable: <verbatim>[C:counter-is-synchronized </example/foo-counter>]</verbatim> <h1>Temporary KBs</h1> Temporary knowledge bases can be created at will, and are stored purely in RAM. They are just IRON objects with an opaque internal structure. Tuples are stored within them, and transparently mapped to compound objects where applicable. Perhaps the most interesting thing about them is that they can "chain" onto other knowledge bases (of any type), which will be consulted to satisfy queries along with the main knowledge base. In the event of |
︙ | ︙ | |||
315 316 317 318 319 320 321 322 323 324 325 | Compound object handlers get to take over the TUNGSTEN storage of their objects, so they can use appropriately compact and updateable representations in terms of the B-Tree. They also need to store their own update timestamps for individually updateable elements of the compound object, so they can correctly maintain the current state. <h1>Remote KBs via [./mercury.wiki|MERCURY]</h1> TODO: Caching and bittorrent. | > > > > | < < | > > > > > > > > > > | | > > > > | > > > > | > > > > > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > < < < < < < < < < < < < < < < < < < | > > > | > | > > > | 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 | Compound object handlers get to take over the TUNGSTEN storage of their objects, so they can use appropriately compact and updateable representations in terms of the B-Tree. They also need to store their own update timestamps for individually updateable elements of the compound object, so they can correctly maintain the current state. Note that the entity does not know its own CARBON name, as it might not have one or might have many, so when opening a TUNGSTEN section KB, an initial value for the default namespace needs to be supplied. <h1>Remote KBs via [./mercury.wiki|MERCURY]</h1> TODO: Caching and bittorrent. Note that as the CARBON protocol opens up TUNGSTEN knowledge base sections, it needs to know an initial default namespace - so the CARBON name being used to access the entity must be supplied in requests. If there is none (eg, we are just working from a raw EID), then that EID is mapped into a CARBON symbol of the form <tt><nowiki>/argon/eid/VERSION NUMBER/CLUSTER ID IN BASE64[/CLUSTER-LOCAL PART OF ENTITY ID IN BASE64[/PERSONA CLASS/PERSONA PARAMETERS IN BASE64]]</nowiki></tt>. <tt>/argon/eid</tt> offers a gateway that resolves these names back to the encoded entity IDs; if no cluster-local part is provided, then an empty string is used, producing the EID of the cluster entity corresponding to the cluster. <h2>Magic super turbo performance optimisations</h2> TODO: Direct access via MERCURY to the published TUNGSTEN sections, including support for ACLs and personae (use the persona class). Sketch the framework of support for a CDN by configuring the cluster to forward the published TUNGSTEN sections of a configured list of its entities out to nominated caching servers, and how clients can be configured to contact caching servers. How rules can be written whose body is not another CARBON query, but instead a pointer to a request that is sent via MERCURY back to the origin entity, actually causing entity code to run. This is necessary for the interesting cases where we're not just publishing information via CARBON, but instead exposing an actual service that generates or otherwise obtains information on demand, ranging from computational services, access to continuously-changing information sources such as sensors, processes that require access to secret information to generat a result, access to existing information systems (such as gateways to the Internet or "legacy systems"), and so on. <h1>The Directory</h1> A single global root EID, run by a non-profit foundation with a suitable governance structure to prevent it ever being monopolised. <dl> <dt><tt>/argon</tt></dt> <dd>Where ARGON system software is published from. This is delegated to a non-profit foundation (which may or may not be the same one as providing the root) which FIXME: Read these documents and gather together all the CARBON names described in them, and document them fully here.</dd> <dt><tt>/org</tt></dt> <dd>Where global non-profit organisations can register their own subtrees, for a small registration fee to the foundation and a small annual fee, to cover costs and to make sure failed organisations release their names back into the pool.</dd> <dt><tt>/com</tt></dt> <dd><tt>Where global companies can register their own subtrees, for a less small registration fee and annual fee, but still easily affordable for startups.</dd> <dt><tt>/gov</tt></dt> <dd><tt>Where international public bodies can register their own subtrees, for no fee but needing to prove their identity.</dd> <dt><tt>/me</tt></dt> <dd>Where anyone who doesn't want to tie their identity to a particular country can register a name, for a small cost-price setup fee and an annual renewal that costs nothing and just involves confirming continued usage (and which might only be required if no other evidence of continued usage can be found automatically). There is no restriction on registration under this prefix - corporations are welcome to, but <tt>/com</tt> looks better. Registrations are strictly anonymous under <tt>/me</tt>, with all that implies.</dd> <dt><tt>/<ISO two-letter country code>/[org|com|gov|me]</tt></dt> <dd>As above, but deliberately choosing to associated with a given geographical jurisdiction. The markup on registration and renewal fees, where it exists, is reduced slightly.</dd> <dt><tt>/example</tt></dt> <dd>A name that will never be bound, except perhaps to an information marker, used purely for examples without fear of it ever clashing with anything.</dd> <dt><tt>/home</tt></dt> <dd>A name that will never be bound in the global directory, except perhaps to an informational marker, reserved as a place for a local override to be presented by user interfaces in order to display information relevant to the context the user interface is in - eg, resources inside the user agent itself, the user's own CARBON space under their user agent entity, links to the user's bookmarked CARBON names, and resources relating to the user interface device being used to interact with the user agent, such as auto-discovered resources on the local network, hardware devices attached to the user interface device, and resources administratively configured into the user interface service such as the nearest printer, information resources about the building containing the device or the organisation providing it, etc.</dd> <dt><tt>[./fluorine.wiki|FLUORINE] gateways</tt></dt> <dd>For convenience, we should give these top-level names.</dd> </dl> It would be nice to also reserve a top-level prefix for some kind of distributed name system, which can even be implemented by a local override pointing to a local copy of the entity gatewaying into it so that the foundation cannot be used to control it. <tt>/argon/eid</tt> is one such, with ugly-looking names. It might be nice to have one that works like tor onion URLs, and maybe one based on something like namecoin, and so on. <h1>TODOs</h1> Note that published CARBON data from entities comes from a section called </argon/carbon/public> and is made available to all callers; but it is possible to declare (in </argon/carbon/configuration> other sections to publish with given ACLs. Talk about the ability to override parts of the namespace on a cluster-wide scale, configuring a list of CARBON namespace roots, each with the EID of the entity to "splice" into the tree at that point. The actual root of the global CARBON directory is configured in at this point, by having a mapping for </>. Also talk about the subsequent ability to demand that a local copy of any given namespace subtree be kept in the cluster at all times, in effect overriding its original name but pointing to a snapshot stored within the cluster. Note that CHROME modules list their dependencies, which can be used to recursively local-copy them. This is used to ensure that critical resources are available "offline", and to configure the cluster to use a specific version of something rather than "the latest". Such local copies do not change when upstream changes occur, but an administrator can view a list of newly available things and opt to upgrade, or downgrade. This is like "installing software". The local copies are stored in one or more nominated WOLFRAM distributed caches, with an infinite replication factor and no expiry timestamp or drop priority so they are kept until otherwise specified. By default, they go to the cluster cache, so are replicated to every node. |
Changes to intro/fluorine.wiki.
1 | Updated content will appear here shortly! | > > | 1 2 3 | <h1>WORK IN PROGRESS</h1> Updated content will appear here shortly! |
Changes to intro/iodine.wiki.
|
| | | 1 2 3 4 5 6 7 8 | <h1>WORK IN PROGRESS</h1> IODINE is a list of standard entity interfaces, defined as MERCURY interfaces and CARBON tuples that the entity is expected to provide. <h1>SEE ALSO</h1> MERCURY defines a protocol by which an entity may publish |
︙ | ︙ | |||
17 18 19 20 21 22 23 | different languages, etc. Look at Dublin Core for inspiration. Also, CARBON defines the containment/link relationship used to form the CARBON namespace used for IRON symbols (based around short names). <h1><tt>/argon/iodine/push</tt>: Object Push</h1> | | > > > > > > > > > > > > | 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 | different languages, etc. Look at Dublin Core for inspiration. Also, CARBON defines the containment/link relationship used to form the CARBON namespace used for IRON symbols (based around short names). <h1><tt>/argon/iodine/push</tt>: Object Push</h1> Here, have an object, represented as a CARBON knowledge bundle full of statements about the object (which is identified as the default namespace binding when the bundle is interpreted)! Usable for all sorts of sending tasks. To a user agent, it can be used to send emails, workflow tasks, appointment invitiations, etc. It's also the logical interface to use for printing; object push something to the printer. A CARBON directory might accept an Object Push (from somebody with sufficient rights) by creating a directed link if an entity ID is pushed. Entities containing user-editable information might accept general bundle pushes containing no explicit object (eg, no statements about the object at the default namespace prefix), but full of metadata about other objects, too. <h1><tt>/argon/iodine/directory</tt>: CARBON directory</h1> A user-manageable CARBON directory node, as opposed to one that generates the directory links programmatically. Link to entity. Edit list of linked entities and their local metadata. |
︙ | ︙ |