In general, the quality of software is critically dependent on the skill with which the problem has been abstracted. A poor abstraction will leave many exceptions and a lot of extra error prone code. Conversely, a good expression of the problem will handle most cases as normal and will be relatively easy to modify in the light of new requirements. The design of a programmable solution to a real life problem is the key skill for IT and it depends far more on clear thinking than anything else. Martin Brampton

ARGON is an attempt to design a next generation computing platform. By computing platform, I mean both an operating system, or a platform like Java that sits atop an existing operating system, but appearing the same to the software running on top.

The design process has been simple: I have studied every applicable field of computing as best I can, learning the pros and cons of existing systems, studying even the most exotic theoretical solutions to a problem. Then I have tried my best to find the best combination of technologies to solve the problem at hand: To build a platform upon which the highest possible quality of software can be constructed with the least effort.

Philosophy

I'm reinventing the wheel.

As a software engineer, in my day to day work, I of course don't do this; the easiest solution to a problem is always to reuse as much existing technology as possible. However, the long-term result of this is that modern software is bloated, buggy, inflexible, and hard to use; and too much of the cost of developing new software is dealing with the failings of the platform it is built on top of.

That gets frustrating after a while.

So ARGON is my attempt to let off the steam built up from fighting POSIX APIs all day. I'm designing everything from scratch, learning from our past mistakes to come up with a software platform that's not a compromise built upon compromises.

I've been tinkering with the overall design since the mid 1990s, seeing how different problems I encounter could be handled in the ARGON world, and more and more of the design is "solidifying". By that, I mean that new challenges can be met elegantly by the current design without needing any further changes. This progression suggests that I'm slowly approaching a fundamental truth of computation, rather than the design being a short-term reaction to current fashions.

Current Status

ARGON is designed as a series of components, with trendy names taken from the periodic table. Each has a conceptual design page, and then a specification grows in Docbook as the conceptual design settles down; I'm migrating the conceptual design pages from the old site, so many of the components listed here lack their design link until the new page is ready. Specifications will be linked as they appear.

The big picture

Entities: ARGON is all about entities. They are the "objects" that users use, administrators organise, and developers develop.
User experience: For pragmatic reasons, I'm focussing on ARGON as a platform for servers and embedded systems for now, rather than general desktop environments. So in that respect, the ARGON user experience will be pretty much invisible - but I do have a conceptual design for what a direct user interface to the world of ARGON would be like.
Developer experience: Software in the world of ARGON always runs within the context of an entity (although libraries might be written to run in the context of any entity rather than a particular one). Here's what software development with ARGON will feel like.
Administrator experience: Ease of administration is a major priority in my design work, mainly because it's so unnecessarily difficult to administer current systems.
Security: Most low-level components of a system are involved in security in one way or another, so security appears throughout this site. However, it's important to explain the underlying security model, and how it all fits together.

The Kernel

HYDROGEN: This is the underlying hardware abstraction layer, covering code generation, basic device access, CPU control, booting, and so on. The concepts are solid; A draft specification is being produced. A hydrogen nucleus being a single proton led me to choose this name for the fundamental kernel of the system.
HELIUM: This is the resource manager, which functions like a "microkernel" for HYDROGEN; it looks after scheduling threads onto the available CPUs, and managing memory. The concepts are quite firm, but the details need working out and prototypes tinkered with before a specification can be started. The name is a play on it being a development of HYDROGEN.
IRON: This is the high-level runtime data representation; the data model upon which all else builds. It needs to be implemented in terms of HYDROGEN and HELIUM. The concepts are solid, and a draft specification exists. The name is chosen to refer to the use of iron as a structural material in the form of steel, as IRON is used to build data structures.
CHROME: This is the high-level programming language. It's to be implemented as a compiler using HYDROGEN code-generation primitives as the back end; the compiler converts source code (represented in IRON) into IRON values such as literal data and executable closures. The concepts are generally there, but need to be made more concrete. Details matter with a programming language.
IRIDIUM: This is a transport-layer network protocol, implemented on top of UDP, providing a high-level network abstraction to the rest of the ARGON stack. The concepts are solid and nearly ready to start writing up as a draft speciciation, as soon as I've thought about how to handle multicast properly. The name is chosen as the element Iridium sits between Mercury and Wolfram on the periodic table.
TUNGSTEN: This is the persistent storage system. It's a transactional tuple store, used to organise the persistent state of entities. However, it is not used directly by applications at this level - application access to it is performed through WOLFRAM, which handles replication of updates across the cluster. The concepts are solidifying fast, but I need to make a prototype to really test them. The name is chosen as tungsten is a particularly durable element, used as an alloying agent in steel to increase its hardness, in its carbide form as a hard material in its own right, and for its high melting point. The name seemed good for a persistent data store.
ARGON: ARGON refers to the system described here as a whole, but in terms of actual source code, it's the glue that combines all of the software components listed here into a running kernel on every node; the boot sequence and shared library code that doesn't fit into any particular module, or deserve a module in its own right. Variations on the structure of the kernel cater for specialised cluster nodes that handle different combinations of the storage and computational tasks within the cluster, or for standalone nodes that do not join a cluster at all, such as minimal embedded systems or user-interface-oriented devices such as thin clients or table computers.

The Cluster

WOLFRAM: This uses IRIDIUM (for communication) and TUNGSTEN (for local storage) to maintain a distributed, replicated, fault-tolerant transactional tuple store, and some other cluster-monitoring services. It brings a group of centrally-managed ARGON nodes together into a cohesive whole. The concepts are still a bit fluid, but are coming together; it unifies a lot of other components, so it's very sensitive to changes in their requirements. Wolfram is another name for the element Tungsten, and there's a close relationship between the two, but I chose the name Wolfram for the traditional association of wolves with computer clusters.
LITHIUM: Now we have distributed storage from TUNGSTEN and WOLFRAM, a high level language implementation from CHROME, and process scheduling from HELIUM, it's largely a matter of plumbing to assemble an infrastructure for invoking entity entry points; read the code from CARBON, compile it with CHROME (skipping the past two steps if we have a cached copy of the code), and run it in a HELIUM thread. As LITHIUM is the component that makes an ARGON system react to events, the name is a play on the reactivity of the element Lithium.
NITROGEN: ARGON attempts to push as much administration as possible into the cluster as a whole, but individual nodes still need some configuration and managment. They need to be assigned clearances, their disks need partitioning, they need network configuration, and they need device driver configuration. This is done by interacting with a special "node entity" that automatically exists for each node; its state is stored in TUNGSTEN and replicated via WOLFRAM like any other, but its interfaces are provided directly by the ARGON kernel on each node, and this is the component that implements them. The concepts are largely stable. The name was chosen purely due to the common initial N between the element name Nitrogen, and the object that NITROGEN manages, a Node.
MERCURY: Entities need to be able to invoke each others' services; MERCURY builds on top of IRIDIUM to provide a communication protocol that can run within or between clusters to invoke public entry points on entities with LITHIUM. The name is chosen not for chemical reasons, but because Mercury was the name of the Roman messenger god.
CAESIUM: Entities might also need to do things without being asked externally - so CAESIUM provides a distributed scheduler, invoking entity entry points using LITHIUM according to a schedule. The name is a nod to the use of the element Caesium inside atomic clocks. old site page
CARBON: TUNGSTEN stores entity state as sets of tuples; CARBON adds an inference engine on top of that (like a PROLOG implementation) to allow high-level querying. CARBON can obtain data from in-memory temporary tuple sets, TUNGSTEN data (via WOLFRAM), or from tuple stores published by entities (accessed via MERCURY). The publishing of tuple stores is such a performance-critical operation that special support for it is included in the implementation of MERCURY and TUNGSTEN to provide low-latency high-throughput access in common cases; this is ARGON's equivalent of the "sendfile" system call... The name is a play on the way that Carbon is used by organic systems to form complex structures, by analogy with the naming of IRON, but emphasising the complexity and versatility of the structures rather than brute strength. conceptual design

Userland

AURUM: HELIUM meters out memory and CPU time at run-time, within the context of a single thread; and TUNGSTEN meters out persistent storage space within the context of a single entity. However, a higher-level infrastructure is needed to allow management of those resources, and to manage other finite resources. AURUM is a set of software libraries for managing them, including distributed operation by communicating with "bank entities" via MERCURY. The name is, of course, a play on the use of gold as a medium of exchange. old site page,
FLUORINE: All of these lovely entities communicating via MERCURY are lovely, but what about the rest of the world? FLUORINE is a toolkit for communicating with standard Internet protocols and data formats. The name is a play on the element Fluorine's extreme electronegativity, which allows it to bind aggressively to other molecules.
IODINE: This is a library of standard MERCURY/CARBON interfaces for entities to provide user-level actions, such as sending things to places. The name is a weak reference to the fact that many of the interfaces relate to user agents that represent human beings, and the letter "I" meaning personal identity.
NEON: Part of IODINE is the concept of a "user entity" that represents a human being in the world of ARGON, but how does a human user actually get the user entity to act on their behalf? NEON provides an interface whereby a human interface device such as a laptop computer, tablet, or mobile phone can let a user authenticate to a user agent, then allow the user agent to control the human interface hardware in order to present an interface to the user. This interface can be delegated in a controlled manner, thereby allowing other entities to present an interface (mediated by the user entity) to the user. The name is a reference to the use of neon in brightly coloured electrical signs, as graphical interfacing is part of its responsibilities. old site page, some blog thoughts, some blog thoughts about events.

Development roadmap

ARGON has been designed with sufficient modularity to allow it to be constructed incrementally - in stages. Here's an approximate road map to implementing a full ARGON system.

HYDROGEN: An interpreted implementation in highly portable C, based on vmgen, to ensure that it can run on any POSIX system. It should run on a UNIX system and provide all the basic threading, networking, and disk I/O interfaces. The boot storage is just a series of standard POSIX files. Native-code compilers of increasing levels of optimisational sophistication and more optional features can be added in parallel.
IRON: The entire IRON spec needs to be implemented up-front.
CHROME: A simple interpreter for the core language can be written directly in HYDROGEN to begin with, and the standard libraries built on top of that as features are required by higher levels. Development of an optimising compiler, type inferencing, and so on can then continue in parallel.
TUNGSTEN: To start with, we can do without support for multiple mass storage devices and encryption, which can continue in parallel.
CARBON: The core of CARBON and its implementation in terms of TUNGSTEN persistent storage sections is now necessary.
IRIDIUM: The basic protocol layer is required for both WOLFRAM and MERCURY. It needs to be implemented in terms of UDP (v4) first; other protocols can come later.
LITHIUM: Now the requirements are in place, this should be fairly simple glue logic.
MERCURY: The clustering support needs to be in the protocol, even though we're not running as a cluster yet, but cryptography and mandatory access control can be left for later.
CARBON: CARBON's implementation over remote knowledge bases via MERCURY, and its LITHIUM shortcut to publish data directly from TUNGSTEN sections are now required.
WOLFRAM: We now have a simple single-node ARGON system, so userland developments can commence. However, we can now implement WOLFRAM and start patching it into all the unused hooks left for it in the rest of the system, in order to enable full clustered operation.

XENON

However, implementing a full ARGON system is quite a lot of work. A full HYDROGEN implementation (even with poor performance) is probably a man-year or more, and everything else builds upon that.

As a testbed for the concepts behind ARGON, and as a way to make use of them in the non-ARGON world of existing software development, and (in the long run) as a way for non-ARGON software to interact with ARGON (sort of the inverse of FLUORINE), I propose XENON: implementations of parts of ARGON in traditional programming languages.

As my main programming language of choice is Chicken Scheme, I will probably do most of the development in that; but network and data interoperability components such as IRON and MERCURY should probably also be implemented in other languages.

TODOs and ideas

This Site

Write a BNF for the IRON syntax, to make sure there aren't any ambiguities (I'm nervous about the overloading of colons as namespace prefix separators, map entry separators, as field name terminators in records, and as a record suffix to represent a wrapping record). See if I can simplify things a bit to reduce the set of characters that must be quoted in symbols.

intro pages to do: AURUM, FLUORINE, NEON

Fully document CARBON/MERCURY interfaces to cluster and node entities. Think about the services they should offer to administrators as well as to users.

Put Disqus or something on this site so people can comment and ask questions.

Create Docbook specifications

Create an index of symbols defined in the specs under /argon/, referring to their source spec, and what information needs to be published in CARBON about them.

Read old OmniOutliner notes in old-site/notes.oo3 - run run.sh in that directory to generate contents.org to read it in Org-Mode format - and incorporate any good stuff into the rest of the site

Further research required

Address "OPEN QUESTION"s.

Design a logging framework (PLATINUM). I want all logs integrated at one point, then distributed as needed, rather than ending up correlating disparate logs. I want logs structured with enough metadata to allow finding relationships between events so log entries can be "folded up" to hide fine detail (eg, a MERCURY incoming request will start off being logged by IRIDIUM then lead to a LITHIUM invocation which will lead to CHROME activity and then user code calling into WOLFRAM, TUNGSTEN and MERCURY to do stuff, then back to MERCURY to IRIDIUM to return a result, then WOLFRAM to commit the TUNGSTEN state transaction, etc - but it should be foldable up into a single "MERCURY invocation" summary event. We don't need CEP systems for this, just threading of event IDs down through the system as part of the dynamic execution context and appropriate importance levels. Log events should be represented in CARBON, so using these links shouldn't be hard. Also, node-specific events need to be fed into the HYDROGEN console system for diagnosis, and a storage system of some kind chosen for log events that need recording (with per-handler, per-entity, per-volume, per-node and cluster-wide log level settings to control storage utilisation, and the ability to remove levels of detail in older logs as they are expired). Where are logs stored? Within entities, or as a special case with direct TUNGSTEN storage? What gets replicated when? Do we log events up to level N to the node only, and events up to some (lower) level M get replicated in real time because they're important and we need them to not be tamperable if a node is compromised, and the level N logs on the node get summarised to level X and then replicated every day for archival purposes? And who can see the logs? A way is needed for logs about a given entity to be made available to the administrators of that entity (see also the debug mode tracing mechanism in LITHIUM/ARGON, which need to boil down to this). This is important and major enough to need an element name, I feel!

Should we allow for N-dimensional arrays as well as just vectors in IRON? They would allow for better predictive coding for sensor data such as images, as we'd be able to take into account pixels above as well as to the left of the next one (and the generalisation into higher dimensions thereof). If so, update the CARBON page's image storage example.

Build a prototype implementation of IRON in Scheme, and tinker with the binary encodings to find the best symbol mappings for packed binary encoding and so on. Try representing CHROME source code and CARBON knowledge in textual IRON to see how the syntax feels.

Write a CHROME interpreter in Scheme. Play with the language.

Grab vmgen and build a HYDROGEN kernel. Play with the language. Build some standard libraries.