* To put tuples on the stack, we need to know both how long they are (number of members) and how big they are (amount of memory), since tuples can contain other tuples.
- All other values must be one stack cell:
*`nil` is its own thing
* numbers are a wrapped `f64` (at least until we get to NaN boxed values)
* booleans are a wrapped `bool`
* keywords are a wrapped `u16` or `u32`, which is an index into a vec of `&str`s, which can be read back into a string when printed
* strings are a `&str` or an `Rc<String>` (with two possible wrappers: `Value::Str` or `Value::String`)
* dicts are `imbl::HashMap<u16, Value>`, with the hash generated on the index of the keyword
* sets are `imbl::HashSet<Value>`, with the caveat that `f64` isn't `Eq`, which means that we can't use it for a hash key. The way around this, I think, is to implement `Eq` for `Value`, with a panic if we try to put NaN in a set
* functions are `Rc<LFn>`
* boxes are `Rc<RefCell>`
* That means everything is either a wrapped `Copy` (`:nil`, `:number`, `:bool`), an interned reference (`:keyword`, `:string`), `Rc` reference types (`:string`, `:box`, `:fn`), or persistent reference types that have their own `clone` (`:list`, `:dict`, `:set`)
* This doesn't cover everything, yet. But other reference types will be `Rc`ed structs: to wit, processes and packages.
- Tuples, meanwhile, have a special representation on the stack.
* They start with a `Value::TupleStart(len: u8, size: u8)`.
* They then have a number of members.
* They end with a `Value::TupleEnd(len: u8, size: u8)`.
*`len` indicates the number of members in the tuple; `size` indicates the size of the tuple on the stack, including the `TupleStart` and `TupleEnd` cells. For `()`, `len` is `0`, and `size` is `2`. Nesting tuples will lead to larger divergences, and will increase `size` but not `len`.
* If sombody tries to stuff more than 255 members in a tuple, nested or not, we get a validation error to tell them to use a list.
- Or promote it to be a reference type? The natural encoding of a list in Ludus is using a `(car, cdr)` encoding (or `(data, next)`). I believe the way to get this out of a scope (block or function) is to expand the tuple fully, which could lead very quickly to very large tuples.
- But we can easily distinguish between argument tuples and value tuples, and promote value tuples with a size larger than 255 to a `Value::BigTuple(Rc<Vec<Value>>)`.
- But in no case should we allow arguments to get bigger than 255.
- Keeping small value tuples on the stack is worthwhile, especially given the importance of result tuples, which should stay on the stack.
* This naturally leads to questions about pattern matching, especially when we get to a stack-based bytecode VM.
- A pattern, like a tuple, is a series of cells.
- The goal is to keep pattern sizes and lengths identical to the tuple data representation.
- That means that, like data representations, a pattern has to include both a set of bytecode instructions and a data representation on the stack.
- In fact, I suspect that the fastest way to encode this will be to push the data representation of the scrutinee on the stack, and then to push the pattern, and to then compare within the stack, at different offsets.
* processes/actors, although given that Ludus will be single-threaded for the forseeable future, it may be lighter weight to just write my own `process` abstraction
* in that case, we will need a ringbuffer, `ringbuf`
#### Which is proving rather harder to handle than I expected
I'm trying to use Chumsky to do this, but it's weirdly much harder to model with Chumsky's parer combinators than I expected.
I suspect the thing to do is to just brute force it in much the same way that I do in the Janet-based scanner: loop through the things and push things onto vectors in the correct ways.
This won't be a one-for-one translation, but I suspect it will be easier to manage than banging my head against, especially, the terrible error messages Chumsky's elaborate types give me.
This makes interpolated strings easy enough to work with.
That said, interpolation patterns are harder.
In particular, I worry whether I'll be able to compile a Chumsky parser with strings that aren't interned/`'static`.
Because the pattern match will actually have to be a little Chumsky parser guy (doo dah), or some equivalent.
(In the Janet-based interpreter, I used Janet's built-in PEGs.)
### On performance
The Rust tree-walk interpreter is something like two orders of magnitude faster than the Janet interpreter.
So in that sense, I think it's a worthwhile middle ground to effectively publish this first, easier-to-develop approach, and then to work on a bytecode VM later.
It's worth noting that my approach to this first tree-walk interpreter still leaves a lot on the table for optimization: the `Value` enum is 64 _bytes_.
This is because `imbl::Vector`s are 64 bytes.
I'm trying to ensure opportunistic mutation throughout, but I have found it hard with dicts.
This sort of thing.
Finally, it's clear that some perf testing will be necessary to determine the final arrangement of things.
Will `box`ing things to get heap pointers help?
Or will the extra indirection cost more speed than even if we squeeze `Value`'s size down to 8 bytes?
Will `box`ing lists, etc., mung up how `imbl` does refcounting and opportunistic mutation?
There are things like `tinyvec` which does some of the dark magic around allocating that might make using tuples easier to manage?
### On parsing in Ludus
I've been thinking about Ludus's built-in parsing capabilities.
Using the interpolition-style string pattern matching parsing for ELIZA makes a lot of sense, but we need something more robust for, say, a Lisp.
Looking at this, I think that Janet's builtin PEG parsing might be a much more interesting solution than just about anything else.
I'm pretty sure I can make a slow, but user-friendly-enough version of that that works in Ludus.
