# VM thoughts

### Initial thoughts
We want numbers and bools as unboxed as possible.

Nil is a singleton, and should be static.

Strings come in two flavours:
* String literals, which are static/interned.
* Constructed strings, which should be `Rc<String>`

Keywords are static/interned.

Tuples should be refcounted for now.

### Optimization and other thoughts
2024-11-09
* To put tuples on the stack, we need to know both how long they are (number of members) and how big they are (amount of memory), since tuples can contain other tuples.
  - All other values must be one stack cell:
    * `nil` is its own thing
    * numbers are a wrapped `f64` (at least until we get to NaN boxed values)
    * booleans are a wrapped `bool`
    * keywords are a wrapped `u16` or `u32`, which is an index into a vec of `&str`s, which can be read back into a string when printed
    * strings are a `&str` or an `Rc<String>` (with two possible wrappers: `Value::Str` or `Value::String`)
    * dicts are `imbl::HashMap<u16, Value>`, with the hash generated on the index of the keyword
    * sets are `imbl::HashSet<Value>`, with the caveat that `f64` isn't `Eq`, which means that we can't use it for a hash key. The way around this, I think, is to implement `Eq` for `Value`, with a panic if we try to put NaN in a set
    * functions are `Rc<LFn>`
    * boxes are `Rc<RefCell>`
    * That means everything is either a wrapped `Copy` (`:nil`, `:number`, `:bool`), an interned reference (`:keyword`, `:string`), `Rc` reference types (`:string`, `:box`, `:fn`), or persistent reference types that have their own `clone` (`:list`, `:dict`, `:set`)
    * This doesn't cover everything, yet. But other reference types will be `Rc`ed structs: to wit, processes and packages.
  - Tuples, meanwhile, have a special representation on the stack.
    * They start with a `Value::TupleStart(len: u8, size: u8)`.
    * They then have a number of members.
    * They end with a `Value::TupleEnd(len: u8, size: u8)`.
    * `len` indicates the number of members in the tuple; `size` indicates the size of the tuple on the stack, including the `TupleStart` and `TupleEnd` cells. For `()`, `len` is `0`, and `size` is `2`. Nesting tuples will lead to larger divergences, and will increase `size` but not `len`.
    * If sombody tries to stuff more than 255 members in a tuple, nested or not, we get a validation error to tell them to use a list.
      - Or promote it to be a reference type? The natural encoding of a list in Ludus is using a `(car, cdr)` encoding (or `(data, next)`). I believe the way to get this out of a scope (block or function) is to expand the tuple fully, which could lead very quickly to very large tuples.
      - But we can easily distinguish between argument tuples and value tuples, and promote value tuples with a size larger than 255 to a `Value::BigTuple(Rc<Vec<Value>>)`.
      - But in no case should we allow arguments to get bigger than 255.
      - Keeping small value tuples on the stack is worthwhile, especially given the importance of result tuples, which should stay on the stack.
* This naturally leads to questions about pattern matching, especially when we get to a stack-based bytecode VM.
  - A pattern, like a tuple, is a series of cells.
  - The goal is to keep pattern sizes and lengths identical to the tuple data representation.
  - That means that, like data representations, a pattern has to include both a set of bytecode instructions and a data representation on the stack.
  - In fact, I suspect that the fastest way to encode this will be to push the data representation of the scrutinee on the stack, and then to push the pattern, and to then compare within the stack, at different offsets.

### Let's not reinvent the wheel
#### Or, crates we will use
* `chumsky` for parsing
* `ariadne` for parsing errors
* `imbl` for persistent data structures
* `boxing` for NaN boxing (eventually?)
* ~~`tailcall` for tail recursion~~ This only works for simple recursion, and we need mutual recursion.

We additionally might want crates for:
* processes/actors, although given that Ludus will be single-threaded for the forseeable future, it may be lighter weight to just write my own `process` abstraction
* in that case, we will need a ringbuffer, `ringbuf`

### On string interpolation
#### Which is proving rather harder to handle than I expected
I'm trying to use Chumsky to do this, but it's weirdly much harder to model with Chumsky's parer combinators than I expected. 
I suspect the thing to do is to just brute force it in much the same way that I do in the Janet-based scanner: loop through the things and push things onto vectors in the correct ways.
This won't be a one-for-one translation, but I suspect it will be easier to manage than banging my head against, especially, the terrible error messages Chumsky's elaborate types give me.

This makes interpolated strings easy enough to work with.

That said, interpolation patterns are harder.
In particular, I worry whether I'll be able to compile a Chumsky parser with strings that aren't interned/`'static`.
Because the pattern match will actually have to be a little Chumsky parser guy (doo dah), or some equivalent.
(In the Janet-based interpreter, I used Janet's built-in PEGs.)

### On performance
The Rust tree-walk interpreter is something like two orders of magnitude faster than the Janet interpreter.
So in that sense, I think it's a worthwhile middle ground to effectively publish this first, easier-to-develop approach, and then to work on a bytecode VM later.