oops: put working document under version control

Cargo.toml

@@ -12,3 +12,9 @@ imbl = "3.0.0"
 struct_scalpel = "0.1.1"
 ran = "2.0.1"
 rust-embed = "8.5.0"
+boxing = "0.1.2"
+ordered-float = "4.5.0"
+index_vec = "0.1.4"
+num-derive = "0.4.2"
+num-traits = "0.2.19"
+regex = "1.11.1"

bytecode_thoughts.md

@@ -0,0 +1,255 @@
+# Working notes on bytecode stuff
+
+### 2024-12-15
+So far, I've done the easy stuff: constants, and ifs.
+
+There's still some easy stuff left:
+* [ ] lists
+* [ ] dicts
+* [ ] when
+* [ ] panic
+
+So I'll do those next.
+
+But then we've got two doozies: patterns and bindings, and tuples.
+
+#### Tuples make things hard
+In fact, it's tuples that make things hard. 
+The idea is that, when possible, tuples should be stored on the stack.
+That makes them a different creature than anything else.
+But the goal is to be able, in a function call, to just push a tuple onto the stack, and then match against it.
+Because a tuple _isn't_ just another `Value`, that makes things challenging.
+BUT: matching against all other `Values` should be straightforward enough?
+
+I think that the way to do this is to reify patterns.
+Rather than try to emit bytecodes to embody patterns, the patterns are some kind of data that get compiled and pushed onto a stack like keywords and interned strings and whatnot.
+And then you can push a pattern onto the stack right behind a value, and then have a `match` opcode that pops them off.
+
+Things get a bit gnarly since patterns can be nested. I'll start with the basic cases and run from there.
+
+But when things get *very* gnarly is considering tuples on the stack.
+How do you pop off a tuple?
+
+Two thoughts:
+1. Just put tuples on the heap. And treat function arguments/matching differently.
+2. Have a "register" that stages values to be pattern matched. 
+
+##### Regarding the first option
+I recall seeing somebody somewhere make a comment that trying to represent function arguments as tuples caused tons of pain.
+I can see why that would be the case, from an implementation standpoint. 
+We should have _values_, and don't do fancy bookkeeping if we don't have to.
+
+_Conceptually_, it makes a great deal of sense to think of tuples as being deeply the same as function invocation.
+But _practically_, they are different things, especially with Rust underneath.
+
+This feels like this cuts along the grain, and so this is what I will try.
+
+I suspect that I'll end up specializing a lot around function arguments and calling, but that feels more tractable than the bookkeeping around stack-based tuples.
+
+### 2024-12-17
+Next thoughts: take some things systematically rather than choosing an approach first.
+
+#### Things that always match
+* Placeholder.
+  - I _think_ this is just a no-op. A `let` expression leaves its rhs pushed on the stack.
+
+* Word: put something on the stack, and bind a name.
+  - This should follow the logic of locals as articulated in _Crafting Interpreters_.
+
+In both of these cases, there's no conditional logic, simply a bind.
+
+#### Things that never bind
+* Atomic values: put the rhs on the stack, then do an equality check, and panic if it fails. Leave the thing on the stack.
+
+#### Analysis
+In terms of bytecode, I think one thing to do, in the simple case, is to do the following:
+* `push` a `pattern` onto the stack
+* `match`--pops the pattern and the value off the stack, and then applies the pattern to the value. It leaves the value on the stack, and pushes a special value onto the stack representing a match, or not.
+  - We'll probably want `match-1`, `match-2`, `match-3`, etc., opcodes for matching a value that's that far back in the stack. E.g., `match-1` matches against not the top element, but the `top - 1` element.
+  - This is _specifically_ for matching function arguments and `loop` forms.
+* There are a few different things we might do from here:
+  - `panic_if_no_match`: panic if the last thing is a `no_match`, or just keep going if not.
+  - `jump_if_no_match`: in a `match` form or a function, we'll want to move to the next clause if there's no match, so jump to the next clause's `pattern` `push` code.
+* Compound patterns are going to be more complex.
+  - I think, for example, what you're going to need to do is to get opcodes that work on our data structures, so, for example, when you have a `match_compound` opcode and you start digging into the pattern.
+* Compound patterns are specifically _data structures_. So simple structures should be stack-allocated, and and complex structures should be pointers to something on the heap. Maybe?
+  
+#### A little note
+For instructions that need more than 256 possibilities, we'll need to mush two `u8`s together into a `u16`. The one liner for this is:
+
+```rust
+let number = ((first as u16) << 8) | second as u16;
+```
+
+#### Oy, stacks and expressions
+One thing that's giving me grief is when to pop and when to note on the value stack.
+
+So, like, we need to make sure that a line of code leaves the stack exactly where it was before it ran, with the exception of binding forms: `let`, `fn`, `box`, etc. Those leave one (or more!) items on the stack.
+
+In the simplest case, we have a line of code that's just a constant:
+
+```
+false
+```
+This should emit the bytecode instructions (more or less):
+```
+push false
+pop
+```
+The push comes from the `false` value. 
+The pop comes from the end of a (nonbinding) line.
+
+The problem is that there's no way (at all, in Ludus) to distinguish between an expression that's just a constant and a line that is a complete line of code that's an expression.
+
+So if we have the following:
+```
+let foo = false
+```
+We want:
+```
+push false
+```
+Or, rather, given that `foo` is a word pattern, what we actually want is:
+```
+push false # constant
+push pattern/word # load pattern
+pop
+pop # compare
+push false # for the binding
+```
+
+But it's worth it here to explore Ludus's semantics.
+It's the case that there are actually only three binding forms (for now): `let`, `fn`, and `box`. 
+Figuring out `let` will help a great deal. 
+Match also binds things, but at the very least, match doesn't bind with expressions on the rhs, but a single value.
+
+Think, too about expressions: everything comes down to a single value (of course), even tuples (especially now that I'm separating function calls from tuple values (probably)).
+So: anything that *isn't* a binding form should, before the `pop` from the end of a line, only leave a single value on the stack.
+Which suggests that, as odd as it is, pushing a single `nil` onto the stack, just to pop it, might make sense.
+Or, perhaps the thing to do is to peek: if the line in question is binding or not, then emit different bytecode.
+That's probably the thing to do. Jesus, Scott.
+
+And **another** thing worth internalizing: every single instruction that's not an explicit push or pop should leave the stack length unchanged.
+So store and load need always to swap in a `nil`
+
+### 2024-12-23
+Compiling functions.
+
+So I'm working through the functions chapter of _CI_, and there are a few things that I'm trying to wrap my head around.
+
+First, I'm thinking that since we're not using raw pointers, we'll need some functional indirection to get our current byte.
+
+So one of the hard things here is that, unlike with Lox, Ludus doesn't have fixed-arity functions. That means that the bindings for function calls can't be as dead simple as in Lox. More to the point, because we don't know everything statically, we'll need to do some dynamic magic.
+
+The Bob Nystrom program uses three useful auxiliary constructs to make functions straightforward:
+
+* `CallFrame`s, which know which function is being called, has their own instruction pointer, and an offset for the first stack slot that can be used by the function.
+
+```c
+typedef struct {
+  ObjFunction* function;
+  uint8_t* ip;
+  Value* slots;
+} CallFrame;
+  
+```
+
+Or the Rust equivalent:
+```rust
+struct CallFrame {
+  function: LFn,
+  ip: usize,
+  stack_root: usize,
+}
+```
+
+* `Closure`s, which are actual objects that live alongside functions. They have a reference to a function and to an array of "upvalues"...
+* `Upvalue`s, which are ways of pointing to values _below_ the `stack_root` of the call frame.
+
+##### Digression: Prelude
+I decided to skip the Prelude resolution in the compiler and only work with locals. But actually, closures, arguments, and the prelude are kind of the same problem: referring to values that aren't currently available on the stack.
+
+We do, however, know at compile time the following:
+* If a binding's target is on the stack, in a closure, or in the prelude.
+* This does, however, require that the function arguments work in a different way.
+
+The way to do this, I reckon, is this:
+* Limit arguments (to, say, no more than 7).
+* A `CallFrame` includes an arity field.
+* It also includes an array of length 7.
+* Each `match` operation in function arguments clones from the call frame, and the first instruction for any given body (i.e. once we've done the match) is to clear the arguments registers in the `CallFrame`, thus decrementing all the refcounts of all the heap-allocated objects.
+* And the current strategy of scoping and popping in the current implementation of `match` will work just fine!
+
+Meanwhile, we don't actually need upvalues, because bindings cannot change in Ludus. So instead of upvalues and their indirection, we can just emit a bunch of instructions to have a `values` field on a closure. The compiler, meanwhile, will know how to extract and emit instructions both to emit those values *and* to offer correct offsets.
+
+The only part I haven't figured out quite yet is how to encode access to what's stored in a closure.
+
+Also, I'm not certain we need the indirection of a closure object in Ludus. The function object itself can do the work, no?
+
+And the compiler knows which function it's closing over, and we can emit a bunch of instructions to close stuff over easily, after compiling the function and putting it in the constants table. The way to do this is to yank the value to the top of the stack using normal name resolution procedures, and then use a two-byte operand, `Op::Close` + index of the function in the constants table.
+
+##### End of digression.
+And, because we know exactly is bound in a given closure, we can actually emit instructions to close over a given value easily.
+
+#### A small optimization
+The lifetimes make things complicated; but I'm not sure that I would want to actually manage them manually, given how much they make my head hurt with Rust. I do get the sense that we will, at some point, need some lifetimes. A `Chunk` right now is chunky, with lots of owned `vec`s.
+
+Uncle Bob separates `Chunk`s and `Compiler`s, which, yes! But then we have a problem: all of the information to climb back to source code is in the `Compiler` and not in the `Chunk`. How to manage that encoding?
+
+(Also the keyword and string intern tables should be global, and not only in a single compiler, since we're about to get nested compilers...)
+
+### 2024-12-24
+Other interesting optimizations abound:
+* `add`, `sub`, `inc`, `dec`, `type`, and other extremely frequently used, simple functions can be compiled directly to built-in opcodes. We still need functions for them, with the same arities, for higher order function use.
+  - The special-case logic is in the `Synthetic` compiler branch, rather than anywhere else.
+  - It's probably best to disallow re-binding these names anywhere _except_ Prelude, where we'll want them shadowed.
+  - We can enforce this in `Validator` rather than `Compiler`.
+* `or` and `and` are likewise built-in, but because they don't evaluate their arguments eagerly, that's another, different special case that's a series of eval, `jump_if_false`, eval, `jump_if_false`, instructions.
+* More to the point, the difference between `or` and `and` here and the built-ins is that `or` and `and` are variadic, where I was originally thinking about `and` and co. as fixed-arity, with variadic behaviours defined by a shadowing/backing Ludus function. That isn't necessary, I don't think.
+* Meanwhile, `and` and `or` will also, of necessity, have backing shadowing functions.
+
+#### More on CallFrames and arg passing
+* We don't actually need the arguments register! I was complicating things. The stack between the `stack_root` and the top will be _exactly_ the same as an arguments register would have been in my imagination. So we can determine the number of arguments passed in with `stack.len() - stack_root`, and we can access argument positions with `stack_root + n`, since the first argument is at `stack_root`.
+  - This has the added benefit of not having to do any dances to keep the refcount of any heap-allocated objects as low as possible. No extra `Clone`s here.
+* In addition, we need two `check_arity` ops: one for fixed-arity clauses, and one for clauses with splatterns. Easily enough done. Remember: opcodes are for special cases!
+
+#### Tail calls
+* The way to implement tail calls is actually now really straightforward! The idea is to simply have a `TailCall` rather than a `Call` opcode. In place of creating a new stack frame and pushing it to the call stack on top of the old call frame, you pop the old call frame, then push the new one to the call stack.
+* That does mean the `Compiler` will need to keep track of tail calls. This should be pretty straightforward, actually, and the logic is already there in `Validator`.
+* The thing here is that the new stack frame simply requires the same return location as the old one it's replacing.
+* That reminds me that there's an issue in terms of keeping track of not just the IP, but the chunk. In Lox, the IP is a pointer to a `u8`, which works great in C. But in Rust, we can't use a raw pointer like that, but an index into a `vec<u8>`. Which means the return location needs both a chunk and an index, not just a `u8` pointer:
+```rust
+struct StackFrame<'a> {
+  function: LFn,
+  stack_root: usize,
+  return: (&'a Chunk, usize),
+}
+```
+(I hate that there's a lifetime here.)
+
+This gives us a way to access everything we need: where to return to, the root of the stack, the chunk (function->chunk), the closures (function->closures).
+
+### 2024-12-26
+One particular concern here, which needs some work: recursion is challenging.
+
+In particular, the issue is that if, as I have been planning, a function closes over all its values at the moment it is compiled, the only value type that requires updating is a function. A function can be declared but not yet defined, and then when another function that uses that function is defined, the closed-over value will be to the declaration but not the definition.
+
+One way to handle this, I think is using `std::cell::OnceCell`. Rather than a `RefCell`, `OnceCell` has no runtime overhead. Instead, what happens is you effectively put a `None` in the cell. Then, once you have the value you want to put in there, you call `set` on the `OnceCell`, and it does what it needs to.
+
+This allows for the closures to be closed over right after compilation.
+
+### 2024-12-27
+Working on `loop` and `recur`, rather than `fn`--this is the gentler slope.
+And discovering that we actually need a `[Value; 15]` return register.
+`recur` needs to push all the arguments to the stack, then yank them off into the return register, then pop back to the loop root, then push all the things back onto the stack, then jump to the beginning of the loop.
+And that also means I need a different `Value` variant that's a true `Nothing`, not even `nil`, which will _never_ end up anywhere other than a placeholder value in the register and on the stack.
+
+So, next steps:
+1. Add `Value::Nothing` and fix all the compiler errors
+2. Make the return register `[Value; 15]`, populated with `Value::Nothing`s at initialization.
+3. Update `load` and `store` to work with the array rather than a single value.
+4. Create `store_n` and `load_n` to work with multiple values.
+5. Create a `Vm.arity` method that computes how many non-nothings were passed into the register.
+6. Then, implement `recur`
+7. And, then, fix up jump indexes in `loop`
+8. Fix all the off-by-one errors in the jumps

may_2025_thoughts.md

@@ -0,0 +1,272 @@
+# Catching back up
+## May 2025
+
+### Bugs
+
+#### `match` is not popping things correctly
+```
+=== source code ===
+
+let foo = match :foo with {
+    :foo -> 1
+    :bar -> 2
+    :baz -> 3
+}
+foo
+
+=== chunk: test ===
+IDX | CODE           | INFO
+0000: reset_match
+0001: constant         0000: :foo
+0003: match_constant   0000: :foo
+0005: jump_if_no_match 0006
+0007: constant         0001: 1
+0009: store
+0010: pop
+0011: jump             0023
+0013: match_constant   0002: :bar
+0015: jump_if_no_match 0006
+0017: constant         0003: 2
+0019: store
+0020: pop
+0021: jump             0013
+0023: match_constant   0004: :baz
+0025: jump_if_no_match 0006
+0027: constant         0005: 3
+0029: store
+0030: pop
+0031: jump             0003
+0033: panic_no_match
+0034: load
+0035: match_word
+0036: panic_if_no_match
+0037: push_binding     0000
+0039: store
+0040: pop
+0041: pop
+0042: load
+
+
+
+=== vm run: test ===
+0000: [] nil
+0000: reset_match
+0001: [] nil
+0001: constant         0000: :foo
+0003: [:10] nil
+0003: match_constant   0000: :foo
+0005: [:10] nil
+0005: jump_if_no_match 0006
+0007: [:10] nil
+0007: constant         0001: 1
+0009: [:10|1] nil
+0009: store
+0010: [:10|nil] 1
+0010: pop
+0011: [:10] 1
+0011: jump             0023
+0036: [:10] 1
+0036: panic_if_no_match
+0037: [:10] 1                <== Should "return" from match here
+0037: push_binding     0000
+0039: [:10|:10] 1
+0039: store
+0040: [:10|nil] :10
+0040: pop
+0041: [:10] :10
+0041: pop
+0042: [] :10
+0042: load
+:foo
+```
+Should return `1`.
+
+Instruction `0037` is where it goes off the rails.
+
+### Things left undone
+Many. But things that were actively under development and in a state of unfinishedness:
+
+1. Tuple patterns
+2. Loops
+3. Function calls
+
+#### Tuple patterns
+This is the blocking issue for loops, function calls, etc.
+You need tuple pattern matching to get proper looping and function calls.
+Here are some of the issues I'm having:
+* Is it possible to represent tuples on the stack? Right now they're allocated on the heap, which isn't great for function calls.
+* How to represent complex patterns? There are a few possibilities:
+    - Hard-coded into the bytecode (this is probably the thing to do?)
+    - Represented as a data structure, which itself would have to be allocated on the heap
+    - Some hybrid of the two:
+        * Easy scalar values are hard-coded: `nil`, `true`, `:foo`, `10` are all built into the bytecode + constants table
+        * Perhaps dict patterns will have to be data structures
+
+#### Patterns, generally
+On reflection, I think the easiest (perhaps not simplest!) way to go is to model the patterns as separate datatypes stored per-chunk in a vec.
+The idea is that we push a value onto the stack, and then have a `match` instruction that takes an index into the pattern vec.
+We don't even need to store all pattern types in that vec: constants (which already get stored in the constant vec), interpolations, and compound patterns.
+`nil`, `false`, etc. are singletons and can be handled like (but not exactly as) the `nil` and `false` _values_.
+This also means we can outsource the pattern matching mechanics to Rust, which means we don't have to fuss with "efficient compiling of pattern matching" titchiness.
+This also has the benefit, while probably being fast _enough_, of reflecting the conceptual domain of Ludus, in which patterns and values are different DSLs within the language.
+So: model it that way.
+
+### Now that we've got tuple patterns
+#### May 23, 2025
+A few thoughts:
+* Patterns aren't _things_, they're complex conditional forms. I had not really learned that; note that the "compiling pattern matching efficiently" paper is about how to optimize those conditionals. The tuple pattern compilation more closely resembles an `if` or `when` form.
+* Tuple patterns break the nice stack-based semantics of binding. So do other patterns. That means I had to separate out bindings and the stack. I did this by introducing a representation of the stack into the compiler (it's just a stack-depth counter).
+    - This ended up being rather titchy. I think there's a lot of room to simplify things by avoiding manipulating this counter directly. My sense is that I should probably move a lot of the `emit_op` calls into methods that ensure all the bookkeeping happens automatically.
+* `when` is much closer to `if` than `match`; remember that!
+* Function calls should be different from tuple pattern matching. Tuples are currently (and maybe forever?) allocated on the heap. Function calls should *not* have to pass through the heap. The good news: `Arguments` is already a different AST node type than `Tuple`; we'll want an `ArgumentsPattern` pattern node type that's different from (and thus compiled differently than) `TuplePattern`. They'll be similar--the matching logic is the same, after all--but the arguments will be on the stack already, and won't need to be unpacked in the same way.
+    - One difficulty will be matching against different arities? But actually, we should compile these arities as different functions.
+    - Given splats, can we actually compile functions into different arities? Consider the following:
+    ```
+      fn foo {
+        (x) -> & arity 1
+        (y, z) -> & arity 2
+        (x, y, 2) -> & arity 3
+        (...z) -> & arity 0+
+      }
+    ```
+    `fn(1, 2, 3)` and `fn(1, 2, 4)` would invoke different "arities" of the function, 3 and 0+, respectively. I suspect the simpler thing really is to just compile each function as a singleton, and just keep track of the number of arguments you're matching.
+* Before we get to function calls, `loop`/`recur` make sense as the starting ground. I had started that before, and there's some code in those branches of the compiler. But I ran into tuple pattern matching. That's now done, although actually, the `loop`/`recur` situation probably needs a rewrite from the ground up.
+* Just remember: we're not aiming for *fast*, we're aiming for *fast enough*. And I don't have a ton of time. So the thing to do is to be as little clever as possible.
+* I suspect the dominoes will fall reasonably quickly from here through the following:
+    - [x] `list` and `dict` patterns
+    - [x] updating `repeat`
+    - [x] standing up `loop`/`recur`
+    - [x] standing up functions
+    - [x] more complex synthetic expressions
+    - [x] `do` expressions
+* That will get me a lot of the way there. What's left after that which might be challenging?
+    - [x] string interpolation
+    - [x] splats
+    - [ ] splatterns
+    - [x] string patterns
+    - [x] partial application
+    - [ ] tail calls
+    - [ ] stack traces in panics
+    - [ ] actually good lexing, parsing, and validation errors. I got some of the way there in the fall, but everything needs to be "good enough."
+* After that, we're in integration hell: taking this thing and putting it together for Computer Class 1. Other things that I want (e.g., `test` forms) are for later on.
+* There's then a whole host of things I'll need to get done for CC2:
+    - some kind of actual parsing strategy (that's good enough for "Dissociated Press"/Markov chains)
+    - actors
+    - animation in the frontend
+* In addition to a lot of this, I think I need some kind of testing solution. The Janet interpreter is pretty well-behaved.
+
+### Now that we've got some additional opcodes and loop/recur working
+#### 2025-05-27
+The `loop` compilation is _almost_ the same as a function body. That said, the thing that's different is that we don't know the arity of the function that's called.
+
+A few possibilities:
+* Probably the best option: enforce a new requirement that splat patterns in function clauses *must* be longer than any explicit arity of the function. So, taking the above:
+    ```
+      fn foo {
+        (x) -> & arity 1
+        (y, z) -> & arity 2
+        (x, y, 2) -> & arity 3
+        (...z) -> & arity 0+
+      }
+    ```
+This would give you a validation error that splats must be longer than any other arity.
+Similarly, we could enforce this:
+```
+fn foo {
+    (x) -> & arity 1
+    (x, y) -> & arity 2
+    (x, ...) & arity n > 1; error! too short
+    (x, y, ...) & arity n > 2; ok!
+}  
+```
+The algorithm for compiling functions ends up being a little bit titchy, because we'll have to store multiple functions (i.e. chunks) with different arities.
+Each arity gets a different chunk.
+And the call function opcode comes with a second argument that specifies the number of arguments, 0 to 7.
+(That means we only get a max of 7 arguments, unless I decide to give the call opcode two bytes, and make the call/return register much bigger.)
+
+Todo, then:
+* [x] reduce the call/return register to 7
+* [x] implement single-arity compilation
+* [x] implement single-arity calls, but with two bytes
+* [x] compile multiple-arity functions
+* [x] add closures
+
+### Some thoughts while chattin w/ MNL
+On `or` and `and`: these should be reserved words with special casing in the parser.
+You can't pass them as functions, because that would change their semantics.
+So they *look* like functions, but they don't *behave* like functions.
+In Clojure, if you try `(def foo or)`, you get an error that you "can't take the value of a macro."
+I'll need to change Ludus so that `or` and `and` expressions actually generate different AST nodes, and then compile them from there.
+
+AND: done.
+
+### Implementing functions & calls, finally
+#### 2025-06-01
+Just to explain where I am to myself:
+* I have a rough draft (probably not yet fully functional) of function compilation in the compiler.
+* Now I have to implement two op codes: `Call` and `Return`.
+* I now need to implement call frames.
+* A frame in _Crafting Interpreters_ has: a pointer to a Lox function object, an ip, and an index into the value stack that indicates the stack bottom for this function call. Taking each of these in turn:
+* The lifetime for the pointer to the function:
+    - The pointer to the function object cannot, I think, be an explicit lifetime reference, since I don't think I know how to prove to the Rust borrow checker that the function will live long enough, especially since it's inside an `Rc`.
+    - That suggests that I actually need the whole `Value::Fn` struct and not just the inner `LFn` struct, so I can borrow it.
+* The ip and stack bottom are just placeholders and don't change.
+
+### Partially applied functions
+#### 2025-06-05
+Partially applied functions are a little complicated, because they involve both the compiler and the VM.
+Maybe.
+My sense is that, perhaps, the way to do them is actually to create a different value branch.
+They ....
+
+### Jumping! Numbers! And giving up the grind
+#### 2025-06-05
+Ok! So.
+This won't be ready for next week.
+That's clear enough now--even though I've made swell progress!
+One thing I just discovered, which, well, it feels silly I haven't found this before.
+Jump instructions, all of them, need 16 bits, not 8.
+
+That means some fancy bit shifting, and likely some refactoring of the compiler & vm to make them easier to work with.
+
+For reference, here's the algorithm for munging u8s and u16s:
+
+```rust
+let a: u16 = 14261;
+let b_high: u8 = (a >> 8) as u8;
+let b_low: u8 = a as u8;
+let c: u16 = ((b_high as u16) << 8) + b_low as u16;
+println!("{a} // {b_high}/{b_low} // {c}");
+```
+
+To reiterate the punch list that *I would have needed for Computer Class 1*:
+* [ ] jump instructions need 16 bits of operand
+* [ ] splatterns
+    - [ ] validator should ensure splatterns are the longest patterns in a form
+* [ ] add guards to loop forms
+* [ ] check loop forms against function calls: do they still work the way we want them to?
+* [ ] tail call elimination
+* [ ] stack traces in panics
+* [ ] actually good error messages
+    - [ ] parsing
+    - [ ] my memory is that validator messages are already good?
+    - [ ] panics, esp. no match panics
+* [ ] getting to prelude
+    - [ ] `base` should load into Prelude
+    - [ ] prelude should run properly
+    - [ ] prelude should be loaded into every context
+* [ ] packaging things up
+    - [ ] add a `to_json` method for values
+    - [ ] teach Rudus to speak our protocols (stdout and turtle graphics)
+    - [ ] there should be a Rust function that takes Ludus source and returns valid Ludus status json
+    - [ ] compile Rust to WASM
+    - [ ] wire rust-based WASM into JS
+    - [ ] FINALLY, test Rudus against Ludus test cases
+
+So this is the work of the week of June 16, maybe?
+
+Just trying to get a sense of what needs to happen for CC2:
+* [ ] Actor model (objects, Spacewar!)
+* [ ] Animation hooked into the web frontend (Spacewar!)
+* [ ] Saving and loading data into Ludus (perceptrons, dissociated press)
+

src/base.rs

@@ -4,40 +4,44 @@ use ran::ran_f64;
 use std::rc::Rc;
 
 #[derive(Clone, Debug)]
-pub enum BaseFn<'src> {
-    Nullary(fn() -> Value<'src>),
-    Unary(fn(&Value<'src>) -> Value<'src>),
-    Binary(fn(&Value<'src>, &Value<'src>) -> Value<'src>),
-    Ternary(fn(&Value<'src>, &Value<'src>, &Value<'src>) -> Value<'src>),
+pub enum BaseFn {
+    Nullary(fn() -> Value),
+    Unary(fn(&Value) -> Value),
+    Binary(fn(&Value, &Value) -> Value),
+    Ternary(fn(&Value, &Value, &Value) -> Value),
 }
 
-pub fn eq<'src>(x: &Value<'src>, y: &Value<'src>) -> Value<'src> {
-    Value::Boolean(x == y)
+pub fn eq(x: &Value, y: &Value) -> Value {
+    if x == y {
+        Value::True
+    } else {
+        Value::False
+    }
 }
 
-pub fn add<'src>(x: &Value<'src>, y: &Value<'src>) -> Value<'src> {
+pub fn add(x: &Value, y: &Value) -> Value {
     match (x, y) {
         (Value::Number(x), Value::Number(y)) => Value::Number(x + y),
         _ => unreachable!("internal Ludus error"),
     }
 }
 
-pub fn sub<'src>(x: &Value<'src>, y: &Value<'src>) -> Value<'src> {
+pub fn sub(x: &Value, y: &Value) -> Value {
     match (x, y) {
         (Value::Number(x), Value::Number(y)) => Value::Number(x - y),
         _ => unreachable!("internal Ludus error"),
     }
 }
 
-pub fn unbox<'src>(x: &Value<'src>) -> Value<'src> {
+pub fn unbox(x: &Value) -> Value {
     match x {
-        Value::Box(_, cell) => cell.borrow().clone(),
+        Value::Box(cell) => cell.as_ref().borrow().clone(),
         _ => unreachable!("internal Ludus error"),
     }
 }
 
-pub fn store<'src>(b: &Value<'src>, val: &Value<'src>) -> Value<'src> {
-    if let Value::Box(_, cell) = b {
+pub fn store(b: &Value, val: &Value) -> Value {
+    if let Value::Box(cell) = b {
         cell.replace(val.clone());
         val.clone()
     } else {
@@ -47,103 +51,88 @@ pub fn store<'src>(b: &Value<'src>, val: &Value<'src>) -> Value<'src> {
 
 // TODO: do better than returning just the docstr
 // name, patterns, AND docstring
-pub fn doc<'src>(f: &Value<'src>) -> Value<'src> {
+pub fn doc(f: &Value) -> Value {
     match f {
-        Value::Fn(f) => {
-            let name = &f.borrow().name;
-            let doc = &f.borrow().doc;
-            if let Some(docstr) = doc {
-                Value::AllocatedString(Rc::new(format!("{name}: {docstr}")))
-            } else {
-                Value::AllocatedString(Rc::new(format!("{name}: no documentation found")))
-            }
-        }
-        _ => Value::InternedString("no documentation found"),
+        Value::Fn(f) => f.as_ref().doc(),
+        _ => Value::Interned("no documentation found"),
     }
 }
 
-pub fn and<'src>(x: &Value<'src>, y: &Value<'src>) -> Value<'src> {
-    Value::Boolean(x.bool() && y.bool())
-}
-
-pub fn assoc<'src>(dict: &Value<'src>, key: &Value<'src>, value: &Value<'src>) -> Value<'src> {
+pub fn assoc(dict: &Value, key: &Value, value: &Value) -> Value {
     match (dict, key) {
-        (Value::Dict(d), Value::Keyword(k)) => Value::Dict(d.update(k, value.clone())),
+        (Value::Dict(d), Value::Keyword(k)) => Value::Dict(Box::new(d.update(k, value.clone()))),
         _ => unreachable!("internal Ludus error"),
     }
 }
 
-pub fn r#bool<'src>(x: &Value<'src>) -> Value<'src> {
-    Value::Boolean(x.bool())
+pub fn r#bool(x: &Value) -> Value {
+    match x {
+        Value::Nil | Value::False => Value::False,
+        _ => Value::True,
+    }
 }
 
-pub fn chars<'src>(x: &Value<'src>) -> Value<'src> {
+pub fn chars(x: &Value) -> Value {
     match x {
-        Value::InternedString(s) => {
+        Value::Interned(s) => {
             let chars = s.chars();
 
             let mut charlist = vector![];
             for char in chars {
                 if char.is_ascii() {
-                    charlist.push_back(Value::AllocatedString(Rc::new(char.to_string())))
+                    charlist.push_back(Value::String(Rc::new(char.to_string())))
                 } else {
                     return Value::Tuple(Rc::new(vec![
                         Value::Keyword("err"),
-                        Value::AllocatedString(Rc::new(format!(
-                            "{char} is not an ascii character"
-                        ))),
+                        Value::String(Rc::new(format!("{char} is not an ascii character"))),
                     ]));
                 }
             }
-            Value::Tuple(Rc::new(vec![Value::Keyword("ok"), Value::List(charlist)]))
+            Value::Tuple(Rc::new(vec![
+                Value::Keyword("ok"),
+                Value::List(Box::new(charlist)),
+            ]))
         }
-        Value::AllocatedString(s) => {
+        Value::String(s) => {
             let chars = s.chars();
 
             let mut charlist = vector![];
             for char in chars {
                 if char.is_ascii() {
-                    charlist.push_back(Value::AllocatedString(Rc::new(char.to_string())))
+                    charlist.push_back(Value::String(Rc::new(char.to_string())))
                 } else {
                     return Value::Tuple(Rc::new(vec![
                         Value::Keyword("err"),
-                        Value::AllocatedString(Rc::new(format!(
-                            "{char} is not an ascii character"
-                        ))),
+                        Value::String(Rc::new(format!("{char} is not an ascii character"))),
                     ]));
                 }
             }
-            Value::Tuple(Rc::new(vec![Value::Keyword("ok"), Value::List(charlist)]))
+            Value::Tuple(Rc::new(vec![
+                Value::Keyword("ok"),
+                Value::List(Box::new(charlist)),
+            ]))
         }
         _ => unreachable!("internal Ludus error"),
     }
 }
 
 // TODO: figure out how to get to opportunistic mutation here
-pub fn concat<'src>(x: &Value<'src>, y: &Value<'src>) -> Value<'src> {
+pub fn concat(x: &Value, y: &Value) -> Value {
     match (x, y) {
-        (Value::InternedString(x), Value::InternedString(y)) => {
-            Value::AllocatedString(Rc::new(format!("{x}{y}")))
-        }
-        (Value::AllocatedString(x), Value::AllocatedString(y)) => {
-            Value::AllocatedString(Rc::new(format!("{x}{y}")))
-        }
-        (Value::AllocatedString(x), Value::InternedString(y)) => {
-            Value::AllocatedString(Rc::new(format!("{x}{y}")))
-        }
-        (Value::InternedString(x), Value::AllocatedString(y)) => {
-            Value::AllocatedString(Rc::new(format!("{x}{y}")))
-        }
+        (Value::Interned(x), Value::Interned(y)) => Value::String(Rc::new(format!("{x}{y}"))),
+        (Value::String(x), Value::String(y)) => Value::String(Rc::new(format!("{x}{y}"))),
+        (Value::String(x), Value::Interned(y)) => Value::String(Rc::new(format!("{x}{y}"))),
+        (Value::Interned(x), Value::String(y)) => Value::String(Rc::new(format!("{x}{y}"))),
         (Value::List(x), Value::List(y)) => {
-            let mut newlist = x.clone();
-            newlist.append(y.clone());
-            Value::List(newlist)
+            let mut newlist = *x.clone();
+            newlist.append(*y.clone());
+            Value::List(Box::new(newlist))
         }
         _ => unreachable!("internal Ludus error"),
     }
 }
 
-pub fn append<'src>(x: &Value<'src>, y: &Value<'src>) -> Value<'src> {
+pub fn append(x: &Value, y: &Value) -> Value {
     match x {
         Value::List(list) => {
             let mut newlist = list.clone();
@@ -154,35 +143,35 @@ pub fn append<'src>(x: &Value<'src>, y: &Value<'src>) -> Value<'src> {
     }
 }
 
-pub fn dec<'src>(x: &Value<'src>) -> Value<'src> {
+pub fn dec(x: &Value) -> Value {
     match x {
         Value::Number(n) => Value::Number(n - 1.0),
         _ => unreachable!("internal Ludus error"),
     }
 }
 
-pub fn inc<'src>(x: &Value<'src>) -> Value<'src> {
+pub fn inc(x: &Value) -> Value {
     match x {
         Value::Number(n) => Value::Number(n + 1.0),
         _ => unreachable!("internal Ludus error"),
     }
 }
 
-pub fn div<'src>(x: &Value<'src>, y: &Value<'src>) -> Value<'src> {
+pub fn div(x: &Value, y: &Value) -> Value {
     match (x, y) {
         (Value::Number(x), Value::Number(y)) => Value::Number(x / y),
         _ => unreachable!("internal Ludus error"),
     }
 }
 
-pub fn mult<'src>(x: &Value<'src>, y: &Value<'src>) -> Value<'src> {
+pub fn mult(x: &Value, y: &Value) -> Value {
     match (x, y) {
         (Value::Number(x), Value::Number(y)) => Value::Number(x * y),
         _ => unreachable!("internal Ludus error"),
     }
 }
 
-pub fn dissoc<'src>(dict: &Value<'src>, key: &Value<'src>) -> Value<'src> {
+pub fn dissoc(dict: &Value, key: &Value) -> Value {
     match (dict, key) {
         (Value::Dict(dict), Value::Keyword(key)) => {
             let mut new = dict.clone();
@@ -193,7 +182,7 @@ pub fn dissoc<'src>(dict: &Value<'src>, key: &Value<'src>) -> Value<'src> {
     }
 }
 
-pub fn first<'src>(ordered: &Value<'src>) -> Value<'src> {
+pub fn first(ordered: &Value) -> Value {
     match ordered {
         Value::List(list) => match list.front() {
             Some(n) => n.clone(),
@@ -209,7 +198,7 @@ pub fn first<'src>(ordered: &Value<'src>) -> Value<'src> {
 
 // TODO: figure out how to handle negative numbers
 // the cast from f64 to usize discards sign info
-pub fn at<'src>(ordered: &Value<'src>, i: &Value<'src>) -> Value<'src> {
+pub fn at(ordered: &Value, i: &Value) -> Value {
     match (ordered, i) {
         (Value::List(list), Value::Number(n)) => {
             let i = *n as usize;
@@ -229,7 +218,7 @@ pub fn at<'src>(ordered: &Value<'src>, i: &Value<'src>) -> Value<'src> {
     }
 }
 
-pub fn get<'src>(dict: &Value<'src>, key: &Value<'src>) -> Value<'src> {
+pub fn get(dict: &Value, key: &Value) -> Value {
     match (dict, key) {
         (Value::Dict(dict), Value::Keyword(key)) => match dict.get(key) {
             Some(x) => x.clone(),
@@ -239,7 +228,7 @@ pub fn get<'src>(dict: &Value<'src>, key: &Value<'src>) -> Value<'src> {
     }
 }
 
-pub fn last<'src>(ordered: &Value<'src>) -> Value<'src> {
+pub fn last(ordered: &Value) -> Value {
     match ordered {
         Value::List(list) => match list.last() {
             Some(x) => x.clone(),
@@ -253,12 +242,8 @@ pub fn last<'src>(ordered: &Value<'src>) -> Value<'src> {
     }
 }
 
-pub fn or<'src>(x: &Value<'src>, y: &Value<'src>) -> Value<'src> {
-    Value::Boolean(x.bool() || y.bool())
-}
-
 // TODO: fix this: x is a list of all the args passed to Ludus's print!
-pub fn print<'src>(x: &Value<'src>) -> Value<'src> {
+pub fn print(x: &Value) -> Value {
     let Value::List(args) = x else {
         unreachable!("internal Ludus error")
     };
@@ -271,30 +256,32 @@ pub fn print<'src>(x: &Value<'src>) -> Value<'src> {
     Value::Keyword("ok")
 }
 
-pub fn show<'src>(x: &Value<'src>) -> Value<'src> {
-    Value::AllocatedString(Rc::new(format!("{x}")))
+pub fn show(x: &Value) -> Value {
+    Value::String(Rc::new(format!("{x}")))
 }
 
-pub fn rest<'src>(ordered: &Value<'src>) -> Value<'src> {
+pub fn rest(ordered: &Value) -> Value {
     match ordered {
-        Value::List(list) => Value::List(list.clone().split_at(1).1),
-        Value::Tuple(tuple) => Value::List(Vector::from_iter(tuple.iter().next().cloned())),
+        Value::List(list) => Value::List(Box::new(list.clone().split_at(1).1)),
+        Value::Tuple(tuple) => {
+            Value::List(Box::new(Vector::from_iter(tuple.iter().next().cloned())))
+        }
         _ => unreachable!("internal Ludus error"),
     }
 }
 
-pub fn count<'src>(coll: &Value<'src>) -> Value<'src> {
+pub fn count(coll: &Value) -> Value {
     match coll {
         Value::Dict(d) => Value::Number(d.len() as f64),
         Value::List(l) => Value::Number(l.len() as f64),
         Value::Tuple(t) => Value::Number(t.len() as f64),
-        Value::AllocatedString(s) => Value::Number(s.len() as f64),
-        Value::InternedString(s) => Value::Number(s.len() as f64),
+        Value::String(s) => Value::Number(s.len() as f64),
+        Value::Interned(s) => Value::Number(s.len() as f64),
         _ => unreachable!("internal Ludus error"),
     }
 }
 
-pub fn range<'src>(start: &Value<'src>, end: &Value<'src>) -> Value<'src> {
+pub fn range(start: &Value, end: &Value) -> Value {
     match (start, end) {
         (Value::Number(start), Value::Number(end)) => {
             let start = *start as isize;
@@ -303,25 +290,25 @@ pub fn range<'src>(start: &Value<'src>, end: &Value<'src>) -> Value<'src> {
             for n in start..end {
                 range.push_back(Value::Number(n as f64))
             }
-            Value::List(range)
+            Value::List(Box::new(range))
         }
         _ => unreachable!("internal Ludus error"),
     }
 }
 
-pub fn slice<'src>(ordered: &Value<'src>, start: &Value<'src>, end: &Value<'src>) -> Value<'src> {
+pub fn slice(ordered: &Value, start: &Value, end: &Value) -> Value {
     match (ordered, start, end) {
         (Value::List(list), Value::Number(start), Value::Number(end)) => {
             let mut newlist = list.clone();
             let start = std::cmp::max(*start as usize, 0);
             let end = std::cmp::min(*end as usize, list.len());
-            Value::List(newlist.slice(start..end))
+            Value::List(Box::new(newlist.slice(start..end)))
         }
         // TODO: figure out something better to do than return an empty string on a bad slice
-        (Value::AllocatedString(string), Value::Number(start), Value::Number(end)) => {
+        (Value::String(string), Value::Number(start), Value::Number(end)) => {
             let start = std::cmp::max(*start as usize, 0);
             let end = std::cmp::min(*end as usize, string.len());
-            Value::AllocatedString(Rc::new(
+            Value::String(Rc::new(
                 string
                     .clone()
                     .as_str()
@@ -330,19 +317,19 @@ pub fn slice<'src>(ordered: &Value<'src>, start: &Value<'src>, end: &Value<'src>
                     .to_string(),
             ))
         }
-        (Value::InternedString(string), Value::Number(start), Value::Number(end)) => {
+        (Value::Interned(string), Value::Number(start), Value::Number(end)) => {
             let start = std::cmp::max(*start as usize, 0);
             let end = std::cmp::min(*end as usize, string.len());
-            Value::AllocatedString(Rc::new(string.get(start..end).unwrap_or("").to_string()))
+            Value::String(Rc::new(string.get(start..end).unwrap_or("").to_string()))
         }
         _ => unreachable!("internal Ludus error"),
     }
 }
 
-pub fn list<'src>(x: &Value<'src>) -> Value<'src> {
+pub fn list(x: &Value) -> Value {
     match x {
         Value::List(_) => x.clone(),
-        Value::Tuple(t) => Value::List(Vector::from_iter(t.iter().cloned())),
+        Value::Tuple(t) => Value::List(Box::new(Vector::from_iter(t.iter().cloned()))),
         Value::Dict(d) => {
             let kvs = d.iter();
             let mut list = vector![];
@@ -350,292 +337,301 @@ pub fn list<'src>(x: &Value<'src>) -> Value<'src> {
                 let kv = Value::Tuple(Rc::new(vec![Value::Keyword(key), value.clone()]));
                 list.push_back(kv);
             }
-            Value::List(list)
+            Value::List(Box::new(list))
         }
-        _ => Value::List(vector![x.clone()]),
+        _ => Value::List(Box::new(vector![x.clone()])),
     }
 }
 
-pub fn number<'src>(x: &Value<'src>) -> Value<'src> {
+pub fn number(x: &Value) -> Value {
     match x {
-        Value::InternedString(string) => match string.parse::<f64>() {
+        Value::Interned(string) => match string.parse::<f64>() {
             Ok(n) => Value::Tuple(Rc::new(vec![Value::Keyword("ok"), Value::Number(n)])),
             Err(_) => Value::Tuple(Rc::new(vec![
                 Value::Keyword("err"),
-                Value::AllocatedString(Rc::new(format!("could not parse `{string}` as a number"))),
+                Value::String(Rc::new(format!("could not parse `{string}` as a number"))),
             ])),
         },
-        Value::AllocatedString(string) => match string.parse::<f64>() {
+        Value::String(string) => match string.parse::<f64>() {
             Ok(n) => Value::Tuple(Rc::new(vec![Value::Keyword("ok"), Value::Number(n)])),
             Err(_) => Value::Tuple(Rc::new(vec![
                 Value::Keyword("err"),
-                Value::AllocatedString(Rc::new(format!("could not parse `{string}` as a number"))),
+                Value::String(Rc::new(format!("could not parse `{string}` as a number"))),
             ])),
         },
         _ => unreachable!("internal Ludus error"),
     }
 }
 
-pub fn r#type<'src>(x: &Value<'src>) -> Value<'src> {
+pub fn r#type(x: &Value) -> Value {
     match x {
         Value::Nil => Value::Keyword("nil"),
         Value::Number(_) => Value::Keyword("number"),
-        Value::Boolean(_) => Value::Keyword("boolean"),
+        Value::True | Value::False => Value::Keyword("bool"),
         Value::Keyword(_) => Value::Keyword("keyword"),
         Value::Tuple(_) => Value::Keyword("tuple"),
-        Value::InternedString(_) => Value::Keyword("string"),
-        Value::AllocatedString(_) => Value::Keyword("string"),
+        Value::Interned(_) => Value::Keyword("string"),
+        Value::String(_) => Value::Keyword("string"),
         Value::List(_) => Value::Keyword("list"),
         Value::Dict(_) => Value::Keyword("dict"),
         Value::Fn(_) => Value::Keyword("fn"),
-        Value::Box(_, _) => Value::Keyword("box"),
-        Value::Placeholder => unreachable!("internal Ludus error"),
-        Value::Args(_) => unreachable!("internal Ludus error"),
-        Value::Base(_) => Value::Keyword("fn"),
-        Value::Recur(..) => unreachable!("internal Ludus error"),
-        Value::FnDecl(_) => Value::Keyword("fn"),
+        Value::Box(_) => Value::Keyword("box"),
+        Value::BaseFn(_) => Value::Keyword("fn"),
+        Value::Partial(_) => Value::Keyword("fn"),
+        Value::Nothing => unreachable!(),
     }
 }
 
-pub fn split<'src>(source: &Value<'src>, splitter: &Value) -> Value<'src> {
+pub fn split(source: &Value, splitter: &Value) -> Value {
     match (source, splitter) {
-        (Value::AllocatedString(source), Value::AllocatedString(splitter)) => {
+        (Value::String(source), Value::String(splitter)) => {
             let parts = source.split_terminator(splitter.as_str());
             let mut list = vector![];
             for part in parts {
-                list.push_back(Value::AllocatedString(Rc::new(part.to_string())));
+                list.push_back(Value::String(Rc::new(part.to_string())));
             }
-            Value::List(list)
+            Value::List(Box::new(list))
         }
-        (Value::AllocatedString(source), Value::InternedString(splitter)) => {
+        (Value::String(source), Value::Interned(splitter)) => {
             let parts = source.split_terminator(splitter);
             let mut list = vector![];
             for part in parts {
-                list.push_back(Value::AllocatedString(Rc::new(part.to_string())));
+                list.push_back(Value::String(Rc::new(part.to_string())));
             }
-            Value::List(list)
+            Value::List(Box::new(list))
         }
-        (Value::InternedString(source), Value::AllocatedString(splitter)) => {
+        (Value::Interned(source), Value::String(splitter)) => {
             let parts = source.split_terminator(splitter.as_str());
             let mut list = vector![];
             for part in parts {
-                list.push_back(Value::AllocatedString(Rc::new(part.to_string())));
+                list.push_back(Value::String(Rc::new(part.to_string())));
             }
-            Value::List(list)
+            Value::List(Box::new(list))
         }
-        (Value::InternedString(source), Value::InternedString(splitter)) => {
+        (Value::Interned(source), Value::Interned(splitter)) => {
             let parts = source.split_terminator(splitter);
             let mut list = vector![];
             for part in parts {
-                list.push_back(Value::AllocatedString(Rc::new(part.to_string())));
+                list.push_back(Value::String(Rc::new(part.to_string())));
             }
-            Value::List(list)
+            Value::List(Box::new(list))
         }
         _ => unreachable!("internal Ludus error"),
     }
 }
 
-pub fn upcase<'src>(string: &Value<'src>) -> Value<'src> {
+pub fn upcase(string: &Value) -> Value {
     match string {
-        Value::AllocatedString(string) => Value::AllocatedString(Rc::new(string.to_uppercase())),
-        Value::InternedString(string) => Value::AllocatedString(Rc::new(string.to_uppercase())),
+        Value::String(string) => Value::String(Rc::new(string.to_uppercase())),
+        Value::Interned(string) => Value::String(Rc::new(string.to_uppercase())),
         _ => unreachable!("internal Ludus error"),
     }
 }
 
-pub fn downcase<'src>(string: &Value<'src>) -> Value<'src> {
+pub fn downcase(string: &Value) -> Value {
     match string {
-        Value::AllocatedString(string) => Value::AllocatedString(Rc::new(string.to_lowercase())),
-        Value::InternedString(string) => Value::AllocatedString(Rc::new(string.to_lowercase())),
+        Value::String(string) => Value::String(Rc::new(string.to_lowercase())),
+        Value::Interned(string) => Value::String(Rc::new(string.to_lowercase())),
         _ => unreachable!("internal Ludus error"),
     }
 }
 
-pub fn trim<'src>(string: &Value<'src>) -> Value<'src> {
+pub fn trim(string: &Value) -> Value {
     match string {
-        Value::AllocatedString(string) => {
-            Value::AllocatedString(Rc::new(string.trim().to_string()))
-        }
-        Value::InternedString(string) => Value::AllocatedString(Rc::new(string.trim().to_string())),
+        Value::String(string) => Value::String(Rc::new(string.trim().to_string())),
+        Value::Interned(string) => Value::String(Rc::new(string.trim().to_string())),
         _ => unreachable!("internal Ludus error"),
     }
 }
 
-pub fn triml<'src>(string: &Value<'src>) -> Value<'src> {
+pub fn triml(string: &Value) -> Value {
     match string {
-        Value::AllocatedString(string) => {
-            Value::AllocatedString(Rc::new(string.trim_start().to_string()))
-        }
-        Value::InternedString(string) => {
-            Value::AllocatedString(Rc::new(string.trim_start().to_string()))
-        }
+        Value::String(string) => Value::String(Rc::new(string.trim_start().to_string())),
+        Value::Interned(string) => Value::String(Rc::new(string.trim_start().to_string())),
         _ => unreachable!("internal Ludus error"),
     }
 }
 
-pub fn trimr<'src>(string: &Value<'src>) -> Value<'src> {
+pub fn trimr(string: &Value) -> Value {
     match string {
-        Value::AllocatedString(string) => {
-            Value::AllocatedString(Rc::new(string.trim_end().to_string()))
-        }
-        Value::InternedString(string) => {
-            Value::AllocatedString(Rc::new(string.trim_end().to_string()))
-        }
+        Value::String(string) => Value::String(Rc::new(string.trim_end().to_string())),
+        Value::Interned(string) => Value::String(Rc::new(string.trim_end().to_string())),
         _ => unreachable!("internal Ludus error"),
     }
 }
 
-pub fn atan_2<'src>(x: &Value, y: &Value) -> Value<'src> {
+pub fn atan_2(x: &Value, y: &Value) -> Value {
     match (x, y) {
         (Value::Number(x), Value::Number(y)) => Value::Number(x.atan2(*y)),
         _ => unreachable!("internal Ludus error"),
     }
 }
 
-pub fn ceil<'src>(x: &Value) -> Value<'src> {
+pub fn ceil(x: &Value) -> Value {
     match x {
         Value::Number(x) => Value::Number(x.ceil()),
         _ => unreachable!("internal Ludus error"),
     }
 }
 
-pub fn cos<'src>(x: &Value) -> Value<'src> {
+pub fn cos(x: &Value) -> Value {
     match x {
         Value::Number(x) => Value::Number(x.cos()),
         _ => unreachable!("internal Ludus error"),
     }
 }
 
-pub fn floor<'src>(x: &Value) -> Value<'src> {
+pub fn floor(x: &Value) -> Value {
     match x {
         Value::Number(x) => Value::Number(x.floor()),
         _ => unreachable!("internal Ludus error"),
     }
 }
 
-pub fn random<'src>() -> Value<'src> {
+pub fn random() -> Value {
     Value::Number(ran_f64())
 }
 
-pub fn round<'src>(x: &Value) -> Value<'src> {
+pub fn round(x: &Value) -> Value {
     match x {
         Value::Number(x) => Value::Number(x.round()),
         _ => unreachable!("internal Ludus error"),
     }
 }
 
-pub fn sin<'src>(x: &Value) -> Value<'src> {
+pub fn sin(x: &Value) -> Value {
     match x {
         Value::Number(x) => Value::Number(x.sin()),
         _ => unreachable!("internal Ludus error"),
     }
 }
 
-pub fn sqrt<'src>(x: &Value) -> Value<'src> {
+pub fn sqrt(x: &Value) -> Value {
     match x {
         Value::Number(x) => Value::Number(x.sqrt()),
         _ => unreachable!("internal Ludus error"),
     }
 }
 
-pub fn tan<'src>(x: &Value) -> Value<'src> {
+pub fn tan(x: &Value) -> Value {
     match x {
         Value::Number(x) => Value::Number(x.tan()),
         _ => unreachable!("internal Ludus error"),
     }
 }
 
-pub fn gt<'src>(x: &Value, y: &Value) -> Value<'src> {
+pub fn gt(x: &Value, y: &Value) -> Value {
     match (x, y) {
-        (Value::Number(x), Value::Number(y)) => Value::Boolean(x > y),
+        (Value::Number(x), Value::Number(y)) => {
+            if x > y {
+                Value::True
+            } else {
+                Value::False
+            }
+        }
         _ => unreachable!("internal Ludus error"),
     }
 }
 
-pub fn gte<'src>(x: &Value, y: &Value) -> Value<'src> {
+pub fn gte(x: &Value, y: &Value) -> Value {
     match (x, y) {
-        (Value::Number(x), Value::Number(y)) => Value::Boolean(x >= y),
+        (Value::Number(x), Value::Number(y)) => {
+            if x >= y {
+                Value::True
+            } else {
+                Value::False
+            }
+        }
         _ => unreachable!("internal Ludus error"),
     }
 }
 
-pub fn lt<'src>(x: &Value, y: &Value) -> Value<'src> {
+pub fn lt(x: &Value, y: &Value) -> Value {
     match (x, y) {
-        (Value::Number(x), Value::Number(y)) => Value::Boolean(x < y),
+        (Value::Number(x), Value::Number(y)) => {
+            if x < y {
+                Value::True
+            } else {
+                Value::False
+            }
+        }
         _ => unreachable!("internal Ludus error"),
     }
 }
 
-pub fn lte<'src>(x: &Value, y: &Value) -> Value<'src> {
+pub fn lte(x: &Value, y: &Value) -> Value {
     match (x, y) {
-        (Value::Number(x), Value::Number(y)) => Value::Boolean(x <= y),
+        (Value::Number(x), Value::Number(y)) => {
+            if x <= y {
+                Value::True
+            } else {
+                Value::False
+            }
+        }
         _ => unreachable!("internal Ludus error"),
     }
 }
 
-pub fn r#mod<'src>(x: &Value, y: &Value) -> Value<'src> {
+pub fn r#mod(x: &Value, y: &Value) -> Value {
     match (x, y) {
         (Value::Number(x), Value::Number(y)) => Value::Number(x % y),
         _ => unreachable!("internal Ludus error"),
     }
 }
 
-pub fn base<'src>() -> Vec<(String, Value<'src>)> {
+pub fn base() -> Value {
     let members = vec![
-        ("add", Value::Base(BaseFn::Binary(add))),
-        ("and", Value::Base(BaseFn::Binary(and))),
-        ("append", Value::Base(BaseFn::Binary(append))),
-        ("assoc", Value::Base(BaseFn::Ternary(assoc))),
-        ("at", Value::Base(BaseFn::Binary(at))),
-        ("atan_2", Value::Base(BaseFn::Binary(atan_2))),
-        ("bool", Value::Base(BaseFn::Unary(r#bool))),
-        ("ceil", Value::Base(BaseFn::Unary(ceil))),
-        ("chars", Value::Base(BaseFn::Unary(chars))),
-        ("concat", Value::Base(BaseFn::Binary(concat))),
-        ("cos", Value::Base(BaseFn::Unary(cos))),
-        ("count", Value::Base(BaseFn::Unary(count))),
-        ("dec", Value::Base(BaseFn::Unary(dec))),
-        ("dissoc", Value::Base(BaseFn::Binary(dissoc))),
-        ("div", Value::Base(BaseFn::Binary(div))),
-        ("doc!", Value::Base(BaseFn::Unary(doc))),
-        ("downcase", Value::Base(BaseFn::Unary(downcase))),
-        ("eq?", Value::Base(BaseFn::Binary(eq))),
-        ("first", Value::Base(BaseFn::Unary(first))),
-        ("floor", Value::Base(BaseFn::Unary(floor))),
-        ("get", Value::Base(BaseFn::Binary(get))),
-        ("gt?", Value::Base(BaseFn::Binary(gt))),
-        ("gte?", Value::Base(BaseFn::Binary(gte))),
-        ("inc", Value::Base(BaseFn::Unary(inc))),
-        ("last", Value::Base(BaseFn::Unary(last))),
-        ("list", Value::Base(BaseFn::Unary(list))),
-        ("lt?", Value::Base(BaseFn::Binary(lt))),
-        ("lte?", Value::Base(BaseFn::Binary(lte))),
-        ("mod", Value::Base(BaseFn::Binary(r#mod))),
-        ("mult", Value::Base(BaseFn::Binary(mult))),
-        ("number", Value::Base(BaseFn::Unary(number))),
-        ("or", Value::Base(BaseFn::Binary(or))),
+        ("add", Value::BaseFn(BaseFn::Binary(add))),
+        ("append", Value::BaseFn(BaseFn::Binary(append))),
+        ("assoc", Value::BaseFn(BaseFn::Ternary(assoc))),
+        ("at", Value::BaseFn(BaseFn::Binary(at))),
+        ("atan_2", Value::BaseFn(BaseFn::Binary(atan_2))),
+        ("bool", Value::BaseFn(BaseFn::Unary(r#bool))),
+        ("ceil", Value::BaseFn(BaseFn::Unary(ceil))),
+        ("chars", Value::BaseFn(BaseFn::Unary(chars))),
+        ("concat", Value::BaseFn(BaseFn::Binary(concat))),
+        ("cos", Value::BaseFn(BaseFn::Unary(cos))),
+        ("count", Value::BaseFn(BaseFn::Unary(count))),
+        ("dec", Value::BaseFn(BaseFn::Unary(dec))),
+        ("dissoc", Value::BaseFn(BaseFn::Binary(dissoc))),
+        ("div", Value::BaseFn(BaseFn::Binary(div))),
+        ("doc!", Value::BaseFn(BaseFn::Unary(doc))),
+        ("downcase", Value::BaseFn(BaseFn::Unary(downcase))),
+        ("eq?", Value::BaseFn(BaseFn::Binary(eq))),
+        ("first", Value::BaseFn(BaseFn::Unary(first))),
+        ("floor", Value::BaseFn(BaseFn::Unary(floor))),
+        ("get", Value::BaseFn(BaseFn::Binary(get))),
+        ("gt?", Value::BaseFn(BaseFn::Binary(gt))),
+        ("gte?", Value::BaseFn(BaseFn::Binary(gte))),
+        ("inc", Value::BaseFn(BaseFn::Unary(inc))),
+        ("last", Value::BaseFn(BaseFn::Unary(last))),
+        ("list", Value::BaseFn(BaseFn::Unary(list))),
+        ("lt?", Value::BaseFn(BaseFn::Binary(lt))),
+        ("lte?", Value::BaseFn(BaseFn::Binary(lte))),
+        ("mod", Value::BaseFn(BaseFn::Binary(r#mod))),
+        ("mult", Value::BaseFn(BaseFn::Binary(mult))),
+        ("number", Value::BaseFn(BaseFn::Unary(number))),
         ("pi", Value::Number(std::f64::consts::PI)),
-        ("print!", Value::Base(BaseFn::Unary(print))),
-        ("random", Value::Base(BaseFn::Nullary(random))),
-        ("range", Value::Base(BaseFn::Binary(range))),
-        ("rest", Value::Base(BaseFn::Unary(rest))),
-        ("round", Value::Base(BaseFn::Unary(round))),
-        ("show", Value::Base(BaseFn::Unary(show))),
-        ("sin", Value::Base(BaseFn::Unary(sin))),
-        ("slice", Value::Base(BaseFn::Ternary(slice))),
-        ("split", Value::Base(BaseFn::Binary(split))),
-        ("sqrt", Value::Base(BaseFn::Unary(sqrt))),
+        ("print!", Value::BaseFn(BaseFn::Unary(print))),
+        ("random", Value::BaseFn(BaseFn::Nullary(random))),
+        ("range", Value::BaseFn(BaseFn::Binary(range))),
+        ("rest", Value::BaseFn(BaseFn::Unary(rest))),
+        ("round", Value::BaseFn(BaseFn::Unary(round))),
+        ("show", Value::BaseFn(BaseFn::Unary(show))),
+        ("sin", Value::BaseFn(BaseFn::Unary(sin))),
+        ("slice", Value::BaseFn(BaseFn::Ternary(slice))),
+        ("split", Value::BaseFn(BaseFn::Binary(split))),
+        ("sqrt", Value::BaseFn(BaseFn::Unary(sqrt))),
         ("sqrt_2", Value::Number(std::f64::consts::SQRT_2)),
-        ("store!", Value::Base(BaseFn::Binary(store))),
-        ("sub", Value::Base(BaseFn::Binary(sub))),
-        ("tan", Value::Base(BaseFn::Unary(tan))),
-        ("trim", Value::Base(BaseFn::Unary(trim))),
-        ("triml", Value::Base(BaseFn::Unary(triml))),
-        ("trimr", Value::Base(BaseFn::Unary(trimr))),
-        ("type", Value::Base(BaseFn::Unary(r#type))),
-        ("unbox", Value::Base(BaseFn::Unary(unbox))),
-        ("upcase", Value::Base(BaseFn::Unary(upcase))),
+        ("store!", Value::BaseFn(BaseFn::Binary(store))),
+        ("sub", Value::BaseFn(BaseFn::Binary(sub))),
+        ("tan", Value::BaseFn(BaseFn::Unary(tan))),
+        ("trim", Value::BaseFn(BaseFn::Unary(trim))),
+        ("triml", Value::BaseFn(BaseFn::Unary(triml))),
+        ("trimr", Value::BaseFn(BaseFn::Unary(trimr))),
+        ("type", Value::BaseFn(BaseFn::Unary(r#type))),
+        ("unbox", Value::BaseFn(BaseFn::Unary(unbox))),
+        ("upcase", Value::BaseFn(BaseFn::Unary(upcase))),
     ];
-    let pkg = Value::Dict(HashMap::from(members));
-    vec![("base".to_string(), pkg)]
+    Value::Dict(Box::new(HashMap::from(members)))
 }

src/lexer.rs

@@ -56,7 +56,7 @@ pub fn lexer(
         "nil" => Token::Nil,
         // todo: hard code these as type constructors
         "as" | "box" | "do" | "else" | "fn" | "if" | "let" | "loop" | "match" | "panic!"
-        | "recur" | "repeat" | "then" | "when" | "with" => Token::Reserved(word),
+        | "recur" | "repeat" | "then" | "when" | "with" | "or" | "and" => Token::Reserved(word),
         _ => Token::Word(word),
     });
 

src/main.rs

@@ -1,148 +1,30 @@
-// an implementation of Ludus
-
-// curently left undone (and not adding for a while yet):
-// * sets
-// * interpolated strings & string patterns
-// * pkgs, namespaces, imports, `use` forms
-// * with forms
-// * test forms
-// * ignored words
-
-// todo:
-// * [x] rewrite fn parser to use chumsky::Recursive::declare/define
-//   - [x] do this to extract/simplify/DRY things like tuple patterns, fn clauses, etc.
-// * [x] Work around chumsky::Stream::from_iter().spanned disappearing in most recent version
-// * [x] investigate using labels (which is behind a compiler flag, somehow)
-// * [ ] write parsing errors
-// * [ ] wire up Ariadne parsing errors
-// * [x] add stack traces and code locations to panics
-// * [x] validation
-// * [x] break this out into multiple files
-// * [x] write a tree-walk VM
-//   - [x] learn how to deal with lifetimes
-//   - [x] with stack mechanics and refcounting
-//   - [ ] with tail-call optimization (nb: this may not be possible w/ a TW-VM)
-//   - [ ] with all the necessary forms for current Ludus
-//     * [x] guards in match clauses
-//     * [x] `as` patterns
-//     * [x] splat patterns in tuples, lists, dicts
-//     * [x] splats in list and dict literals
-//     * [x] `loop` and `recur`
-//     * [x] string patterns
-//     * [x] string interpolation
-//     * [x] docstrings
-// * [x] write `base` in Rust
-// * [ ] turn this into a library function
-// * [ ] compile this into WASM
-// * [ ] perf testing
-
 use chumsky::{input::Stream, prelude::*};
-use rust_embed::Embed;
+use std::env;
 
-mod spans;
-
-mod lexer;
-use crate::lexer::*;
-
-mod value;
-use crate::value::*;
-
-mod parser;
-use crate::parser::*;
+const DEBUG_COMPILE: bool = true;
+const DEBUG_RUN: bool = true;
 
 mod base;
-use crate::base::*;
+
+mod spans;
+use crate::spans::Spanned;
+
+mod lexer;
+use crate::lexer::lexer;
+
+mod parser;
+use crate::parser::{parser, Ast};
 
 mod validator;
-use crate::validator::*;
 
-mod process;
-use crate::process::*;
+mod compiler;
+use crate::compiler::Compiler;
 
-mod errors;
-use crate::errors::*;
+mod pattern;
+mod value;
 
-#[derive(Embed)]
-#[folder = "assets/"]
-struct Asset;
-
-pub fn prelude<'src>() -> (
-    Vec<(String, Value<'src>)>,
-    std::collections::HashMap<*const Ast, FnInfo>,
-) {
-    let prelude = Asset::get("prelude.ld").unwrap().data.into_owned();
-    // we know for sure Prelude should live through the whole run of the program
-    let leaked = Box::leak(Box::new(prelude));
-    let prelude = std::str::from_utf8(leaked).unwrap();
-
-    let (ptoks, perrs) = lexer().parse(prelude).into_output_errors();
-    if !perrs.is_empty() {
-        println!("Errors lexing Prelude");
-        println!("{:?}", perrs);
-        panic!();
-    }
-
-    let ptoks = ptoks.unwrap();
-
-    let (p_ast, perrs) = parser()
-        .parse(Stream::from_iter(ptoks).map((0..prelude.len()).into(), |(t, s)| (t, s)))
-        .into_output_errors();
-    if !perrs.is_empty() {
-        println!("Errors parsing Prelude");
-        println!("{:?}", perrs);
-        panic!();
-    }
-
-    let prelude_parsed = Box::leak(Box::new(p_ast.unwrap()));
-    let base_pkg = base();
-
-    let mut v6or = Validator::new(
-        &prelude_parsed.0,
-        prelude_parsed.1,
-        "prelude",
-        prelude,
-        &base_pkg,
-    );
-    v6or.validate();
-
-    if !v6or.errors.is_empty() {
-        report_invalidation(v6or.errors);
-        panic!("interal Ludus error: invalid prelude")
-    }
-
-    let mut base_ctx = Process::<'src> {
-        input: "prelude",
-        src: prelude,
-        locals: base_pkg.clone(),
-        ast: &prelude_parsed.0,
-        span: prelude_parsed.1,
-        prelude: vec![],
-        fn_info: v6or.fn_info,
-    };
-
-    let prelude = base_ctx.eval();
-
-    let mut p_ctx = vec![];
-
-    match prelude {
-        Ok(Value::Dict(p_dict)) => {
-            for (key, value) in p_dict.iter() {
-                p_ctx.push((key.to_string(), value.clone()))
-            }
-        }
-        Ok(_) => {
-            println!("Bad Prelude export");
-            panic!();
-        }
-        Err(LErr { msg, .. }) => {
-            println!("Error running Prelude");
-            println!("{:?}", msg);
-            panic!();
-        }
-    };
-
-    (p_ctx, base_ctx.fn_info)
-}
+mod vm;
+use vm::Vm;
 
 pub fn run(src: &'static str) {
     let (tokens, lex_errs) = lexer().parse(src).into_output_errors();
@@ -150,59 +32,61 @@ pub fn run(src: &'static str) {
         println!("{:?}", lex_errs);
         return;
     }
+
     let tokens = tokens.unwrap();
-    let to_parse = tokens.clone();
 
     let (parse_result, parse_errors) = parser()
-        .parse(Stream::from_iter(to_parse).map((0..src.len()).into(), |(t, s)| (t, s)))
+        .parse(Stream::from_iter(tokens).map((0..src.len()).into(), |(t, s)| (t, s)))
         .into_output_errors();
     if !parse_errors.is_empty() {
         println!("{:?}", parse_errors);
         return;
     }
 
-    let parsed = parse_result.unwrap();
+    // ::sigh:: The AST should be 'static
+    // This simplifies lifetimes, and
+    // in any event, the AST should live forever
+    let parsed: &'static Spanned<Ast> = Box::leak(Box::new(parse_result.unwrap()));
 
-    let (prelude_ctx, mut prelude_fn_info) = prelude();
-
-    let mut v6or = Validator::new(&parsed.0, parsed.1, "script", src, &prelude_ctx);
-
-    v6or.validate();
-
-    if !v6or.errors.is_empty() {
-        report_invalidation(v6or.errors);
-        return;
+    let mut compiler = Compiler::new(parsed, "test", src, None);
+    compiler.compile();
+    if DEBUG_COMPILE {
+        println!("=== source code ===");
+        println!("{src}");
+        compiler.disassemble();
+        println!("\n\n")
     }
 
-    prelude_fn_info.extend(&mut v6or.fn_info.into_iter());
+    if DEBUG_RUN {
+        println!("=== vm run: test ===");
+    }
 
-    let mut proc = Process {
-        input: "script",
-        src,
-        locals: vec![],
-        prelude: prelude_ctx,
-        ast: &parsed.0,
-        span: parsed.1,
-        fn_info: prelude_fn_info,
+    let vm_chunk = compiler.chunk;
+
+    let mut vm = Vm::new(vm_chunk);
+    let result = vm.run();
+    let output = match result {
+        Ok(val) => val.show(),
+        Err(panic) => format!("Ludus panicked! {panic}"),
     };
-
-    let result = proc.eval();
-
-    match result {
-        Ok(result) => println!("{}", result),
-        Err(err) => report_panic(err),
-    }
+    vm.print_stack();
+    println!("{output}");
 }
 
 pub fn main() {
+    // env::set_var("RUST_BACKTRACE", "1");
     let src = "
-loop (100000, 1) with {
-    (1, acc) -> acc
-    (n, acc) -> recur (dec (n), add (n, acc))
+let x = {
+    match #{:a 1, :b 2, :c 3} with {
+        #{a} -> :one
+        #{a, b, :c 3} -> :two
+        #{a, b, c} -> :three
+        (1, 2, 3) -> :thing
+        (4, 5, (6, 7, a)) -> if or (true, false, false, true) then :thing_1 else :thing_2
+        ([:a, :b, :c, [:d, [:e, (:f, :g)]]]) -> if or (true, false, false, true) then :thing_1 else :thing_2
+    }
 }
-    ";
+
+   ";
     run(src);
-    // struct_scalpel::print_dissection_info::<value::Value>()
-    // struct_scalpel::print_dissection_info::<parser::Ast>();
-    // println!("{}", std::mem::size_of::<parser::Ast>())
 }

src/memory_sandbox.rs

@@ -0,0 +1,58 @@
+use imbl::{HashMap, Vector};
+use index_vec::Idx;
+use std::cell::RefCell;
+use std::ops::Range;
+use std::rc::Rc;
+
+struct Word(&'static str);
+
+struct Keyword(&'static str);
+
+struct Interned(&'static str);
+
+enum StringPart {
+    Word(&'static str),
+    Data(&'static str),
+    Inline(&'static str),
+}
+
+#[derive(Clone, Debug, PartialEq)]
+struct LBox {
+    name: usize,
+    cell: RefCell<Value>,
+}
+
+#[derive(Clone, Debug, PartialEq)]
+struct Fn {
+    name: &'static str,
+    body: Vec<String>,
+    //...etc
+}
+
+#[derive(Clone, Debug, PartialEq)]
+enum Value {
+    Nil,
+    Placeholder,
+    Boolean(bool),
+    Keyword(usize),
+    Interned(usize),
+    FnDecl(usize),
+    String(Rc<String>),
+    Number(f64),
+    Tuple(Rc<Vec<Value>>),
+    List(Box<Vector<Value>>),
+    Dict(Box<HashMap<&'static str, Value>>),
+    Box(Rc<LBox>),
+    Fn(Rc<RefCell<Fn>>),
+}
+
+fn futz() {
+    let foo: &'static str = "foo";
+    let baz: Vec<u8> = vec![];
+    let bar: Range<usize> = 1..3;
+    let quux: Vector<u8> = Vector::new();
+    let fuzz = Rc::new(quux);
+    let blah = Box::new(foo);
+    let val = Value::Number(12.09);
+    let foo: f64 = 12.0;
+}

src/old_main.rs

@@ -0,0 +1,212 @@
+// an implementation of Ludus
+
+// curently left undone (and not adding for a while yet):
+// * sets
+// * interpolated strings & string patterns
+// * pkgs, namespaces, imports, `use` forms
+// * with forms
+// * test forms
+// * ignored words
+
+// todo:
+// * [x] rewrite fn parser to use chumsky::Recursive::declare/define
+//   - [x] do this to extract/simplify/DRY things like tuple patterns, fn clauses, etc.
+// * [x] Work around chumsky::Stream::from_iter().spanned disappearing in most recent version
+// * [x] investigate using labels (which is behind a compiler flag, somehow)
+// * [ ] write parsing errors
+// * [ ] wire up Ariadne parsing errors
+// * [x] add stack traces and code locations to panics
+// * [x] validation
+// * [x] break this out into multiple files
+// * [x] write a tree-walk VM
+//   - [x] learn how to deal with lifetimes
+//   - [x] with stack mechanics and refcounting
+//   - [ ] with tail-call optimization (nb: this may not be possible w/ a TW-VM)
+//   - [ ] with all the necessary forms for current Ludus
+//     * [x] guards in match clauses
+//     * [x] `as` patterns
+//     * [x] splat patterns in tuples, lists, dicts
+//     * [x] splats in list and dict literals
+//     * [x] `loop` and `recur`
+//     * [x] string patterns
+//     * [x] string interpolation
+//     * [x] docstrings
+// * [x] write `base` in Rust
+// * [ ] turn this into a library function
+// * [ ] compile this into WASM
+// * [ ] perf testing
+
+use chumsky::{input::Stream, prelude::*};
+use rust_embed::Embed;
+
+mod spans;
+
+mod lexer;
+use crate::lexer::*;
+
+mod value;
+use crate::value::*;
+
+mod parser;
+use crate::parser::*;
+
+mod base;
+use crate::base::*;
+
+mod validator;
+use crate::validator::*;
+
+mod process;
+use crate::process::*;
+
+mod errors;
+use crate::errors::*;
+
+mod byte_values;
+mod compiler;
+mod memory_sandbox;
+
+#[derive(Embed)]
+#[folder = "assets/"]
+struct Asset;
+
+pub fn prelude<'src>() -> (
+    Vec<(String, Value<'src>)>,
+    std::collections::HashMap<*const Ast, FnInfo>,
+) {
+    let prelude = Asset::get("prelude.ld").unwrap().data.into_owned();
+    // we know for sure Prelude should live through the whole run of the program
+    let leaked = Box::leak(Box::new(prelude));
+    let prelude = std::str::from_utf8(leaked).unwrap();
+
+    let (ptoks, perrs) = lexer().parse(prelude).into_output_errors();
+    if !perrs.is_empty() {
+        println!("Errors lexing Prelude");
+        println!("{:?}", perrs);
+        panic!();
+    }
+
+    let ptoks = ptoks.unwrap();
+
+    let (p_ast, perrs) = parser()
+        .parse(Stream::from_iter(ptoks).map((0..prelude.len()).into(), |(t, s)| (t, s)))
+        .into_output_errors();
+    if !perrs.is_empty() {
+        println!("Errors parsing Prelude");
+        println!("{:?}", perrs);
+        panic!();
+    }
+
+    let prelude_parsed = Box::leak(Box::new(p_ast.unwrap()));
+    let base_pkg = base();
+
+    let mut v6or = Validator::new(
+        &prelude_parsed.0,
+        prelude_parsed.1,
+        "prelude",
+        prelude,
+        &base_pkg,
+    );
+    v6or.validate();
+
+    if !v6or.errors.is_empty() {
+        report_invalidation(v6or.errors);
+        panic!("interal Ludus error: invalid prelude")
+    }
+
+    let mut base_ctx = Process::<'src> {
+        input: "prelude",
+        src: prelude,
+        locals: base_pkg.clone(),
+        ast: &prelude_parsed.0,
+        span: prelude_parsed.1,
+        prelude: vec![],
+        fn_info: v6or.fn_info,
+    };
+
+    let prelude = base_ctx.eval();
+
+    let mut p_ctx = vec![];
+
+    match prelude {
+        Ok(Value::Dict(p_dict)) => {
+            for (key, value) in p_dict.iter() {
+                p_ctx.push((key.to_string(), value.clone()))
+            }
+        }
+        Ok(_) => {
+            println!("Bad Prelude export");
+            panic!();
+        }
+        Err(LErr { msg, .. }) => {
+            println!("Error running Prelude");
+            println!("{:?}", msg);
+            panic!();
+        }
+    };
+
+    (p_ctx, base_ctx.fn_info)
+}
+
+pub fn run(src: &'static str) {
+    let (tokens, lex_errs) = lexer().parse(src).into_output_errors();
+    if !lex_errs.is_empty() {
+        println!("{:?}", lex_errs);
+        return;
+    }
+    let tokens = tokens.unwrap();
+    let to_parse = tokens.clone();
+
+    let (parse_result, parse_errors) = parser()
+        .parse(Stream::from_iter(to_parse).map((0..src.len()).into(), |(t, s)| (t, s)))
+        .into_output_errors();
+    if !parse_errors.is_empty() {
+        println!("{:?}", parse_errors);
+        return;
+    }
+
+    let parsed = parse_result.unwrap();
+
+    let (prelude_ctx, mut prelude_fn_info) = prelude();
+
+    let mut v6or = Validator::new(&parsed.0, parsed.1, "script", src, &prelude_ctx);
+
+    v6or.validate();
+
+    if !v6or.errors.is_empty() {
+        report_invalidation(v6or.errors);
+        return;
+    }
+
+    prelude_fn_info.extend(&mut v6or.fn_info.into_iter());
+
+    let mut proc = Process {
+        input: "script",
+        src,
+        locals: vec![],
+        prelude: prelude_ctx,
+        ast: &parsed.0,
+        span: parsed.1,
+        fn_info: prelude_fn_info,
+    };
+
+    let result = proc.eval();
+
+    match result {
+        Ok(result) => println!("{}", result),
+        Err(err) => report_panic(err),
+    }
+}
+
+pub fn main() {
+    let src = "
+loop (100000, 1) with {
+    (1, acc) -> acc
+    (n, acc) -> recur (dec (n), add (n, acc))
+}
+    ";
+    run(src);
+    // struct_scalpel::print_dissection_info::<value::Value>()
+    // struct_scalpel::print_dissection_info::<parser::Ast>();
+    // println!("{}", std::mem::size_of::<parser::Ast>())
+}

src/old_value.rs

@@ -0,0 +1,193 @@
+use crate::base::*;
+use crate::parser::*;
+use crate::spans::*;
+use imbl::*;
+use std::cell::RefCell;
+use std::fmt;
+use std::rc::Rc;
+use struct_scalpel::Dissectible;
+
+#[derive(Clone, Debug)]
+pub struct Fn<'src> {
+    pub name: String,
+    pub body: &'src Vec<Spanned<Ast>>,
+    pub doc: Option<String>,
+    pub enclosing: Vec<(String, Value<'src>)>,
+    pub has_run: bool,
+    pub input: &'static str,
+    pub src: &'static str,
+}
+
+#[derive(Debug, Dissectible)]
+pub enum Value<'src> {
+    Nil,
+    Placeholder,
+    Boolean(bool),
+    Number(f64),
+    Keyword(&'static str),
+    InternedString(&'static str),
+    AllocatedString(Rc<String>),
+    // on the heap for now
+    Tuple(Rc<Vec<Self>>),
+    Args(Rc<Vec<Self>>),
+    List(Vector<Self>),
+    Dict(HashMap<&'static str, Self>),
+    Box(&'static str, Rc<RefCell<Self>>),
+    Fn(Rc<RefCell<Fn<'src>>>),
+    FnDecl(&'static str),
+    Base(BaseFn<'src>),
+    Recur(Vec<Self>),
+    // Set(HashSet<Self>),
+    // Sets are hard
+    // Sets require Eq
+    // Eq is not implemented on f64, because NaNs
+    // We could use ordered_float::NotNan
+    // Let's defer that
+    // We're not really using sets in Ludus
+
+    // Other things we're not implementing yet:
+    // pkgs, nses, tests
+}
+
+impl<'src> Clone for Value<'src> {
+    fn clone(&self) -> Value<'src> {
+        match self {
+            Value::Nil => Value::Nil,
+            Value::Boolean(b) => Value::Boolean(*b),
+            Value::InternedString(s) => Value::InternedString(s),
+            Value::AllocatedString(s) => Value::AllocatedString(s.clone()),
+            Value::Keyword(s) => Value::Keyword(s),
+            Value::Number(n) => Value::Number(*n),
+            Value::Tuple(t) => Value::Tuple(t.clone()),
+            Value::Args(a) => Value::Args(a.clone()),
+            Value::Fn(f) => Value::Fn(f.clone()),
+            Value::FnDecl(name) => Value::FnDecl(name),
+            Value::List(l) => Value::List(l.clone()),
+            Value::Dict(d) => Value::Dict(d.clone()),
+            Value::Box(name, b) => Value::Box(name, b.clone()),
+            Value::Placeholder => Value::Placeholder,
+            Value::Base(b) => Value::Base(b.clone()),
+            Value::Recur(..) => unreachable!(),
+        }
+    }
+}
+
+impl fmt::Display for Value<'_> {
+    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
+        match self {
+            Value::Nil => write!(f, "nil"),
+            Value::Boolean(b) => write!(f, "{b}"),
+            Value::Number(n) => write!(f, "{n}"),
+            Value::Keyword(k) => write!(f, ":{k}"),
+            Value::InternedString(s) => write!(f, "\"{s}\""),
+            Value::AllocatedString(s) => write!(f, "\"{s}\""),
+            Value::Fn(fun) => write!(f, "fn {}", fun.borrow().name),
+            Value::FnDecl(name) => write!(f, "fn {name}"),
+            Value::Tuple(t) | Value::Args(t) => write!(
+                f,
+                "({})",
+                t.iter()
+                    .map(|x| x.to_string())
+                    .collect::<Vec<_>>()
+                    .join(", ")
+            ),
+            Value::List(l) => write!(
+                f,
+                "[{}]",
+                l.iter()
+                    .map(|x| x.to_string())
+                    .collect::<Vec<_>>()
+                    .join(", ")
+            ),
+            Value::Dict(d) => write!(
+                f,
+                "#{{{}}}",
+                d.iter()
+                    .map(|(k, v)| format!(":{k} {v}"))
+                    .collect::<Vec<_>>()
+                    .join(", ")
+            ),
+            Value::Box(name, value) => {
+                write!(
+                    f,
+                    "box {}: [{}]",
+                    name,
+                    &value.try_borrow().unwrap().to_string()
+                )
+            }
+            Value::Placeholder => write!(f, "_"),
+            Value::Base(..) => unreachable!(),
+            Value::Recur(..) => unreachable!(),
+        }
+    }
+}
+
+impl Value<'_> {
+    pub fn bool(&self) -> bool {
+        !matches!(self, Value::Nil | Value::Boolean(false))
+    }
+}
+
+impl<'src> PartialEq for Value<'src> {
+    fn eq(&self, other: &Value<'src>) -> bool {
+        match (self, other) {
+            // value equality types
+            (Value::Nil, Value::Nil) => true,
+            (Value::Boolean(x), Value::Boolean(y)) => x == y,
+            (Value::Number(x), Value::Number(y)) => x == y,
+            (Value::InternedString(x), Value::InternedString(y)) => x == y,
+            (Value::AllocatedString(x), Value::AllocatedString(y)) => x == y,
+            (Value::InternedString(x), Value::AllocatedString(y)) => *x == **y,
+            (Value::AllocatedString(x), Value::InternedString(y)) => **x == *y,
+            (Value::Keyword(x), Value::Keyword(y)) => x == y,
+            (Value::Tuple(x), Value::Tuple(y)) => x == y,
+            (Value::List(x), Value::List(y)) => x == y,
+            (Value::Dict(x), Value::Dict(y)) => x == y,
+            // reference equality types
+            (Value::Fn(x), Value::Fn(y)) => {
+                Rc::<RefCell<Fn<'_>>>::as_ptr(x) == Rc::<RefCell<Fn<'_>>>::as_ptr(y)
+            }
+            (Value::Box(_, x), Value::Box(_, y)) => {
+                Rc::<RefCell<Value<'_>>>::as_ptr(x) == Rc::<RefCell<Value<'_>>>::as_ptr(y)
+            }
+            _ => false,
+        }
+    }
+}
+
+impl Eq for Value<'_> {}
+
+impl Value<'_> {
+    pub fn interpolate(&self) -> String {
+        match self {
+            Value::Nil => String::new(),
+            Value::Boolean(b) => format!("{b}"),
+            Value::Number(n) => format!("{n}"),
+            Value::Keyword(k) => format!(":{k}"),
+            Value::AllocatedString(s) => format!("{s}"),
+            Value::InternedString(s) => s.to_string(),
+            Value::Box(_, x) => x.borrow().interpolate(),
+            Value::Tuple(xs) => xs
+                .iter()
+                .map(|x| x.interpolate())
+                .collect::<Vec<_>>()
+                .join(", "),
+            Value::List(xs) => xs
+                .iter()
+                .map(|x| x.interpolate())
+                .collect::<Vec<_>>()
+                .join(", "),
+            Value::Dict(xs) => xs
+                .iter()
+                .map(|(k, v)| format!(":{} {}", k, v.interpolate()))
+                .collect::<Vec<_>>()
+                .join(", "),
+            Value::Fn(x) => format!("fn {}", x.borrow().name),
+            Value::FnDecl(name) => format!("fn {name}"),
+            Value::Placeholder => unreachable!(),
+            Value::Args(_) => unreachable!(),
+            Value::Recur(_) => unreachable!(),
+            Value::Base(_) => unreachable!(),
+        }
+    }
+}

src/old_vm.rs

@@ -0,0 +1,533 @@
+use crate::base::*;
+use crate::parser::*;
+use crate::value::*;
+use imbl::HashMap;
+use imbl::Vector;
+use std::cell::RefCell;
+use std::rc::Rc;
+
+#[derive(Clone, Debug)]
+pub struct LudusError {
+    pub msg: String,
+}
+
+// oy
+// lifetimes are a mess
+// I need 'src kind of everywhere
+// But (maybe) using 'src in eval
+// for ctx
+// means I can't borrow it mutably
+// I guess the question is how to get
+// the branches for Ast::Block and Ast::If
+// to work with a mutable borrow of ctx
+
+// pub struct Ctx<'src> {
+//     pub locals: Vec<(&'src str, Value<'src>)>,
+//     // pub names: Vec<&'src str>,
+//     // pub values: Vec<Value<'src>>,
+// }
+
+// impl<'src> Ctx<'src> {
+//     pub fn resolve(&self, name: &'src str) -> Value {
+//         if let Some((_, val)) = self.locals.iter().rev().find(|(bound, _)| *bound == name) {
+//             val.clone()
+//         } else {
+//             unreachable!()
+//         }
+//     }
+
+//     pub fn store(&mut self, name: &'src str, value: Value<'src>) {
+//         self.locals.push((name, value));
+//     }
+// }
+
+type Context<'src> = Vec<(String, Value<'src>)>;
+
+pub fn match_eq<T, U>(x: T, y: T, z: U) -> Option<U>
+where
+    T: PartialEq,
+{
+    if x == y {
+        Some(z)
+    } else {
+        None
+    }
+}
+
+pub fn match_pattern<'src, 'a>(
+    patt: &Pattern,
+    val: &Value<'src>,
+    ctx: &'a mut Context<'src>,
+) -> Option<&'a mut Context<'src>> {
+    match (patt, val) {
+        (Pattern::Nil, Value::Nil) => Some(ctx),
+        (Pattern::Placeholder, _) => Some(ctx),
+        (Pattern::Number(x), Value::Number(y)) => match_eq(x, y, ctx),
+        (Pattern::Boolean(x), Value::Boolean(y)) => match_eq(x, y, ctx),
+        (Pattern::Keyword(x), Value::Keyword(y)) => match_eq(x, y, ctx),
+        (Pattern::String(x), Value::InternedString(y)) => match_eq(x, y, ctx),
+        (Pattern::String(x), Value::AllocatedString(y)) => match_eq(&x.to_string(), y, ctx),
+        (Pattern::Interpolated(_, StringMatcher(matcher)), Value::InternedString(y)) => {
+            match matcher(y.to_string()) {
+                Some(matches) => {
+                    let mut matches = matches
+                        .iter()
+                        .map(|(word, string)| {
+                            (
+                                word.clone(),
+                                Value::AllocatedString(Rc::new(string.clone())),
+                            )
+                        })
+                        .collect::<Vec<_>>();
+                    ctx.append(&mut matches);
+                    Some(ctx)
+                }
+                None => None,
+            }
+        }
+        (Pattern::Word(w), val) => {
+            ctx.push((w.to_string(), val.clone()));
+            Some(ctx)
+        }
+        (Pattern::As(word, type_str), value) => {
+            let ludus_type = r#type(value);
+            let type_kw = Value::Keyword(type_str);
+            if type_kw == ludus_type {
+                ctx.push((word.to_string(), value.clone()));
+                Some(ctx)
+            } else {
+                None
+            }
+        }
+        // todo: add splats to these match clauses
+        (Pattern::Tuple(x), Value::Tuple(y)) => {
+            let has_splat = x
+                .iter()
+                .any(|patt| matches!(patt, (Pattern::Splattern(_), _)));
+            if x.len() > y.len() || (!has_splat && x.len() != y.len()) {
+                return None;
+            };
+            let to = ctx.len();
+            for i in 0..x.len() {
+                if let Pattern::Splattern(patt) = &x[i].0 {
+                    let mut list = Vector::new();
+                    for i in i..y.len() {
+                        list.push_back(y[i].clone())
+                    }
+                    let list = Value::List(list);
+                    match_pattern(&patt.0, &list, ctx);
+                } else if match_pattern(&x[i].0, &y[i], ctx).is_none() {
+                    while ctx.len() > to {
+                        ctx.pop();
+                    }
+                    return None;
+                }
+            }
+            Some(ctx)
+        }
+        (Pattern::List(x), Value::List(y)) => {
+            let has_splat = x
+                .iter()
+                .any(|patt| matches!(patt, (Pattern::Splattern(_), _)));
+            if x.len() > y.len() || (!has_splat && x.len() != y.len()) {
+                return None;
+            };
+            let to = ctx.len();
+            for (i, (patt, _)) in x.iter().enumerate() {
+                if let Pattern::Splattern(patt) = &patt {
+                    let list = Value::List(y.skip(i));
+                    match_pattern(&patt.0, &list, ctx);
+                } else if match_pattern(patt, y.get(i).unwrap(), ctx).is_none() {
+                    while ctx.len() > to {
+                        ctx.pop();
+                    }
+                    return None;
+                }
+            }
+            Some(ctx)
+        }
+        // TODO: optimize this on several levels
+        // - [ ] opportunistic mutation
+        // - [ ] get rid of all the pointer indirection in word splats
+        (Pattern::Dict(x), Value::Dict(y)) => {
+            let has_splat = x
+                .iter()
+                .any(|patt| matches!(patt, (Pattern::Splattern(_), _)));
+            if x.len() > y.len() || (!has_splat && x.len() != y.len()) {
+                return None;
+            };
+            let to = ctx.len();
+            let mut matched = vec![];
+            for (pattern, _) in x {
+                match pattern {
+                    Pattern::Pair(key, patt) => {
+                        if let Some(val) = y.get(key) {
+                            if match_pattern(&patt.0, val, ctx).is_none() {
+                                while ctx.len() > to {
+                                    ctx.pop();
+                                }
+                                return None;
+                            } else {
+                                matched.push(key);
+                            }
+                        } else {
+                            return None;
+                        };
+                    }
+                    Pattern::Splattern(pattern) => match pattern.0 {
+                        Pattern::Word(w) => {
+                            // TODO: find a way to take ownership
+                            // this will ALWAYS make structural changes, because of this clone
+                            // we want opportunistic mutation if possible
+                            let mut unmatched = y.clone();
+                            for key in matched.iter() {
+                                unmatched.remove(*key);
+                            }
+                            ctx.push((w.to_string(), Value::Dict(unmatched)));
+                        }
+                        Pattern::Placeholder => (),
+                        _ => unreachable!(),
+                    },
+                    _ => unreachable!(),
+                }
+            }
+            Some(ctx)
+        }
+        _ => None,
+    }
+}
+
+pub fn match_clauses<'src>(
+    value: &Value<'src>,
+    clauses: &'src [MatchClause],
+    ctx: &mut Context<'src>,
+) -> Result<Value<'src>, LudusError> {
+    let to = ctx.len();
+    for MatchClause { patt, body, guard } in clauses.iter() {
+        if let Some(ctx) = match_pattern(&patt.0, value, ctx) {
+            let pass_guard = match guard {
+                None => true,
+                Some((ast, _)) => {
+                    let guard_res = eval(ast, ctx);
+                    match &guard_res {
+                        Err(_) => return guard_res,
+                        Ok(val) => val.bool(),
+                    }
+                }
+            };
+            if !pass_guard {
+                while ctx.len() > to {
+                    ctx.pop();
+                }
+                continue;
+            }
+            let res = eval(&body.0, ctx);
+            while ctx.len() > to {
+                ctx.pop();
+            }
+            return res;
+        }
+    }
+    Err(LudusError {
+        msg: "no match".to_string(),
+    })
+}
+
+pub fn apply<'src>(
+    callee: Value<'src>,
+    caller: Value<'src>,
+    ctx: &mut Context,
+) -> Result<Value<'src>, LudusError> {
+    match (callee, caller) {
+        (Value::Keyword(kw), Value::Dict(dict)) => {
+            if let Some(val) = dict.get(kw) {
+                Ok(val.clone())
+            } else {
+                Ok(Value::Nil)
+            }
+        }
+        (Value::Dict(dict), Value::Keyword(kw)) => {
+            if let Some(val) = dict.get(kw) {
+                Ok(val.clone())
+            } else {
+                Ok(Value::Nil)
+            }
+        }
+        (Value::Fn(f), Value::Tuple(args)) => {
+            let args = Value::Tuple(args);
+            match_clauses(&args, f.body, ctx)
+        }
+        (Value::Fn(_f), Value::Args(_args)) => todo!(),
+        (_, Value::Keyword(_)) => Ok(Value::Nil),
+        (_, Value::Args(_)) => Err(LudusError {
+            msg: "you may only call a function".to_string(),
+        }),
+        (Value::Base(f), Value::Tuple(args)) => match f {
+            Base::Nullary(f) => {
+                if args.len() != 0 {
+                    Err(LudusError {
+                        msg: "wrong arity: expected 0 arguments".to_string(),
+                    })
+                } else {
+                    Ok(f())
+                }
+            }
+            Base::Unary(f) => {
+                if args.len() != 1 {
+                    Err(LudusError {
+                        msg: "wrong arity: expected 1 argument".to_string(),
+                    })
+                } else {
+                    Ok(f(&args[0]))
+                }
+            }
+            Base::Binary(r#fn) => {
+                if args.len() != 2 {
+                    Err(LudusError {
+                        msg: "wrong arity: expected 2 arguments".to_string(),
+                    })
+                } else {
+                    Ok(r#fn(&args[0], &args[1]))
+                }
+            }
+            Base::Ternary(f) => {
+                if args.len() != 3 {
+                    Err(LudusError {
+                        msg: "wrong arity: expected 3 arguments".to_string(),
+                    })
+                } else {
+                    Ok(f(&args[0], &args[1], &args[2]))
+                }
+            }
+        },
+        _ => unreachable!(),
+    }
+}
+
+pub fn eval<'src, 'a>(
+    ast: &'src Ast,
+    ctx: &'a mut Vec<(String, Value<'src>)>,
+) -> Result<Value<'src>, LudusError> {
+    match ast {
+        Ast::Nil => Ok(Value::Nil),
+        Ast::Boolean(b) => Ok(Value::Boolean(*b)),
+        Ast::Number(n) => Ok(Value::Number(*n)),
+        Ast::Keyword(k) => Ok(Value::Keyword(k)),
+        Ast::String(s) => Ok(Value::InternedString(s)),
+        Ast::Interpolated(parts) => {
+            let mut interpolated = String::new();
+            for part in parts {
+                match &part.0 {
+                    StringPart::Data(s) => interpolated.push_str(s.as_str()),
+                    StringPart::Word(w) => {
+                        let val = if let Some((_, value)) =
+                            ctx.iter().rev().find(|(name, _)| w == name)
+                        {
+                            value.clone()
+                        } else {
+                            return Err(LudusError {
+                                msg: format!("unbound name {w}"),
+                            });
+                        };
+                        interpolated.push_str(val.interpolate().as_str())
+                    }
+                    StringPart::Inline(_) => unreachable!(),
+                }
+            }
+            Ok(Value::AllocatedString(Rc::new(interpolated)))
+        }
+        Ast::Block(exprs) => {
+            let to = ctx.len();
+            let mut result = Value::Nil;
+            for (expr, _) in exprs {
+                result = eval(expr, ctx)?;
+            }
+            while ctx.len() > to {
+                ctx.pop();
+            }
+            Ok(result)
+        }
+        Ast::If(cond, if_true, if_false) => {
+            let truthy = eval(&cond.0, ctx)?.bool();
+            if truthy {
+                eval(&if_true.0, ctx)
+            } else {
+                eval(&if_false.0, ctx)
+            }
+        }
+        Ast::List(members) => {
+            let mut vect = Vector::new();
+            for member in members {
+                if let Ast::Splat(_) = member.0 {
+                    let to_splat = eval(&member.0, ctx)?;
+                    match to_splat {
+                        Value::List(list) => vect.append(list),
+                        _ => {
+                            return Err(LudusError {
+                                msg: "only lists may be splatted into lists".to_string(),
+                            })
+                        }
+                    }
+                } else {
+                    vect.push_back(eval(&member.0, ctx)?)
+                }
+            }
+            Ok(Value::List(vect))
+        }
+        Ast::Tuple(members) => {
+            let mut vect = Vec::new();
+            for member in members {
+                vect.push(eval(&member.0, ctx)?);
+            }
+            Ok(Value::Tuple(Rc::new(vect)))
+        }
+        Ast::Word(w) | Ast::Splat(w) => {
+            let val = if let Some((_, value)) = ctx.iter().rev().find(|(name, _)| w == name) {
+                value.clone()
+            } else {
+                return Err(LudusError {
+                    msg: format!("unbound name {w}"),
+                });
+            };
+            Ok(val)
+        }
+        Ast::Let(patt, expr) => {
+            let val = eval(&expr.0, ctx)?;
+            match match_pattern(&patt.0, &val, ctx) {
+                Some(_) => Ok(val),
+                None => Err(LudusError {
+                    msg: "No match".to_string(),
+                }),
+            }
+        }
+        Ast::Placeholder => Ok(Value::Placeholder),
+        Ast::Error => unreachable!(),
+        Ast::Arguments(a) => {
+            let mut args = vec![];
+            for (arg, _) in a.iter() {
+                let arg = eval(arg, ctx)?;
+                args.push(arg);
+            }
+            if args.iter().any(|arg| matches!(arg, Value::Placeholder)) {
+                Ok(Value::Args(Rc::new(args)))
+            } else {
+                Ok(Value::Tuple(Rc::new(args)))
+            }
+        }
+        Ast::Dict(terms) => {
+            let mut dict = HashMap::new();
+            for term in terms {
+                let (term, _) = term;
+                match term {
+                    Ast::Pair(key, value) => {
+                        let value = eval(&value.0, ctx)?;
+                        dict.insert(*key, value);
+                    }
+                    Ast::Splat(_) => {
+                        let resolved = eval(term, ctx)?;
+                        let Value::Dict(to_splat) = resolved else {
+                            return Err(LudusError {
+                                msg: "cannot splat non-dict into dict".to_string(),
+                            });
+                        };
+                        dict = to_splat.union(dict);
+                    }
+                    _ => unreachable!(),
+                }
+            }
+            Ok(Value::Dict(dict))
+        }
+        Ast::Box(name, expr) => {
+            let val = eval(&expr.0, ctx)?;
+            let boxed = Value::Box(name, Rc::new(RefCell::new(val)));
+            ctx.push((name.to_string(), boxed.clone()));
+            Ok(boxed)
+        }
+        Ast::Synthetic(root, first, rest) => {
+            let root = eval(&root.0, ctx)?;
+            let first = eval(&first.0, ctx)?;
+            let mut curr = apply(root, first, ctx)?;
+            for term in rest.iter() {
+                let next = eval(&term.0, ctx)?;
+                curr = apply(curr, next, ctx)?;
+            }
+            Ok(curr)
+        }
+        Ast::When(clauses) => {
+            for clause in clauses.iter() {
+                let WhenClause { cond, body } = &clause.0;
+                if eval(&cond.0, ctx)?.bool() {
+                    return eval(&body.0, ctx);
+                };
+            }
+            Err(LudusError {
+                msg: "no match".to_string(),
+            })
+        }
+        Ast::Match(value, clauses) => {
+            let value = eval(&value.0, ctx)?;
+            match_clauses(&value, clauses, ctx)
+        }
+        Ast::Fn(name, clauses, doc) => {
+            let doc = doc.map(|s| s.to_string());
+            let the_fn = Value::Fn::<'src>(Rc::new(Fn::<'src> {
+                name: name.to_string(),
+                body: clauses,
+                doc,
+            }));
+            ctx.push((name.to_string(), the_fn.clone()));
+            Ok(the_fn)
+        }
+        Ast::FnDeclaration(_name) => todo!(),
+        Ast::Panic(msg) => {
+            let msg = eval(&msg.0, ctx)?;
+            Err(LudusError {
+                msg: msg.to_string(),
+            })
+        }
+        Ast::Repeat(times, body) => {
+            let times_num = match eval(&times.0, ctx) {
+                Ok(Value::Number(n)) => n as usize,
+                _ => {
+                    return Err(LudusError {
+                        msg: "repeat may only take numbers".to_string(),
+                    })
+                }
+            };
+            for _ in 0..times_num {
+                eval(&body.0, ctx)?;
+            }
+            Ok(Value::Nil)
+        }
+        Ast::Do(terms) => {
+            let mut result = eval(&terms[0].0, ctx)?;
+            for (term, _) in terms.iter().skip(1) {
+                let next = eval(term, ctx)?;
+                let arg = Value::Tuple(Rc::new(vec![result]));
+                result = apply(next, arg, ctx)?;
+            }
+            Ok(result)
+        }
+        Ast::Pair(..) => {
+            unreachable!()
+        }
+        Ast::Loop(init, clauses) => {
+            let mut args = eval(&init.0, ctx)?;
+            loop {
+                let result = match_clauses(&args, clauses, ctx)?;
+                if let Value::Recur(recur_args) = result {
+                    args = Value::Tuple(Rc::new(recur_args));
+                } else {
+                    return Ok(result);
+                }
+            }
+        }
+        Ast::Recur(args) => {
+            let mut vect = Vec::new();
+            for arg in args {
+                vect.push(eval(&arg.0, ctx)?);
+            }
+            Ok(Value::Recur(vect))
+        }
+    }
+}

src/parser.rs

@@ -51,12 +51,21 @@ impl fmt::Display for StringPart {
     }
 }
 
+pub struct LFn {
+    name: &'static str,
+    clauses: Vec<Spanned<Ast>>,
+    doc: Option<&'static str>,
+}
+
 #[derive(Clone, Debug, PartialEq, Dissectible)]
 pub enum Ast {
     // a special Error node
     // may come in handy?
     Error,
 
+    And,
+    Or,
+
     // expression nodes
     Placeholder,
     Nil,
@@ -83,7 +92,8 @@ pub enum Ast {
         Box<Option<Spanned<Self>>>,
         Box<Spanned<Self>>,
     ),
-    Fn(&'static str, Vec<Spanned<Self>>, Option<&'static str>),
+    Fn(&'static str, Box<Spanned<Ast>>, Option<&'static str>),
+    FnBody(Vec<Spanned<Ast>>),
     FnDeclaration(&'static str),
     Panic(Box<Spanned<Self>>),
     Do(Vec<Spanned<Self>>),
@@ -114,6 +124,8 @@ impl fmt::Display for Ast {
     fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
         use Ast::*;
         match self {
+            And => write!(f, "And"),
+            Or => write!(f, "Or"),
             Error => write!(f, "Error"),
             Nil => write!(f, "nil"),
             String(s) => write!(f, "String: \"{}\"", s),
@@ -191,7 +203,7 @@ impl fmt::Display for Ast {
                     .collect::<Vec<_>>()
                     .join("\n")
             ),
-            Placeholder => todo!(),
+            Placeholder => write!(f, "Placeholder"),
             LBox(_name, _rhs) => todo!(),
             Match(value, clauses) => {
                 write!(
@@ -205,11 +217,10 @@ impl fmt::Display for Ast {
                         .join("\n")
                 )
             }
-            Fn(name, clauses, _) => {
+            FnBody(clauses) => {
                 write!(
                     f,
-                    "fn: {}\n{}",
-                    name,
+                    "{}",
                     clauses
                         .iter()
                         .map(|clause| clause.0.to_string())
@@ -217,6 +228,9 @@ impl fmt::Display for Ast {
                         .join("\n")
                 )
             }
+            Fn(name, body, ..) => {
+                write!(f, "fn: {name}\n{}", body.0)
+            }
             FnDeclaration(_name) => todo!(),
             Panic(_expr) => todo!(),
             Do(terms) => {
@@ -420,6 +434,7 @@ fn is_word_char(c: char) -> bool {
 }
 
 fn parse_string(s: &'static str, span: SimpleSpan) -> Result<Vec<Spanned<StringPart>>, String> {
+    println!("parsing string pattern: {s}");
     let mut parts = vec![];
     let mut current_part = String::new();
     let mut start = span.start;
@@ -488,6 +503,12 @@ fn parse_string(s: &'static str, span: SimpleSpan) -> Result<Vec<Spanned<StringP
             StringPart::Inline(current_part),
             SimpleSpan::new(start, span.end),
         ))
+    } else if !current_part.is_empty() {
+        let part_len = current_part.len();
+        parts.push((
+            StringPart::Data(current_part),
+            SimpleSpan::new(start, part_len),
+        ))
     }
 
     Ok(parts)
@@ -718,7 +739,11 @@ where
         .delimited_by(just(Token::Punctuation("(")), just(Token::Punctuation(")")))
         .map_with(|args, e| (Arguments(args), e.span()));
 
-    let synth_root = word.or(keyword);
+    let or = just(Token::Reserved("or")).map_with(|_, e| (Or, e.span()));
+
+    let and = just(Token::Reserved("and")).map_with(|_, e| (And, e.span()));
+
+    let synth_root = or.or(and).or(word).or(keyword);
 
     let synth_term = keyword.or(args);
 
@@ -922,7 +947,12 @@ where
 
     let lambda = just(Token::Reserved("fn"))
         .ignore_then(fn_unguarded.clone())
-        .map_with(|clause, e| (Fn("anonymous", vec![clause], None), e.span()));
+        .map_with(|clause, e| {
+            (
+                Fn("", Box::new((Ast::FnBody(vec![clause]), e.span())), None),
+                e.span(),
+            )
+        });
 
     let fn_clauses = fn_clause
         .clone()
@@ -1010,7 +1040,10 @@ where
             } else {
                 unreachable!()
             };
-            (Fn(name, vec![clause], None), e.span())
+            (
+                Fn(name, Box::new((Ast::FnBody(vec![clause]), e.span())), None),
+                e.span(),
+            )
         });
 
     let docstr = select! {Token::String(s) => s};
@@ -1032,7 +1065,10 @@ where
             } else {
                 unreachable!()
             };
-            (Fn(name, clauses, docstr), e.span())
+            (
+                Fn(name, Box::new((Ast::FnBody(clauses), e.span())), docstr),
+                e.span(),
+            )
         });
 
     let fn_ = fn_named.or(fn_compound).or(fn_decl);

src/validator.rs

@@ -1,5 +1,9 @@
+// TODO:
+// * [ ] ensure `or` and `and` never get passed by reference
+// * [ ] ensure no placeholder in `or` and `and`
+
 use crate::parser::*;
-use crate::spans::Span;
+use crate::spans::{Span, Spanned};
 use crate::value::Value;
 use std::collections::{HashMap, HashSet};
 
@@ -53,9 +57,9 @@ fn match_arities(arities: &HashSet<Arity>, num_args: u8) -> bool {
 }
 
 #[derive(Debug, PartialEq)]
-pub struct Validator<'a, 'src> {
+pub struct Validator<'a> {
     pub locals: Vec<(String, Span, FnInfo)>,
-    pub prelude: &'a Vec<(String, Value<'src>)>,
+    pub prelude: &'a Vec<(&'static str, Value)>,
     pub input: &'static str,
     pub src: &'static str,
     pub ast: &'a Ast,
@@ -65,14 +69,14 @@ pub struct Validator<'a, 'src> {
     status: VStatus,
 }
 
-impl<'a, 'src: 'a> Validator<'a, 'src> {
+impl<'a> Validator<'a> {
     pub fn new(
         ast: &'a Ast,
         span: Span,
         input: &'static str,
         src: &'static str,
-        prelude: &'a Vec<(String, Value<'src>)>,
-    ) -> Validator<'a, 'src> {
+        prelude: &'a Vec<(&'static str, Value)>,
+    ) -> Validator<'a> {
         Validator {
             input,
             src,
@@ -109,7 +113,7 @@ impl<'a, 'src: 'a> Validator<'a, 'src> {
 
     fn resolved(&self, name: &str) -> bool {
         self.locals.iter().any(|(bound, ..)| name == bound.as_str())
-            || self.prelude.iter().any(|(bound, _)| name == bound.as_str())
+            || self.prelude.iter().any(|(bound, _)| name == *bound)
     }
 
     fn bound(&self, name: &str) -> Option<&(String, Span, FnInfo)> {
@@ -143,6 +147,13 @@ impl<'a, 'src: 'a> Validator<'a, 'src> {
         }
     }
 
+    fn visit(&mut self, node: &'a Spanned<Ast>) {
+        let (expr, span) = node;
+        self.ast = expr;
+        self.span = *span;
+        self.validate();
+    }
+
     pub fn validate(&mut self) {
         use Ast::*;
         let root = self.ast;
@@ -179,18 +190,13 @@ impl<'a, 'src: 'a> Validator<'a, 'src> {
                 }
                 let to = self.locals.len();
                 let tailpos = self.status.tail_position;
-                for (expr, span) in block.iter().take(block.len() - 1) {
+                for line in block.iter().take(block.len() - 1) {
                     self.status.tail_position = false;
-                    self.ast = expr;
-                    self.span = *span;
-                    self.validate();
+                    self.visit(line);
                 }
 
-                let (expr, span) = block.last().unwrap();
-                self.ast = expr;
-                self.span = *span;
                 self.status.tail_position = tailpos;
-                self.validate();
+                self.visit(block.last().unwrap());
 
                 let block_bindings = self.locals.split_off(to);
 
@@ -207,22 +213,12 @@ impl<'a, 'src: 'a> Validator<'a, 'src> {
                 let tailpos = self.status.tail_position;
                 self.status.tail_position = false;
 
-                let (expr, span) = cond.as_ref();
-                self.ast = expr;
-                self.span = *span;
-                self.validate();
+                self.visit(cond.as_ref());
 
                 // pass through tailpos only to then/else
                 self.status.tail_position = tailpos;
-                let (expr, span) = then.as_ref();
-                self.ast = expr;
-                self.span = *span;
-                self.validate();
-
-                let (expr, span) = r#else.as_ref();
-                self.ast = expr;
-                self.span = *span;
-                self.validate();
+                self.visit(then.as_ref());
+                self.visit(r#else.as_ref());
             }
             Tuple(members) => {
                 if members.is_empty() {
@@ -230,10 +226,8 @@ impl<'a, 'src: 'a> Validator<'a, 'src> {
                 }
                 let tailpos = self.status.tail_position;
                 self.status.tail_position = false;
-                for (expr, span) in members {
-                    self.ast = expr;
-                    self.span = *span;
-                    self.validate();
+                for member in members {
+                    self.visit(member);
                 }
                 self.status.tail_position = tailpos;
             }
@@ -244,10 +238,8 @@ impl<'a, 'src: 'a> Validator<'a, 'src> {
                 }
                 let tailpos = self.status.tail_position;
                 self.status.tail_position = false;
-                for (expr, span) in args {
-                    self.ast = expr;
-                    self.span = *span;
-                    self.validate();
+                for arg in args {
+                    self.visit(arg);
                 }
                 self.status.has_placeholder = false;
                 self.status.tail_position = tailpos;
@@ -267,30 +259,21 @@ impl<'a, 'src: 'a> Validator<'a, 'src> {
                 }
                 let tailpos = self.status.tail_position;
                 self.status.tail_position = false;
-                for (expr, span) in list {
-                    self.ast = expr;
-                    self.span = *span;
-                    self.validate();
+                for member in list {
+                    self.visit(member);
                 }
 
                 self.status.tail_position = tailpos;
             }
-            Pair(_, value) => {
-                let (expr, span) = value.as_ref();
-                self.ast = expr;
-                self.span = *span;
-                self.validate();
-            }
+            Pair(_, value) => self.visit(value.as_ref()),
             Dict(dict) => {
                 if dict.is_empty() {
                     return;
                 }
                 let tailpos = self.status.tail_position;
                 self.status.tail_position = false;
-                for (expr, span) in dict {
-                    self.ast = expr;
-                    self.span = *span;
-                    self.validate();
+                for pair in dict {
+                    self.visit(pair)
                 }
                 self.status.tail_position = tailpos;
             }
@@ -299,31 +282,16 @@ impl<'a, 'src: 'a> Validator<'a, 'src> {
             // check arity against fn info if first term is word and second term is args
             Synthetic(first, second, rest) => {
                 match (&first.0, &second.0) {
-                    (Ast::Word(_), Ast::Keyword(_)) => {
-                        let (expr, span) = first.as_ref();
-                        self.ast = expr;
-                        self.span = *span;
-                        self.validate();
-                    }
+                    (Ast::Word(_), Ast::Keyword(_)) => self.visit(first.as_ref()),
                     (Ast::Keyword(_), Ast::Arguments(args)) => {
                         if args.len() != 1 {
                             self.err("called keywords may only take one argument".to_string())
                         }
-                        let (expr, span) = second.as_ref();
-                        self.ast = expr;
-                        self.span = *span;
-                        self.validate();
+                        self.visit(second.as_ref());
                     }
                     (Ast::Word(name), Ast::Arguments(args)) => {
-                        let (expr, span) = first.as_ref();
-                        self.ast = expr;
-                        self.span = *span;
-                        self.validate();
-
-                        let (expr, span) = second.as_ref();
-                        self.ast = expr;
-                        self.span = *span;
-                        self.validate();
+                        self.visit(first.as_ref());
+                        self.visit(second.as_ref());
 
                         //TODO: check arities of prelude fns, too
                         let fn_binding = self.bound(name);
@@ -337,32 +305,20 @@ impl<'a, 'src: 'a> Validator<'a, 'src> {
                     _ => unreachable!(),
                 }
                 for term in rest {
-                    let (expr, span) = term;
-                    self.ast = expr;
-                    self.span = *span;
-                    self.validate();
+                    self.visit(term);
                 }
             }
             WhenClause(cond, body) => {
                 let tailpos = self.status.tail_position;
                 self.status.tail_position = false;
-                let (expr, span) = cond.as_ref();
-                self.ast = expr;
-                self.span = *span;
-                self.validate();
-
+                self.visit(cond.as_ref());
+                //pass through tail position for when bodies
                 self.status.tail_position = tailpos;
-                let (expr, span) = body.as_ref();
-                self.ast = expr;
-                self.span = *span;
-                self.validate();
+                self.visit(body.as_ref());
             }
             When(clauses) => {
                 for clause in clauses {
-                    let (expr, span) = clause;
-                    self.ast = expr;
-                    self.span = *span;
-                    self.validate();
+                    self.visit(clause);
                 }
             }
 
@@ -374,54 +330,30 @@ impl<'a, 'src: 'a> Validator<'a, 'src> {
                 } else {
                     self.bind(name.to_string());
                 }
-                let (expr, span) = boxed.as_ref();
-                self.ast = expr;
-                self.span = *span;
-                self.validate();
+                self.visit(boxed.as_ref());
             }
             Let(lhs, rhs) => {
-                let (expr, span) = rhs.as_ref();
-                self.ast = expr;
-                self.span = *span;
-                self.validate();
-
-                let (expr, span) = lhs.as_ref();
-                self.ast = expr;
-                self.span = *span;
-                self.validate();
+                self.visit(rhs.as_ref());
+                self.visit(lhs.as_ref());
             }
             MatchClause(pattern, guard, body) => {
                 let to = self.locals.len();
 
-                let (patt, span) = pattern.as_ref();
-                self.ast = patt;
-                self.span = *span;
-                self.validate();
+                self.visit(pattern.as_ref());
 
-                if let Some((expr, span)) = guard.as_ref() {
-                    self.ast = expr;
-                    self.span = *span;
-                    self.validate();
+                if let Some(guard) = guard.as_ref() {
+                    self.visit(guard);
                 }
 
-                let (expr, span) = body.as_ref();
-                self.ast = expr;
-                self.span = *span;
-                self.validate();
+                self.visit(body.as_ref());
 
                 self.locals.truncate(to);
             }
             Match(scrutinee, clauses) => {
-                let (expr, span) = scrutinee.as_ref();
-                self.ast = expr;
-                self.span = *span;
-                self.validate();
+                self.visit(scrutinee.as_ref());
 
                 for clause in clauses {
-                    let (expr, span) = clause;
-                    self.ast = expr;
-                    self.span = *span;
-                    self.validate();
+                    self.visit(clause);
                 }
             }
             FnDeclaration(name) => {
@@ -434,7 +366,8 @@ impl<'a, 'src: 'a> Validator<'a, 'src> {
                 self.declare_fn(name.to_string());
                 self.status.tail_position = tailpos;
             }
-            Fn(name, clauses, ..) => {
+            FnBody(..) => unreachable!(),
+            Fn(name, body, ..) => {
                 let mut is_declared = false;
                 match self.bound(name) {
                     Some((_, _, FnInfo::Declared)) => is_declared = true,
@@ -452,8 +385,12 @@ impl<'a, 'src: 'a> Validator<'a, 'src> {
                 let from = self.status.used_bindings.len();
                 let mut arities = HashSet::new();
 
+                let (Ast::FnBody(clauses), _) = body.as_ref() else {
+                    unreachable!()
+                };
+
                 for clause in clauses {
-                    // TODO: validate all parts of clauses
+                    // we have to do this explicitly here because of arity checking
                     let (expr, span) = clause;
                     self.ast = expr;
                     self.span = *span;
@@ -462,12 +399,7 @@ impl<'a, 'src: 'a> Validator<'a, 'src> {
                     self.validate();
                 }
 
-                // this should be right
-                // we can't bind anything that's already bound,
-                // even in arg names
-                // so anything that is already bound and used
-                // will, of necessity, be closed over
-                // we don't want to try to close over locals in functions
+                // collect info about what the function closes over
                 let mut closed_over = HashSet::new();
                 for binding in self.status.used_bindings.iter().skip(from) {
                     if self.bound(binding.as_str()).is_some() {
@@ -488,10 +420,7 @@ impl<'a, 'src: 'a> Validator<'a, 'src> {
             Panic(msg) => {
                 let tailpos = self.status.tail_position;
                 self.status.tail_position = false;
-                let (expr, span) = msg.as_ref();
-                self.ast = expr;
-                self.span = *span;
-                self.validate();
+                self.visit(msg.as_ref());
                 self.status.tail_position = tailpos;
             }
             // TODO: fix the tail call here?
@@ -500,39 +429,23 @@ impl<'a, 'src: 'a> Validator<'a, 'src> {
                     return self.err("do expressions must have at least two terms".to_string());
                 }
                 for term in terms.iter().take(terms.len() - 1) {
-                    let (expr, span) = term;
-                    self.ast = expr;
-                    self.span = *span;
-                    self.validate();
+                    self.visit(term);
                 }
-
-                let (expr, span) = terms.last().unwrap();
-                self.ast = expr;
-                self.span = *span;
-                if matches!(expr, Ast::Recur(_)) {
+                let last = terms.last().unwrap();
+                self.visit(last);
+                if matches!(last.0, Ast::Recur(_)) {
                     self.err("`recur` may not be used in `do` forms".to_string());
                 }
-                self.validate();
             }
             Repeat(times, body) => {
                 self.status.tail_position = false;
-                let (expr, span) = times.as_ref();
-                self.ast = expr;
-                self.span = *span;
-                self.validate();
-
-                let (expr, span) = body.as_ref();
-                self.ast = expr;
-                self.span = *span;
-                self.validate();
+                self.visit(times.as_ref());
+                self.visit(body.as_ref());
             }
             Loop(with, body) => {
-                let (expr, span) = with.as_ref();
-                self.span = *span;
-                self.ast = expr;
-                self.validate();
+                self.visit(with.as_ref());
 
-                let Ast::Tuple(input) = expr else {
+                let Ast::Tuple(input) = &with.0 else {
                     unreachable!()
                 };
 
@@ -588,10 +501,7 @@ impl<'a, 'src: 'a> Validator<'a, 'src> {
 
                 self.status.tail_position = false;
                 for arg in args {
-                    let (expr, span) = arg;
-                    self.ast = expr;
-                    self.span = *span;
-                    self.validate();
+                    self.visit(arg);
                 }
             }
             WordPattern(name) => match self.bound(name) {
@@ -654,27 +564,17 @@ impl<'a, 'src: 'a> Validator<'a, 'src> {
                     return;
                 }
                 for term in terms.iter().take(terms.len() - 1) {
-                    let (patt, span) = term;
-                    self.ast = patt;
-                    self.span = *span;
-                    self.validate();
+                    self.visit(term);
                 }
 
                 self.status.last_term = true;
-                let (patt, span) = terms.last().unwrap();
-                self.ast = patt;
-                self.span = *span;
-                self.validate();
+                let last = terms.last().unwrap();
+                self.visit(last);
                 self.status.last_term = false;
             }
-            PairPattern(_, patt) => {
-                let (patt, span) = patt.as_ref();
-                self.ast = patt;
-                self.span = *span;
-                self.validate();
-            }
+            PairPattern(_, patt) => self.visit(patt.as_ref()),
             // terminals can never be invalid
-            Nil | Boolean(_) | Number(_) | Keyword(_) | String(_) => (),
+            Nil | Boolean(_) | Number(_) | Keyword(_) | String(_) | And | Or => (),
             // terminal patterns can never be invalid
             NilPattern | BooleanPattern(..) | NumberPattern(..) | StringPattern(..)
             | KeywordPattern(..) | PlaceholderPattern => (),

src/value.rs

@@ -1,193 +1,271 @@
-use crate::base::*;
-use crate::parser::*;
-use crate::spans::*;
-use imbl::*;
+use crate::base::BaseFn;
+use crate::compiler::Chunk;
+use crate::parser::Ast;
+use crate::spans::Spanned;
+use imbl::{HashMap, Vector};
 use std::cell::RefCell;
-use std::fmt;
 use std::rc::Rc;
-use struct_scalpel::Dissectible;
 
 #[derive(Clone, Debug)]
-pub struct Fn<'src> {
-    pub name: String,
-    pub body: &'src Vec<Spanned<Ast>>,
-    pub doc: Option<String>,
-    pub enclosing: Vec<(String, Value<'src>)>,
-    pub has_run: bool,
-    pub input: &'static str,
-    pub src: &'static str,
+pub enum LFn {
+    Declared {
+        name: &'static str,
+    },
+    Defined {
+        name: &'static str,
+        doc: Option<&'static str>,
+        chunks: Vec<(u8, Chunk)>,
+        closed: RefCell<Vec<Value>>,
+    },
 }
 
-#[derive(Debug, Dissectible)]
-pub enum Value<'src> {
+impl LFn {
+    pub fn close(&self, value: Value) {
+        match self {
+            LFn::Declared { .. } => unreachable!(),
+            LFn::Defined { closed, .. } => {
+                closed.borrow_mut().push(value);
+            }
+        }
+    }
+
+    pub fn doc(&self) -> Value {
+        match self {
+            LFn::Declared { name } => {
+                Value::String(Rc::new(format!("fn {name}: undefined function")))
+            }
+            LFn::Defined {
+                name,
+                doc: Some(doc),
+                ..
+            } => Value::String(Rc::new(format!("fn {name}\n{doc}"))),
+            LFn::Defined { name, .. } => {
+                Value::String(Rc::new(format!("fn {name}: no documentation")))
+            }
+        }
+    }
+
+    pub fn name(&self) -> &'static str {
+        match self {
+            LFn::Declared { name } | LFn::Defined { name, .. } => name,
+        }
+    }
+
+    pub fn chunk(&self, arity: u8) -> &Chunk {
+        // println!("Getting chunk of {arity} from {:?}", self);
+        match self {
+            LFn::Declared { .. } => unreachable!(),
+            LFn::Defined { chunks, .. } => &chunks.iter().find(|(a, _)| *a == arity).unwrap().1,
+        }
+    }
+
+    pub fn upvalue(&self, idx: u8) -> Value {
+        match self {
+            LFn::Declared { .. } => unreachable!(),
+            LFn::Defined { closed, .. } => closed.borrow()[idx as usize].clone(),
+        }
+    }
+}
+
+#[derive(Clone, Debug, PartialEq)]
+pub struct Partial {
+    pub args: Vec<Value>,
+    pub name: &'static str,
+    pub function: Value,
+}
+
+#[derive(Clone, Debug)]
+pub enum Value {
+    Nothing,
     Nil,
-    Placeholder,
-    Boolean(bool),
-    Number(f64),
+    True,
+    False,
     Keyword(&'static str),
-    InternedString(&'static str),
-    AllocatedString(Rc<String>),
-    // on the heap for now
-    Tuple(Rc<Vec<Self>>),
-    Args(Rc<Vec<Self>>),
-    List(Vector<Self>),
-    Dict(HashMap<&'static str, Self>),
-    Box(&'static str, Rc<RefCell<Self>>),
-    Fn(Rc<RefCell<Fn<'src>>>),
-    FnDecl(&'static str),
-    Base(BaseFn<'src>),
-    Recur(Vec<Self>),
-    // Set(HashSet<Self>),
-    // Sets are hard
-    // Sets require Eq
-    // Eq is not implemented on f64, because NaNs
-    // We could use ordered_float::NotNan
-    // Let's defer that
-    // We're not really using sets in Ludus
-
-    // Other things we're not implementing yet:
-    // pkgs, nses, tests
+    Interned(&'static str),
+    String(Rc<String>),
+    Number(f64),
+    Tuple(Rc<Vec<Value>>),
+    List(Box<Vector<Value>>),
+    Dict(Box<HashMap<&'static str, Value>>),
+    Box(Rc<RefCell<Value>>),
+    Fn(Rc<LFn>),
+    BaseFn(BaseFn),
+    Partial(Rc<Partial>),
 }
 
-impl<'src> Clone for Value<'src> {
-    fn clone(&self) -> Value<'src> {
-        match self {
-            Value::Nil => Value::Nil,
-            Value::Boolean(b) => Value::Boolean(*b),
-            Value::InternedString(s) => Value::InternedString(s),
-            Value::AllocatedString(s) => Value::AllocatedString(s.clone()),
-            Value::Keyword(s) => Value::Keyword(s),
-            Value::Number(n) => Value::Number(*n),
-            Value::Tuple(t) => Value::Tuple(t.clone()),
-            Value::Args(a) => Value::Args(a.clone()),
-            Value::Fn(f) => Value::Fn(f.clone()),
-            Value::FnDecl(name) => Value::FnDecl(name),
-            Value::List(l) => Value::List(l.clone()),
-            Value::Dict(d) => Value::Dict(d.clone()),
-            Value::Box(name, b) => Value::Box(name, b.clone()),
-            Value::Placeholder => Value::Placeholder,
-            Value::Base(b) => Value::Base(b.clone()),
-            Value::Recur(..) => unreachable!(),
-        }
-    }
-}
-
-impl fmt::Display for Value<'_> {
-    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
-        match self {
-            Value::Nil => write!(f, "nil"),
-            Value::Boolean(b) => write!(f, "{b}"),
-            Value::Number(n) => write!(f, "{n}"),
-            Value::Keyword(k) => write!(f, ":{k}"),
-            Value::InternedString(s) => write!(f, "\"{s}\""),
-            Value::AllocatedString(s) => write!(f, "\"{s}\""),
-            Value::Fn(fun) => write!(f, "fn {}", fun.borrow().name),
-            Value::FnDecl(name) => write!(f, "fn {name}"),
-            Value::Tuple(t) | Value::Args(t) => write!(
-                f,
-                "({})",
-                t.iter()
-                    .map(|x| x.to_string())
-                    .collect::<Vec<_>>()
-                    .join(", ")
-            ),
-            Value::List(l) => write!(
-                f,
-                "[{}]",
-                l.iter()
-                    .map(|x| x.to_string())
-                    .collect::<Vec<_>>()
-                    .join(", ")
-            ),
-            Value::Dict(d) => write!(
-                f,
-                "#{{{}}}",
-                d.iter()
-                    .map(|(k, v)| format!(":{k} {v}"))
-                    .collect::<Vec<_>>()
-                    .join(", ")
-            ),
-            Value::Box(name, value) => {
-                write!(
-                    f,
-                    "box {}: [{}]",
-                    name,
-                    &value.try_borrow().unwrap().to_string()
-                )
-            }
-            Value::Placeholder => write!(f, "_"),
-            Value::Base(..) => unreachable!(),
-            Value::Recur(..) => unreachable!(),
-        }
-    }
-}
-
-impl Value<'_> {
-    pub fn bool(&self) -> bool {
-        !matches!(self, Value::Nil | Value::Boolean(false))
-    }
-}
-
-impl<'src> PartialEq for Value<'src> {
-    fn eq(&self, other: &Value<'src>) -> bool {
+impl PartialEq for Value {
+    fn eq(&self, other: &Value) -> bool {
+        use Value::*;
         match (self, other) {
-            // value equality types
-            (Value::Nil, Value::Nil) => true,
-            (Value::Boolean(x), Value::Boolean(y)) => x == y,
-            (Value::Number(x), Value::Number(y)) => x == y,
-            (Value::InternedString(x), Value::InternedString(y)) => x == y,
-            (Value::AllocatedString(x), Value::AllocatedString(y)) => x == y,
-            (Value::InternedString(x), Value::AllocatedString(y)) => *x == **y,
-            (Value::AllocatedString(x), Value::InternedString(y)) => **x == *y,
-            (Value::Keyword(x), Value::Keyword(y)) => x == y,
-            (Value::Tuple(x), Value::Tuple(y)) => x == y,
-            (Value::List(x), Value::List(y)) => x == y,
-            (Value::Dict(x), Value::Dict(y)) => x == y,
-            // reference equality types
-            (Value::Fn(x), Value::Fn(y)) => {
-                Rc::<RefCell<Fn<'_>>>::as_ptr(x) == Rc::<RefCell<Fn<'_>>>::as_ptr(y)
-            }
-            (Value::Box(_, x), Value::Box(_, y)) => {
-                Rc::<RefCell<Value<'_>>>::as_ptr(x) == Rc::<RefCell<Value<'_>>>::as_ptr(y)
-            }
+            (Nothing, Nothing) | (Nil, Nil) | (True, True) | (False, False) => true,
+            (Keyword(str1), Keyword(str2)) | (Interned(str1), Interned(str2)) => str1 == str2,
+            (String(x), String(y)) => x == y,
+            (String(x), Interned(y)) => x.as_ref() == y,
+            (Interned(x), String(y)) => x == y.as_ref(),
+            (Number(x), Number(y)) => x == y,
+            (Tuple(x), Tuple(y)) => x == y,
+            (List(x), List(y)) => x == y,
+            (Dict(x), Dict(y)) => x == y,
+            (Box(x), Box(y)) => std::ptr::eq(x.as_ref().as_ptr(), y.as_ref().as_ptr()),
+            (Fn(x), Fn(y)) => std::ptr::eq(x, y),
+            (BaseFn(x), BaseFn(y)) => std::ptr::eq(x, y),
+            (Partial(x), Partial(y)) => x == y,
             _ => false,
         }
     }
 }
 
-impl Eq for Value<'_> {}
-
-impl Value<'_> {
-    pub fn interpolate(&self) -> String {
+impl std::fmt::Display for Value {
+    fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
+        use Value::*;
         match self {
-            Value::Nil => String::new(),
-            Value::Boolean(b) => format!("{b}"),
-            Value::Number(n) => format!("{n}"),
-            Value::Keyword(k) => format!(":{k}"),
-            Value::AllocatedString(s) => format!("{s}"),
-            Value::InternedString(s) => s.to_string(),
-            Value::Box(_, x) => x.borrow().interpolate(),
-            Value::Tuple(xs) => xs
-                .iter()
-                .map(|x| x.interpolate())
-                .collect::<Vec<_>>()
-                .join(", "),
-            Value::List(xs) => xs
-                .iter()
-                .map(|x| x.interpolate())
-                .collect::<Vec<_>>()
-                .join(", "),
-            Value::Dict(xs) => xs
-                .iter()
-                .map(|(k, v)| format!(":{} {}", k, v.interpolate()))
-                .collect::<Vec<_>>()
-                .join(", "),
-            Value::Fn(x) => format!("fn {}", x.borrow().name),
-            Value::FnDecl(name) => format!("fn {name}"),
-            Value::Placeholder => unreachable!(),
-            Value::Args(_) => unreachable!(),
-            Value::Recur(_) => unreachable!(),
-            Value::Base(_) => unreachable!(),
+            Nothing => write!(f, "_"),
+            Nil => write!(f, "nil"),
+            True => write!(f, "true"),
+            False => write!(f, "false"),
+            Keyword(str) => write!(f, ":{str}"),
+            Interned(str) => write!(f, "\"{str}\""),
+            String(str) => write!(f, "\"{str}\""),
+            Number(n) => write!(f, "{n}"),
+            Tuple(members) => write!(
+                f,
+                "({})",
+                members
+                    .iter()
+                    .map(|x| x.to_string())
+                    .collect::<Vec<_>>()
+                    .join(", ")
+            ),
+            List(members) => write!(
+                f,
+                "[{}]",
+                members
+                    .iter()
+                    .map(|x| x.to_string())
+                    .collect::<Vec<_>>()
+                    .join(", ")
+            ),
+            Dict(members) => write!(
+                f,
+                "#{{{}}}",
+                members
+                    .iter()
+                    .map(|(k, v)| format!("{k} {v}"))
+                    .collect::<Vec<_>>()
+                    .join(", ")
+            ),
+            Box(value) => write!(f, "box {{ {} }}", value.as_ref().borrow()),
+            Fn(lfn) => write!(f, "fn {}", lfn.name()),
+            BaseFn(_) => write!(f, "base fn"),
+            Partial(partial) => write!(f, "fn {}/partial", partial.name),
+        }
+    }
+}
+
+impl Value {
+    pub fn show(&self) -> String {
+        use Value::*;
+        match &self {
+            Nil => "nil".to_string(),
+            True => "true".to_string(),
+            False => "false".to_string(),
+            Number(n) => format!("{n}"),
+            Interned(str) => format!("\"{str}\""),
+            String(str) => {
+                let str_str = str.to_string();
+                format!("\"{str_str}\"")
+            }
+            Keyword(str) => format!(":{str}"),
+            Tuple(t) => {
+                let members = t.iter().map(|e| e.show()).collect::<Vec<_>>().join(", ");
+                format!("({members})")
+            }
+            List(l) => {
+                let members = l.iter().map(|e| e.show()).collect::<Vec<_>>().join(", ");
+                format!("[{members}]")
+            }
+            Dict(d) => {
+                let members = d
+                    .iter()
+                    .map(|(k, v)| {
+                        let key_show = Value::Keyword(k).show();
+                        let value_show = v.show();
+                        format!("{key_show} {value_show}")
+                    })
+                    .collect::<Vec<_>>()
+                    .join(", ");
+                format!("#{{{members}}}")
+            }
+            Box(x) => format!("box {{ {} }}", x.as_ref().borrow().show()),
+            Fn(lfn) => format!("fn {}", lfn.name()),
+            Partial(partial) => format!("fn {}/partial", partial.name),
+            BaseFn(_) => "base fn".to_string(),
+            Nothing => unreachable!(),
+        }
+    }
+
+    pub fn stringify(&self) -> String {
+        use Value::*;
+        match &self {
+            Nil => "nil".to_string(),
+            True => "true".to_string(),
+            False => "false".to_string(),
+            Number(n) => format!("{n}"),
+            Interned(str) => str.to_string(),
+            Keyword(str) => str.to_string(),
+            Tuple(t) => {
+                let members = t
+                    .iter()
+                    .map(|e| e.stringify())
+                    .collect::<Vec<_>>()
+                    .join(", ");
+                members.to_string()
+            }
+            List(l) => {
+                let members = l
+                    .iter()
+                    .map(|e| e.stringify())
+                    .collect::<Vec<_>>()
+                    .join(", ");
+                members.to_string()
+            }
+            Dict(d) => {
+                let members = d
+                    .iter()
+                    .map(|(k, v)| {
+                        let key_show = Value::Keyword(k).stringify();
+                        let value_show = v.stringify();
+                        format!("{key_show} {value_show}")
+                    })
+                    .collect::<Vec<_>>()
+                    .join(", ");
+                members.to_string()
+            }
+            String(s) => s.as_ref().clone(),
+            Box(x) => x.as_ref().borrow().stringify(),
+            Fn(lfn) => format!("fn {}", lfn.name()),
+            _ => todo!(),
+        }
+    }
+
+    pub fn type_of(&self) -> &'static str {
+        use Value::*;
+        match self {
+            Nothing => unreachable!(),
+            Nil => "nil",
+            True => "bool",
+            False => "bool",
+            Keyword(..) => "keyword",
+            Interned(..) => "string",
+            String(..) => "string",
+            Number(..) => "number",
+            Tuple(..) => "tuple",
+            List(..) => "list",
+            Dict(..) => "dict",
+            Box(..) => "box",
+            Fn(..) => "fn",
+            BaseFn(..) => "fn",
+            Partial(..) => "fn",
         }
     }
 }