ugh. spin my wheels a lot. decide to start work on the receive special form

assets/test_prelude.ld

@@ -1252,7 +1252,13 @@ fn msgs {
 
 fn flush! {
 	"Clears the current process's mailbox."
-	() -> base :process (:flush)}
+	() -> base :process (:flush)
+}
+
+fn flush_i! {
+	"Flushes the message at the indicated index in the current process's mailbox."
+	(i as :number) -> base :process (:flush_i, i)
+}
 
 fn sleep! {
 	"Puts the current process to sleep for at least the specified number of milliseconds."
@@ -1265,11 +1271,12 @@ fn sleep! {
 	msgs
  	spawn!
 	yield!
-
-	alive?
-	link!
-	flush!
 	sleep!
+	alive?
+	flush!
+	flush_i!
+
+	link!
 
 	abs
 	abs

may_2025_thoughts.md

@@ -121,7 +121,7 @@ A few thoughts:
 * Function calls should be different from tuple pattern matching. Tuples are currently (and maybe forever?) allocated on the heap. Function calls should *not* have to pass through the heap. The good news: `Arguments` is already a different AST node type than `Tuple`; we'll want an `ArgumentsPattern` pattern node type that's different from (and thus compiled differently than) `TuplePattern`. They'll be similar--the matching logic is the same, after all--but the arguments will be on the stack already, and won't need to be unpacked in the same way.
     - One difficulty will be matching against different arities? But actually, we should compile these arities as different functions.
     - Given splats, can we actually compile functions into different arities? Consider the following:
-    ```
+    ```ludus
       fn foo {
         (x) -> & arity 1
         (y, z) -> & arity 2
@@ -161,7 +161,7 @@ The `loop` compilation is _almost_ the same as a function body. That said, the t
 
 A few possibilities:
 * Probably the best option: enforce a new requirement that splat patterns in function clauses *must* be longer than any explicit arity of the function. So, taking the above:
-    ```
+    ```ludus
       fn foo {
         (x) -> & arity 1
         (y, z) -> & arity 2
@@ -171,7 +171,7 @@ A few possibilities:
     ```
 This would give you a validation error that splats must be longer than any other arity.
 Similarly, we could enforce this:
-```
+```ludus
 fn foo {
     (x) -> & arity 1
     (x, y) -> & arity 2
@@ -340,7 +340,7 @@ And then: quality of life improvements:
 ### Bugs discovered while trying to compile prelude
 #### 2025-06-20
 Consider the following code:
-```
+```ludus
 fn one {
     (x as :number) -> {
         fn two () -> :number
@@ -400,7 +400,7 @@ What we need to have happen is that if a function is closing over values _inside
 I think I need to consult Uncle Bob Nystrom to get a sense of what to do here.
 ***
 So I found the minimal test case:
-```
+```ludus
 let foo = {
     let thing = :thing
     let bar = :bar
@@ -445,7 +445,7 @@ To the best of my ability to tell, `if` has proper stack behaviour.
 So the question is what's happening in the interaction between the `jump_if_false` instruction and `recur`.
 
 To wit, the following code works just fine:
-```
+```ludus
 fn not {
     (false) -> true
     (nil) -> true
@@ -463,7 +463,7 @@ loop ([1, 2, 3]) with {
 ```
 
 But the following code does not:
-```
+```ludus
 let test = 2
 loop ([1, 2, 3]) with {
 	([]) -> false
@@ -522,11 +522,7 @@ SOLUTION: test to see if the function has been forward-declared, and if it has,
 NEW PROBLEM: a lot of instructions in the VM don't properly offset from the call frame's stack base, which leads to weirdness when doing things inside function calls.
 
 NEW SOLUTION: create a function that does the offset properly, and replace everywhere we directly access the stack.
-<<<<<<< HEAD
-<<<<<<< Updated upstream
 This is the thing I am about to do
-||||||| Stash base
-This is the thing I am about to do.
 
 ### I think the interpreter, uh, works?
 #### 2025-06-24
@@ -832,12 +828,12 @@ I've implemented what I decribe above. It works! I'm low-key astonished.
 It perfectly handles an infinitely recurring process! What the fuck.
 Anyway, things left to do:
 * [ ] `receive` forms are the big one: they require threading through the whole interpreter
-* [ ] implement the missing process functions at the end of prelude
+* [x] implement the missing process functions at the end of prelude
 * [ ] research how Elixir/Erlang's abstractions over processes work (I'm especially interested in how to make synchronous-looking calls); take a look especially at https://medium.com/qixxit-development/build-your-own-genserver-in-49-lines-of-code-1a9db07b6f13
 * [ ] write some examples just using these simple tools (not even GenServer, etc.) to see how they work, and to start building curriculum
 * [ ] develop a design for how to deal with asynchronous io with js
 
-```
+```ludus
 fn agent/get (pid) -> {
     send (pid, (:get, self ()))
     receive {
@@ -872,5 +868,199 @@ Two things that pop out to me:
 * The way this works is just to yield immediately. This actually makes a lot of sense. If we put them next to one another, there's no risk that there'll be backlogged `(:response, x)` messages in the mbx, right? But that makes me a little queasy.
 * The way `gen_server` works is pretty deep inversion of control; you effectively write callbacks for the `gen_server` to call. I'm not sure that's how we want to do things in Ludus; it's a handy pattern, and easy. But it's not simple. But also worth investigating. In any event, it's the foundation of all the other process patterns Elixir has developed. I need an intuiation around it.
 
+### Rethinking reception
+#### 2025-06-27
+So one thing that's stuck with me is that in Elixir, `receive` isn't a special form: it's a function that takes a block.
+It may be a macro, but it's still mostly normalish, and doesn't invovle compiler shenanigans.
+
+So, this is what I want to write:
+
+```ludus
+fn receiver () -> receive {
+    :foo -> :bar
+    :bar -> :baz
+    :baz -> :quux
+}  
+```
+There's a way to model this without the magic of receive.
+Imagine instead a function that just receives a message and matches against it:
+```ludus
+fn receive_msg (msg) -> match msg with {
+    :foo -> :bar
+    :bar -> :baz
+    :baz -> :quux
+}
+```
+But if there's no matching message clause, we get a panic.
+And panics stop the world.
+Meanwhile, we need to know whether the message matched.
+So this desugaring goes a little further:
+```ludus
+fn receive_msg (msg) -> match msg with {
+    :foo -> :bar
+    :bar -> :baz
+    :baz -> :quux
+    _ -> :does_not_understand
+}
+```
+This way we avoid a panic when there's no matching message.
+There's an easy wrapping function which looks like this:
+```ludus
+fn receive (receiver) -> {
+    let my_msgs = msgs ()
+    loop (my_msgs, 0) with {
+        ([], _) -> yield! ()
+        (xs, i) -> match receiver(first (xs)) with {
+            :does_not_understand -> recur (rest (xs), inc (i))
+            x -> {
+                flush_n! (i)
+                x
+            }
+        }
+    }
+}
+
+receive (receive_msg)
+```
+There's a thing I both like and don't like about this.
+The fact that we use a magic keyword, `:does_not_understand`, means it's actually easy to hook into the behaviour of not understanding.
+I don't know if we should panic on a process receiving a message it doesn't understand.
+Maybe we do that for now?
+Crash early and often; thanks Erlang.
+And so we do have to worry about the last clause.
+
+At an implementation level, it's worth noting that we're not optimizing for scanning through the messages--we'll scan through any messages we don't understand every time we call `receive`.
+That's probably fine!
+
+So now the thing, without sugar, is just:
+```ludus
+fn agent (x) -> receive (fn (msg) {
+    (:get, pid) -> {
+        send (pid, (:response, x))
+        agent (x)
+    }
+    (:set, y) -> agent(y)
+    (:update, f) -> agent (f (x))
+    _ -> :does_not_understand
+})
+```
+So I don't need any sugar to make `receive` work?
+And it doesn't even need to hook into the vm?
+What?
+
+#### some time later
+It turns out I do.
+The problem is that the `flush_i` instruction never gets called from Ludus in a thing like an agent, because of the recursive call.
+So the flushing would need to happen in the receiver.
 
 
+A few things that are wrong right now:
+* [ ] `loop`/`recur` is still giving me a very headache. It breaks stack discipline to avoid packing tuples into a heap-allocated vec. There may be a way to fix this in the current compiler scheme around how I do and don't handle arguments--make recur stupider and don't bother loading anything. A different solution would be to desugar loop into an anonymous function call. A final solution would be just to use a heap-allocated tuple.
+
+Minimal failing case:
+```ludus
+match (nil) with {
+    (x) -> match x with {
+        (y) -> recur (y)
+    }
+}
+```
+* [ ] The amount of sugar needed to get to `receive` without making a special form is too great.
+
+In particular, function arities need to be changed, flushes need to be inserted, anonymous lambdas need to be created (which can't be multi-clause), etc.
+
+* [ ] There was another bug that I was going to write down and fix, but I forgot what it was. Something with processes.
+
+* [ ] Also: the `butlast` bug is still outstanding: `base :slice` causes a panic in that function, but not the call to the Ludus function. Still have to investigate that one.
+
+* [ ] In testing this, it's looking like `match` is misbehaving; none of the matches that *should* happen in my fully sugarless `receive` testing are matching how they ought. 
+
+I haven't got to a minimal case, but here's what's not working:
+```ludus
+ fn receive (receiver) -> {
+  fn looper {
+   ([], _) -> yield! ()
+   (xs, i) -> {
+    print!("looping through messages:", xs)
+    match receiver (first (xs), i) with {
+     :does_not_understand -> looper (rest (xs), inc (i))
+     x -> x
+   }}
+ }
+ print! ("receiving in", self (), "with messages", msgs())
+ looper (msgs (), 0)
+}
+
+fn agent (x) -> receive (fn (msg, i) -> {
+print!("received msg in agent: ", msg)
+match msg with {
+  (:get, pid) -> {
+    flush_i! (i)
+    print!("getted from {pid}")
+    send (pid, (:response, x))
+    agent (x)
+  }
+  (:set, y) -> {flush_i!(i); print!("setted! {y}"); agent (y)}
+  (:update, f) -> {flush_i!(i);print!("updated: {f}"); agent (f (x))}
+  y -> {print!("no agent reception match!!!! {y}");:does_not_understand}
+}
+})
+
+fn agent/get (pid) -> {
+  send (pid, (:get, self ()))
+  yield! ()
+  receive (fn (msg, i) -> match msg with {
+    (:response, x) -> {flush_i! (i); x}
+  })
+}
+
+fn agent/set (pid, val) -> send (pid, (:set, val))
+
+fn agent/update (pid, f) -> send (pid, (:update, f))
+
+let counter = spawn! (fn () -> agent (0))
+agent/set (counter, 12)
+agent/update (counter, inc)
+agent/update (counter, mult(_, 3))
+agent/get (counter)
+```
+
+_I haven't been able to recreate this, and the code above is subtle enough that the Ludus may be behaving as expected; the following works as expected:_
+
+```ludus
+fn receive (receiver) -> {
+  print! ("receiving in", self (), "with msgs", msgs())
+  if empty? (msgs ())
+    then {yield! (); receive (receiver)}
+    else do msgs () > first > receiver
+}
+
+fn foo? (val) -> receive (fn (msg) -> match report!("scrutinee is", msg) with {
+  (:report) -> {
+    print! ("LUDUS SAYS ==> value is {val}")
+    flush! ()
+    foo? (val)
+  }
+  (:set, x) -> {
+    print! ("LUDUS SAYS ==> foo! was {val}, now is {x}")
+    flush! ()
+    foo? (x)
+  }
+  (:get, pid) -> {
+    print! ("LUDUS SAYS ==> value is {val}")
+    send (pid, (:response, val))
+    flush! ()
+    foo? (val)
+  }
+  x -> print! ("LUDUS SAYS ==> no match, got {x}")
+})
+
+let foo = spawn! (fn () -> foo? (42))
+print! (foo)
+send (foo, (:set, 23))
+yield! ()
+send (foo, (:get, self ()))
+yield! ()
+fn id (x) -> x
+receive(id)
+```

sandbox.ld

@@ -1,20 +1,35 @@
-fn reporter () -> {
+fn receive (receiver) -> {
-  print! (self (), msgs ())
+  print! ("receiving in", self (), "with msgs", msgs())
+  if empty? (msgs ())
+    then {yield! (); receive (receiver)}
+    else do msgs () > first > receiver
 }
 
-fn printer () -> {
+fn foo? (val) -> receive (fn (msg) -> match report!("scrutinee is", msg) with {
-  print!("LUDUS SAYS ==> hi")
+  (:report) -> {
-  sleep! (1)
+    print! ("LUDUS SAYS ==> value is {val}")
-  printer ()
+    flush! ()
+    foo? (val)
   }
+  (:set, x) -> {
+    print! ("LUDUS SAYS ==> foo! was {val}, now is {x}")
+    flush! ()
+    foo? (x)
+  }
+  (:get, pid) -> {
+    print! ("LUDUS SAYS ==> value is {val}")
+    send (pid, (:response, val))
+    flush! ()
+    foo? (val)
+  }
+  x -> print! ("LUDUS SAYS ==> no match, got {x}")
+})
 
-
+let foo = spawn! (fn () -> foo? (42))
-let foo = spawn! (printer)
+print! (foo)
-let bar = spawn! (reporter)
+send (foo, (:set, 23))
-
+yield! ()
-send (bar, [:foo, :bar, :baz])
+send (foo, (:get, self ()))
-
+yield! ()
-& yield! ()
+fn id (x) -> x
-sleep! (5)
+receive(id)
-
-:done

src/vm.rs

@@ -13,7 +13,7 @@ use std::fmt;
 use std::mem::swap;
 use std::rc::Rc;
 
-const MAX_REDUCTIONS: usize = 100;
+const MAX_REDUCTIONS: usize = 1000;
 
 #[derive(Debug, Clone, PartialEq)]
 pub enum Panic {
@@ -91,7 +91,7 @@ pub struct Creature {
     pub result: Option<Result<Value, Panic>>,
     debug: bool,
     last_code: usize,
-    pub id: &'static str,
+    pub pid: &'static str,
     pub mbx: VecDeque<Value>,
     pub reductions: usize,
     pub zoo: Rc<RefCell<Zoo>>,
@@ -100,7 +100,7 @@ pub struct Creature {
 
 impl std::fmt::Display for Creature {
     fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
-        write!(f, "Creature. {} @{}", self.id, self.ip)
+        write!(f, "Creature. {} @{}", self.pid, self.ip)
     }
 }
 
@@ -136,7 +136,7 @@ impl Creature {
             result: None,
             debug,
             last_code: 0,
-            id: "",
+            pid: "",
             zoo,
             mbx: VecDeque::new(),
             reductions: 0,
@@ -197,7 +197,7 @@ impl Creature {
             .join("/");
         println!(
             "{:04}: [{inner}] ({register}) {} {{{mbx}}}",
-            self.last_code, self.id
+            self.last_code, self.pid
         );
     }
 
@@ -230,11 +230,13 @@ impl Creature {
 
     pub fn panic(&mut self, msg: &'static str) {
         let msg = format!("{msg}\nPanic traceback:\n{}", self.call_stack());
+        println!("process {} panicked!\n{msg}", self.pid);
         self.result = Some(Err(Panic::String(msg)));
     }
 
     pub fn panic_with(&mut self, msg: String) {
         let msg = format!("{msg}\nPanic traceback:\n{}", self.call_stack());
+        println!("process {} panicked!\n{msg}", self.pid);
         self.result = Some(Err(Panic::String(msg)));
     }
 
@@ -266,22 +268,35 @@ impl Creature {
     }
 
     fn handle_msg(&mut self, args: Vec<Value>) {
-        println!("message received! {}", args[0]);
+        println!("message received by {}: {}", self.pid, args[0]);
         let Value::Keyword(msg) = args.first().unwrap() else {
             return self.panic("malformed message to Process");
         };
         match *msg {
-            "self" => self.push(Value::Keyword(self.id)),
+            "self" => self.push(Value::Keyword(self.pid)),
             "msgs" => {
                 let msgs = self.mbx.iter().cloned().collect::<Vec<_>>();
                 let msgs = Vector::from(msgs);
+                println!(
+                    "delivering messages: {}",
+                    msgs.iter()
+                        .map(|x| x.show())
+                        .collect::<Vec<_>>()
+                        .join(" | ")
+                );
                 self.push(Value::List(Box::new(msgs)));
             }
             "send" => {
                 let Value::Keyword(pid) = args[1] else {
                     return self.panic("malformed pid");
                 };
-                if self.id == pid {
+                println!(
+                    "sending msg from {} to {} of {}",
+                    self.pid,
+                    pid,
+                    args[2].show()
+                );
+                if self.pid == pid {
                     self.mbx.push_back(args[2].clone());
                 } else {
                     self.zoo
@@ -300,7 +315,7 @@ impl Creature {
             }
             "yield" => {
                 self.r#yield = true;
-                println!("yielding from {}", self.id);
+                println!("yielding from {}", self.pid);
                 self.push(Value::Keyword("ok"));
             }
             "alive" => {
@@ -317,28 +332,64 @@ impl Creature {
             "link" => todo!(),
             "flush" => {
                 self.mbx = VecDeque::new();
+                println!("flushing messages in {}", self.pid);
                 self.push(Value::Keyword("ok"));
             }
             "sleep" => {
-                println!("sleeping {} for {}", self.id, args[1]);
+                println!("sleeping {} for {}", self.pid, args[1]);
                 let Value::Number(ms) = args[1] else {
                     unreachable!()
                 };
-                self.zoo.as_ref().borrow_mut().sleep(self.id, ms as usize);
+                self.zoo.as_ref().borrow_mut().sleep(self.pid, ms as usize);
                 self.r#yield = true;
                 self.push(Value::Keyword("ok"));
             }
+            // "flush_i" => {
+            //     let Value::Number(n) = args[1] else {
+            //         unreachable!()
+            //     };
+            //     println!("flushing message at {n}");
+            //     self.mbx.remove(n as usize);
+            //     println!(
+            //         "mailbox is now: {}",
+            //         self.mbx
+            //             .iter()
+            //             .map(|msg| msg.to_string())
+            //             .collect::<Vec<_>>()
+            //             .join(" | ")
+            //     );
+            //     self.push(Value::Keyword("ok"));
+            // }
             msg => panic!("Process does not understand message: {msg}"),
         }
     }
 
     pub fn interpret(&mut self) {
+        println!("starting process {}", self.pid);
+        println!(
+            "mbx: {}",
+            self.mbx
+                .iter()
+                .map(|x| x.show())
+                .collect::<Vec<_>>()
+                .join(" | ")
+        );
         loop {
             if self.at_end() {
-                self.result = Some(Ok(self.stack.pop().unwrap()));
+                let result = self.stack.pop().unwrap();
+                println!("process {} has returned {result}", self.pid);
+                self.result = Some(Ok(result));
                 return;
             }
-            if self.reductions >= MAX_REDUCTIONS || self.r#yield {
+            if self.r#yield {
+                println!("process {} has explicitly yielded", self.pid);
+                return;
+            }
+            if self.reductions >= MAX_REDUCTIONS {
+                println!(
+                    "process {} is yielding after {MAX_REDUCTIONS} reductions",
+                    self.pid
+                );
                 return;
             }
             let code = self.read();
@@ -1144,6 +1195,7 @@ impl Creature {
                             self.push(value);
                         }
                         None => {
+                            println!("process {} has returned with {}", self.pid, value);
                             self.result = Some(Ok(value));
                             return;
                         }

src/world.rs

@@ -106,7 +106,7 @@ impl Zoo {
         if self.empty.is_empty() {
             let id = self.new_id();
             let idx = self.procs.len();
-            proc.id = id;
+            proc.pid = id;
             self.procs.push(Status::Nested(proc));
             self.ids.insert(id, idx);
             id
@@ -114,7 +114,7 @@ impl Zoo {
             let idx = self.empty.pop().unwrap();
             let rand = ran_u8() as usize % 24;
             let id = format!("{}_{idx}", ANIMALS[rand]).leak();
-            proc.id = id;
+            proc.pid = id;
             self.ids.insert(id, idx);
             self.procs[idx] = Status::Nested(proc);
             id
@@ -186,7 +186,7 @@ impl Zoo {
     }
 
     pub fn release(&mut self, proc: Creature) {
-        let id = proc.id;
+        let id = proc.pid;
         if let Some(idx) = self.ids.get(id) {
             let mut proc = Status::Nested(proc);
             swap(&mut proc, &mut self.procs[*idx]);
@@ -203,7 +203,7 @@ impl Zoo {
         match &self.procs[self.active_idx] {
             Status::Empty => false,
             Status::Borrowed => false,
-            Status::Nested(proc) => !self.sleeping.contains_key(proc.id),
+            Status::Nested(proc) => !self.sleeping.contains_key(proc.pid),
         }
     }
 
@@ -219,7 +219,7 @@ impl Zoo {
         }
         match &self.procs[self.active_idx] {
             Status::Empty | Status::Borrowed => unreachable!(),
-            Status::Nested(proc) => proc.id,
+            Status::Nested(proc) => proc.pid,
         }
     }
 
@@ -253,26 +253,11 @@ impl World {
         }
     }
 
-    // pub fn spawn(&mut self, proc: Creature) -> Value {
-    //     let id = self.zoo.put(proc);
-    //     Value::Keyword(id)
-    // }
-
-    // pub fn send_msg(&mut self, id: &'static str, msg: Value) {
-    //     let mut proc = self.zoo.catch(id);
-    //     proc.receive(msg);
-    //     self.zoo.release(proc);
-    // }
-    //
     fn next(&mut self) {
         let mut active = None;
         swap(&mut active, &mut self.active);
         let mut zoo = self.zoo.as_ref().borrow_mut();
         zoo.release(active.unwrap());
-        // at the moment, active is None, process is released.
-        // zoo should NOT need an id--it has a representation of the current active process
-        // world has an active process for memory reasons
-        // not for state-keeping reasons
         let new_active_id = zoo.next();
         let mut new_active_proc = zoo.catch(new_active_id);
         new_active_proc.reset_reductions();
@@ -280,62 +265,13 @@ impl World {
         swap(&mut new_active_opt, &mut self.active);
     }
 
-    // fn old_next(&mut self) {
-    //     let id = self
-    //         .zoo
-    //         .as_ref()
-    //         .borrow_mut()
-    //         .next(self.active.as_ref().unwrap().id);
-    //     println!("next id is {id}");
-    //     let mut active = None;
-    //     std::mem::swap(&mut active, &mut self.active);
-    //     let mut holding_pen = self.zoo.as_ref().borrow_mut().catch(id);
-    //     let mut active = active.unwrap();
-    //     std::mem::swap(&mut active, &mut holding_pen);
-    //     println!("now in the holding pen: {}", holding_pen.id);
-    //     holding_pen.reset_reductions();
-    //     self.zoo.as_ref().borrow_mut().release(holding_pen);
-    //     let mut active = Some(active);
-    //     std::mem::swap(&mut active, &mut self.active);
-    // }
-
-    // pub fn sleep(&mut self, id: &'static str) {
-    //     // check if the id is the actually active process
-    //     if self.active.id != id {
-    //         panic!("attempted to sleep a process from outside that process: active = {}; to sleep: = {id}", self.active.id);
-    //     }
-    //     self.next(id);
-    // }
-
-    // pub fn panic(&mut self, id: &'static str, panic: Panic) {
-    //     // TODO: devise some way of linking processes (study the BEAM on this)
-    //     // check if the id is active
-    //     if self.active.id != id {
-    //         panic!("attempted to panic from a process from outside that process: active = {}; panicking = {id}; panic = {panic}", self.active.id);
-    //     }
-    //     // check if the process is `main`, and crash the program if it is
-    //     if self.main == id {
-    //         self.result = self.active.result.clone();
-    //     }
-    //     // kill the process
-    //     self.zoo.kill(id);
-    //     self.next(id);
-    // }
-
-    // pub fn complete(&mut self) {
-    //     if self.main == self.active.id {
-    //         self.result = self.active.result.clone();
-    //     }
-    //     self.next(id);
-    // }
-
     pub fn activate_main(&mut self) {
         let main = self.zoo.as_ref().borrow_mut().catch(self.main);
         self.active = Some(main);
     }
 
     pub fn active_id(&mut self) -> &'static str {
-        self.active.as_ref().unwrap().id
+        self.active.as_ref().unwrap().pid
     }
 
     pub fn kill_active(&mut self) {
@@ -363,7 +299,11 @@ impl World {
                         self.result = self.active_result().clone();
                         return;
                     }
-                    println!("process died: {}", self.active_id());
+                    println!(
+                        "process {} died with {:?}",
+                        self.active_id(),
+                        self.active_result().clone()
+                    );
                     self.kill_active();
                 }
             }