Thoughts about concurrency, actors, and objects #96

New Issue

scott · 2024-06-29T05:48:05Z

scott commented

2024-06-29 05:48:05 +00:00

I've started work in (in the twc/jactors repo) to wrap Janet's fibers into Erlang/Elixir-style processes. This is both because I like actors, AND, beyond personal taste, actors allow BOTH for really interesting turtle graphics and natural systems processing AND to simulate Smalltalk-style objects. (Alan Kay is on record as saying that Erlang is the only true object-oriented programming environment, even more than Smalltalk; and while that's a variety of nonsense, it does allow me some license here.)

So, how would Ludus actors work?

A naive translation of Elixir/Erlang/BEAM actors would perhaps look like this:

let my_actor = spawn! (fn () -> print! (:foo)) &=> Actor ID: 2 
my_actor :status &=> :dead
self () &=> Actor ID: 1
send! (self (), :bar) & => :ok

receive {
    :bar -> :yay!
    _ -> :oops
} &=> :yay!
~> :foo

Consider this Elixir code:

defmodule KV do
  def start_link do
    Task.start_link(fn -> loop(%{}) end)
  end

  defp loop(map) do
    receive do
      {:get, key, caller} ->
        send(caller, Map.get(map, key))
        loop(map)
      {:put, key, value} ->
        loop(Map.put(map, key, value))
    end
  end
end

The Ludus equivalent (again, naively):

ns KV {

  fn loop (dict) -> receive {
    (:get, key, caller) -> {
      send! (caller, get (dict, key))
      loop (dict)
    }
    (:put, key, value) -> 
      loop (assoc (dict, key, value))
  }

  fn init () -> spawn! (fn () -> loop (#{}))

}

let my_kv = KV :init ()
send! (my_kv, (:put, :foo, 1))
send! (my_kv, (:put, :bar, 2))
send! (my_kv, (:get, :foo, self ())
flush! ()
~> 1

fn local_get (kv, key) -> {
  send! (kv, (:get, key, self ())
  receive {
    msg -> msg
  }
}

local_get (my_kv, :bar) & => 2

This local_get function shows that self () is pretty powerful: this allows for what looks like synchronous code that is asynchronous. As well as for abstractions that are built in Ludus on top of pretty basic actors.

If, however, actors are going to be our objects, the send! function feels somewhat cumbersome, no? send! (turtle_1, :forward, 100) isn't terrible, but receive { (:forward, steps) -> forward! (steps); ... } seems rather cumbersome.

You end up writing objects as recursive functions that keep state in their arguments with a giant receive form. This isn't terrible per se, but it leads me to want some simple syntax sugar.

Perhaps actor ::method (args) is shorthand for send! (actor, :method, (args), self ()); receive {msg -> msg}. Sure! That's swell. (We don't want bare keywords, since that suggests that actor :method will be non-nil.

What about making objects?

In many ways, having a recursive function keep state is the model here, NOT anything else. So I think there's some research to do to understand some syntax sugar that doesn't obfuscate the idea that (a) objects are processes: recursive functions running and yielding, not things; and (b) methods are actually messages, not special functions.

So NOT:

obj KV {
  :get (key) -> get (dict, key)
}

There's too much magic there! We need to remember that we're sending the result as a message and not returning a value from a function. And also, that the function we're actually in is recurring, not returning.

One possibility is really just to model objects explicitly as recursive functions that pass their state back to themselves. This keeps objects simple! Consider the following:

fn kv {
  () -> kv (#{})
  (dict) -> receive {
    (:put, key, value) -> kv (assoc (dict, key, value))
    (:get, key, sender) -> {
      send! (sender, get (dict, key))
      kv (dict)
    }
  }
}

fn deliver! () -> receive { msg -> msg } 

let mykv = spawn! (kv)
send! (mykv, :put, :foo, 1)
send! (mykv, :get, :foo)
let myfoo = deliver! ()

(Note that probably we need some transaction ID for the message to be safe; if there are messages waiting, then we may return one of those, rather than the value we wanted to extract from KV.)

(Also note that this is independent of whether we want to do selective messages, à la the BEAM, since receive { msg -> msg} will, of course, indiscriminately return the first message in the mailbox.)

What we see here, though, is a simplified multi-clause function that looks like a pretty cute little pattern: the state we want to hold is the argument, the init or new function is just the nullity call to that function.

Next up: thoughts about errors; and then reading into Smalltalk's way of doing things to see if this is actually a useful way of modelling/thinking about Smalltalk and objects.

I've started work in (in the twc/jactors repo) to wrap Janet's fibers into Erlang/Elixir-style processes. This is both because I like actors, AND, beyond personal taste, actors allow BOTH for really interesting turtle graphics and natural systems processing AND to simulate Smalltalk-style objects. (Alan Kay is on record as saying that Erlang is the only true object-oriented programming environment, even more than Smalltalk; and while that's a variety of nonsense, it does allow me some license here.) So, how would Ludus actors work? --- A naive translation of Elixir/Erlang/BEAM actors would perhaps look like this: ``` let my_actor = spawn! (fn () -> print! (:foo)) &=> Actor ID: 2 my_actor :status &=> :dead self () &=> Actor ID: 1 send! (self (), :bar) & => :ok receive { :bar -> :yay! _ -> :oops } &=> :yay! ~> :foo ``` Consider this Elixir code: ```elixir defmodule KV do def start_link do Task.start_link(fn -> loop(%{}) end) end defp loop(map) do receive do {:get, key, caller} -> send(caller, Map.get(map, key)) loop(map) {:put, key, value} -> loop(Map.put(map, key, value)) end end end ``` The Ludus equivalent (again, naively): ``` ns KV { fn loop (dict) -> receive { (:get, key, caller) -> { send! (caller, get (dict, key)) loop (dict) } (:put, key, value) -> loop (assoc (dict, key, value)) } fn init () -> spawn! (fn () -> loop (#{})) } let my_kv = KV :init () send! (my_kv, (:put, :foo, 1)) send! (my_kv, (:put, :bar, 2)) send! (my_kv, (:get, :foo, self ()) flush! () ~> 1 fn local_get (kv, key) -> { send! (kv, (:get, key, self ()) receive { msg -> msg } } local_get (my_kv, :bar) & => 2 ``` This `local_get` function shows that `self ()` is pretty powerful: this allows for what looks like synchronous code that is asynchronous. As well as for abstractions that are built in Ludus on top of pretty basic actors. --- If, however, actors are going to be our objects, the `send!` function feels somewhat cumbersome, no? `send! (turtle_1, :forward, 100)` isn't terrible, but `receive { (:forward, steps) -> forward! (steps); ... }` seems rather cumbersome. You end up writing objects as recursive functions that keep state in their arguments with a giant `receive` form. This isn't terrible _per se_, but it leads me to want some simple syntax sugar. Perhaps `actor ::method (args)` is shorthand for `send! (actor, :method, (args), self ()); receive {msg -> msg}`. Sure! That's swell. (We don't want bare keywords, since that suggests that `actor :method` will be non-`nil`. What about making objects? In many ways, having a recursive function keep state is the model here, NOT anything else. So I think there's some research to do to understand some syntax sugar that doesn't obfuscate the idea that (a) objects are _processes_: recursive functions running and yielding, not _things_; and (b) methods are actually _messages_, not special functions. So NOT: ``` obj KV { :get (key) -> get (dict, key) } ``` There's too much magic there! We need to remember that we're _sending the result as a message_ and not _returning a value from a function_. And also, that the function we're actually in is recurring, not returning. --- One possibility is really just to model objects explicitly as recursive functions that pass their state back to themselves. This keeps objects simple! Consider the following: ``` fn kv { () -> kv (#{}) (dict) -> receive { (:put, key, value) -> kv (assoc (dict, key, value)) (:get, key, sender) -> { send! (sender, get (dict, key)) kv (dict) } } } fn deliver! () -> receive { msg -> msg } let mykv = spawn! (kv) send! (mykv, :put, :foo, 1) send! (mykv, :get, :foo) let myfoo = deliver! () ``` (Note that probably we need some transaction ID for the message to be safe; if there are messages waiting, then we may return one of those, rather than the value we wanted to extract from KV.) (Also note that this is independent of whether we want to do selective messages, à la the BEAM, since `receive { msg -> msg}` will, of course, indiscriminately return the first message in the mailbox.) What we see here, though, is a simplified multi-clause function that looks like a pretty cute little pattern: the state we want to hold is the argument, the `init` or `new` function is just the nullity call to that function. --- Next up: thoughts about errors; and then reading into Smalltalk's way of doing things to see if this is actually a useful way of modelling/thinking about Smalltalk and objects.

scott commented

2024-06-29 05:56:06 +00:00

Oh, oh, and I want to add that a very interesting form of prototypical inheritance could be ~~happily added~~ maybe possible by simply sending a state dict to the prototype function:

fn proto {
  () -> proto (#{:foo 23})
  (dict) -> receive {
    (:get, key, caller) -> { send! (caller, get (dict, key)); proto (dict) }
  }
}

fn instance {
  () -> instance (#{:foo 42})
  (dict) -> receive {
    (:foo, caller) -> { send! (caller, dict :foo); instance (dict) }
    _ -> proto (dict)
  }
}

But of course, the call out to proto means that proto's recursion takes over. There's something here, but I don't know what it is yet.

Oh, oh, and I want to add that a very interesting form of prototypical inheritance could be ~~happily added~~ maybe possible by simply sending a state dict to the prototype function: ``` fn proto { () -> proto (#{:foo 23}) (dict) -> receive { (:get, key, caller) -> { send! (caller, get (dict, key)); proto (dict) } } } fn instance { () -> instance (#{:foo 42}) (dict) -> receive { (:foo, caller) -> { send! (caller, dict :foo); instance (dict) } _ -> proto (dict) } } ``` But of course, the call out to `proto` means that `proto`'s recursion takes over. There's something here, but I don't know what it is yet.

scott commented

2024-07-01 23:31:27 +00:00

The comment above was written in the small hours of the morning. What we'd need is something more like this:

fn super (state, message, cb) -> {
  match message with {
    (:bar, caller) -> { send! (caller, state :bar); cb (state) }
  }
}

fn instance {
  () -> instance (#{:foo 42, :bar 23})
  (dict) -> receive {
    (:foo, caller) -> { send! (caller, dict :foo); instance (dict) }
    msg -> super (dict, msg, instance)
  }
}

That is: this is more like a class/inheritance model than a prototype model. The super function here has a different interface than the instance function: the super delegates back to a callback/instance function, and takes a message as a normal function argument; the instance is a tail-recursive function that has a receive form for getting messages.

In other words, the lovely symmetry of prototypical inheritance isn't here; it's not "just objects" all the way down.

Meanwhile, because these are functions, rather than data structures, they're not really available for runtime manipulation. (Then again, almost nothing in Ludus is available for that type of manipulation.)

That said, it's fairly easy to use arity to describe an interface: nullary call for init (or unary, to pass in state), unary call for the loop, ternary call to use the prototype.

Thus we get something more like this:

fn my_object {
  () -> my_object (#{})
  (state) -> receive {
    msg -> my_object (state, msg, my_object) & use recursion to self-prototype
  }
  (state, msg, cb) -> match msg with {
    (:get, key, caller) -> {
      send! (caller, get (state, key))
      cb (state)   
    }
    &...
    _ -> super (state, msg, cb)
  }
}

That's a fair amount of machinery and ceremony, but it does something like exactly what we want. This is a reasonable way, coupled with something like Elixir's Agent or GenServer to get true message-passing object-oriented behaviour (with method_missing options) in Ludus.

We'd still also need some transaction-bookkeeping to mock immediate-return method calls.

Ultimately, the question is whether this is too much machinery. Or if it can be made easier (I think it's already, actually, quite simple). Or rather, message passing and the idea of multiple instances of a "class" whose behaviour is described in a single recursive function is plenty simple. Do we really need inheritance hierarchies?

The comment above was written in the small hours of the morning. What we'd need is something more like this: ``` fn super (state, message, cb) -> { match message with { (:bar, caller) -> { send! (caller, state :bar); cb (state) } } } fn instance { () -> instance (#{:foo 42, :bar 23}) (dict) -> receive { (:foo, caller) -> { send! (caller, dict :foo); instance (dict) } msg -> super (dict, msg, instance) } } ``` That is: this is more like a class/inheritance model than a prototype model. The `super` function here has a different interface than the `instance` function: the `super` delegates back to a callback/instance function, and takes a message as a normal function argument; the `instance` is a tail-recursive function that has a `receive` form for getting messages. In other words, the lovely symmetry of prototypical inheritance isn't here; it's not "just objects" all the way down. Meanwhile, because these are functions, rather than data structures, they're not really available for runtime manipulation. (Then again, almost nothing in Ludus is available for that type of manipulation.) That said, it's fairly easy to use arity to describe an interface: nullary call for `init` (or unary, to pass in state), unary call for the loop, ternary call to use the prototype. Thus we get something more like this: ``` fn my_object { () -> my_object (#{}) (state) -> receive { msg -> my_object (state, msg, my_object) & use recursion to self-prototype } (state, msg, cb) -> match msg with { (:get, key, caller) -> { send! (caller, get (state, key)) cb (state) } &... _ -> super (state, msg, cb) } } ``` That's a fair amount of machinery and ceremony, but it does something like exactly what we want. This is a reasonable way, coupled with something like Elixir's `Agent` or `GenServer` to get true message-passing object-oriented behaviour (with `method_missing` options) in Ludus. We'd still also need some transaction-bookkeeping to mock immediate-return method calls. Ultimately, the question is whether this is too _much_ machinery. Or if it can be made easier (I think it's already, actually, quite simple). Or rather, message passing and the idea of multiple instances of a "class" whose behaviour is described in a single recursive function is plenty simple. Do we really need inheritance hierarchies?

scott referenced this issue

2024-07-01 23:52:49 +00:00

Thoughts about dicts and objects #97

scott commented

2024-07-02 23:42:43 +00:00

After a conversation with @matt this afternoon, we've decided to go this way! The Actor Model it is.

scott added the

labels 2024-07-02 23:43:50 +00:00

scott added this to the Actors project 2024-07-03 00:16:27 +00:00

scott commented

2024-07-21 20:33:09 +00:00

NB: The first object-orientation in Logo was the Actor Model, in 1973.

cf. Solomon et al., "The History of LOGO" in ACM HOPL; and also Kay, "The Early History of Smalltalk," also in HOPL.

NB: The first object-orientation in Logo was the Actor Model, in 1973. cf. Solomon et al., "The History of LOGO" in ACM HOPL; and also Kay, "The Early History of Smalltalk," also in HOPL.

Sign in to join this conversation.