FOOP references [update 4]

[hartrock]

After some experimenting with interesting and helpful error messages - about 'protected container of (self)' - I've come to the following technique of using FOOP references; combining features of FOOPs with (additional) contexts used as shared references to them.

Note: this is advanced and new (at least for me) stuff, so you have been warned ;-)

Code first
Let's code speak first; starting newlisp with:

Code: Select all

(define (F:F) ; same as (new Class 'F), but with informational output
  (++ F:foopCount)
  (println "\nF " F:foopCount " functor")
  (cons (context) (args)))
;; some getter/setter
(define (F:g1 a) (self 1))          (define (F:g2 a) (self 2))
(define (F:s1 a) (setq (self 1) a)) (define (F:s2 a) (setq (self 2) a))
;;
(define (fun-mod-foop)
  (:s1 foop (append (:g1 foop) " -> modified by fun-mod-foop")))
;;
;; work with a foop
(set 'foop (F "one" "two"))
(println "foop before modifying: "foop)
(fun-mod-foop)
(println "foop after modifying: "foop)
;;
;;
(define (fun-mod-foop-arg aFoop)
  (:s1 aFoop (append (:g1 aFoop) " -> modified by fun-mod-foop-arg"))
  (if (context? aFoop)
      (println "aFoop arg (modified): " aFoop
               ",\n  (default aFoop): " (default aFoop))
      (println "aFoop arg (modified): " aFoop)))
;;
;; work with a foop
(set 'foop (F "one" "two"))
(println "foop before modifying as arg: " foop)
(fun-mod-foop-arg foop)
(println "foop after modifying as arg: " foop
         "\n-> does not work (call by value)")
;;
;;
;; FOOP references
;;
(context 'FR)
(define (FR:FR)
  (++ id)
  (println "\nFR " id " functor")
  (letn ((ref_contextStr (string "FR_" id))
         (ref_contextSym (sym ref_contextStr MAIN))
         (ref_context (context ref_contextSym))
         (ref_functorSym (sym ref_contextStr ref_context)))
    (set 'foop (append (list MAIN:FR ref_context) (args)))
    (set ref_functorSym foop)
    ref_context))
;; some getter/setter (ix incremented by one, because of storing ref at 1)
(define (g1 a) (self 2))          (define (g2 a) (self 3))
(define (s1 a) (setq (self 2) a)) (define (s2 a) (setq (self 3) a))
;;
;;
(context MAIN)
;; work with a foop reference
(set 'foop (FR "one" "two"))
(println "foop before modifying as ref arg: " foop
         ",\n  (default foop): " (default foop))
(fun-mod-foop-arg foop)
(println "foop after modifying as ref arg: " foop
         ",\n  (default foop): " (default foop))
(println "-> does work (call by ref)")
(println "\n(symbols F):\n  " (symbols F) "\n(symbols FR):\n  " (symbols FR) "\n(symbols FR_1):\n  " (symbols FR_1))


, results into:

Code: Select all

sr@free:~/newLISP/Examples$ newlisp FOOPRefs.lsp

F 1 functor
foop before modifying: (F "one" "two")
foop after modifying: (F "one -> modified by fun-mod-foop" "two")

F 2 functor
foop before modifying as arg: (F "one" "two")
aFoop arg (modified): (F "one -> modified by fun-mod-foop-arg" "two")
foop after modifying as arg: (F "one" "two")
-> does not work (call by value)

FR 1 functor
foop before modifying as ref arg: FR_1,
  (default foop): (FR FR_1 "one" "two")
aFoop arg (modified): FR_1,
  (default aFoop): (FR FR_1 "one -> modified by fun-mod-foop-arg" "two")
foop after modifying as ref arg: FR_1,
  (default foop): (FR FR_1 "one -> modified by fun-mod-foop-arg" "two")
-> does work (call by ref)

(symbols F):
  (F:F F:foopCount F:g1 F:g2 F:s1 F:s2)
(symbols FR):
  (FR:FR FR:a FR:foop FR:g1 FR:g2 FR:id FR:ref_context FR:ref_contextStr
 FR:ref_contextSym FR:ref_functorSym FR:s1 FR:s2)
(symbols FR_1):
  (FR_1:FR_1)
newLISP v.10.6.4 64-bit on Linux IPv4/6 UTF-8 libffi, options: newlisp -h

>


Why this effort?
Idea is to combine:
- reuse of code by utilizing FOOPs, and
- using FOOPs as referenced objects to share state changes of them.

My current usecase are multiple state machines modeled as FOOPs, which are given as argument to functions changing their state.

Usecase multiple state machines (SMs)
[Update 4]:
There is a switch to a better usecase - easier to understand and more to the point of illustrating the usefulness of FOOPReferences -, see

• http://www.newlispfanclub.alh.net/forum/viewtopic.php?f=5&t=4746

PS: Sometime there may be some code regarding FM (flow machine) mentioned below (but this would be another topic).

[Update 3]:

I've made the mistake:

Newcomers to the state machine formalism often confuse state diagrams with flowcharts.

; taken from
https://en.wikipedia.org/wiki/State_dia ... flowcharts
.
Therefrom I've come to the conclusion, that it's better to rename SM (state machine) to FM (flow(chart) machine): nodes in graph (b) from wikipedia link just given - including the decision, which node 'do Y' or 'do Z' will be entered after node 'do X' - are roughly the ones modeled in usecase below.
'Roughly', because a node in FM (wrongly named SM) below may advance-to different nodes dependent on its computation (like the decision in graph (b) extended to branch to more successors (and optionally to do some computation itself)).

After updating terminology (at first in software) a further update will follow as a new topic post.

[Update 2]:

To avoid confusion about State Machine terminology: I've looked into
https://en.wikipedia.org/wiki/Finite-st ... erminology
: simple SM in usecase below is not an SM in the strict sense of definition there.

In usecase below

• a state handler
  
  1. actively looks for data and conditions in its environment - input or other things -,
     
  2. does some computation, and
     
  3. decides itself about the transition to the next state;
• the simple SM just maps the id of a state to a function to be evaluated;
• both together will be combined in a loop - currently outside simple SM only (but this will be changed later).

This active behavior of states stems from transforming parts of an existing program into named states mapped to functions; and is non-standard.

What do You think:
Any ideas for a better naming? Program Flow State Machine? Or totally different with not using the term 'SM' at all'?
But it feels like being an SM in a more general sense...

A simple (please note [Update 2]) state machine (pseudo code):

Code: Select all

(set 'sm_1 (SM))
(:add-states sm_1 '(
  ("start"               handle_START ...)
  ("check_preconditions" handle_check_preconditions_1 ...)
  ...))
(set 'sm_2 (SM))
(:add-states sm_2 '(
  ("start"               handle_START ...)
  ("check_preconditions" handle_notImplemented ...)
  ...))


Somewhere there is:

Code: Select all

(:advance sm_1 "start")
;; and:
(:advance sm_2 "start")


setting state to "start" for getting it started.

Current state like "start" or "check_preconditions" is stored as element in SM's FOOP; then there may be:

Code: Select all

;; could be the same for both state machines:
(define (handle_START sm) ; ref arg to update state at caller, too
  (:advance sm "check_preconditions"))
(define (handle_notImplemented sm)
  (error "not implemented")
  (exit 1))

;; sm_1 may map state "check_preconditions" to:
(define (handle_check_preconditions_1 sm)
  ... ; do some checking
  (:advance sm "doSomeWork"))
;; the other one to handle_notImplemented above


For actively changing current state of state machine these handler_* funcs need access to the SM calling them: besides of advancing to the next state, this may be manipulating some data - e.g. for sharing it between states - stored inside SM, or changing SM's behavior.
Note: passively changing state of SM may also be modeled by returning some value containing at least next state (and potentially more data).

For sharing a handler between multiple SMs, it is helpful to have calling SM as a call parameter.
[Update 1] Valid for active handlers: for passive ones there is no need for referencing the caller.

Caller of handle_* funcs is state machine FOOP: to give a reference to it as call parameter, it is needed to keep FOOP reference as part of FOOP itself; e.g. part of FOOP Class code is:

Code: Select all

(define (eval-curr-func)
  ((eval (curr-func-sym)) (self 1)))


; where (curr-func-sym) is something like handle_start, and (self 1) refers to FOOP's reference context (besides of (self 0) referring to its well known Class context).

As an addition it's possible to use FOOP's reference context directly by forwarding referenced FOOP calls:

Code: Select all

(set (sym "advance" ref_context)
     (lambda (next) (:advance (context) next)))


; which allows to replace:

Code: Select all

(:advance sm "do_cond")


by:

Code: Select all

(sm:advance "do_cond")


This makes sense for both

1. to have a selection of API funcs without direct access to non-API ones, or
2. forwarding all funcs, with having the working part of the code at FOOP's Class (for sharing).

Contexts with mixed-in working funcs doing all the work are no alternative to referenced FOOPs here.

Advantages and disadvantages of FOOP references
Advantages:

• context related: state change transfer of FOOP from callee to caller, without the need to share a var;
• FOOP related:
  • reuse of FOOP Class funcs, instead of using bloated mixed-in contexts;
    
  • polymorphism;
• both together: selected API funcs for direct access as part of FOOP's reference context, to have a separation of API and other code.

Disadvantages:

• FOOP references are polluting MAIN namespace;
• no automated garbage collection of FOOP reference contexts (but there may be a delete method - e.g. as part of FOOP Class, which has created them - iterating over all ref context instances easing this);
• call-by-ref has problems, which call-by-value does not have.

Conclusion
FOOP references may be used for shared FOOP objects with many and/or much code containing Class funcs, which is typically for 'heavy' objects not occuring in a big number (another usecase already in my mind is multiple loggers).

What do you think?

PS:
This post may be updated, because things may be seen in a different light later (still during writing this post usecase SM code has been improved...).

[ralph.ronnquist]

Impressive stuff. Too complex (for me) to say much about it without diving in and using it, which unfortunately would be slightly at odds with my current newlisp noosphere. Except, maybe, the random thought that with some huffing and puffing becomes the question of: why keep the state(s) of state machine(s) together with their process models? Or: who needs recursion?

Being a grumpy old engineer, I sound negative without even making an effort, and I probably should turn back to scanning #auspol tweets...

[hartrock]

Hello Ralph,
thanks for your feedback.

Except, maybe, the random thought that with some huffing and puffing becomes the question of: why keep the state(s) of state machine(s) together with their process models? Or: who needs recursion?

Note to recursion
There is a danger regarding recursion with respect to SMs (I've seen example code with just this error...). If there is no return from a method coupled to some state, but instead it calls the method of the next state directly or indirectly (e.g. by calling the SM which calls next method then), then this leads to a stack overflow after a while (at least, if this pattern will be used for all state transitions). Such kind of recursion has to be avoided.

A little bit more to usecase state machine
I've started to implement some web service without SM. But after a while the program flow has become more and more complicated, so introducing some states seemed to be appropriate. I've started with informational states: that is just to mark parts of nested conditions as a state. Thereafter this has become a loop around a cond just switching according current state (further using these states in an informational manner (good to log transitions)), and so to reduce the amount and depth of nested conditions. Here it has started to put these states together in an informational-only SM, to bundle states (good for modeling), check and track transitions (with statistics, how often some state has been reached).

Now I'm refactoring another part of the code with a fuller featured SM really having mappings of states to functions doing corresponding stuff. This is the second one, and compatible with the former. Because I've seen, that more than one SMs makes sense here, I've come to FOOP references: references to a few SMs, sharing their code as FOOPs.

From all above there is the important design goal, to support such refactoring of code from non-SM to SM code: simple things should stay simple, with more and more capabilities added as needed. During refactoring a mixture of SM and no-SM code should remain possible (and it is, as it is running now), to support an evolutionary path to more and more better structured SM code.

Current SM is a simple one compared with ones having enter/exit or more functions per state: currently it's just one (or none, if just informational) function per state, triggering its transition to the next one (which may also be triggered outside this mechanism like in above cond loop outside SM, where each part of the cond sets its next state). It grows with my usecase and during refactoring its code (have just started to introduce a SM_StateState Class, which bundles information belonging to a state during inside it (result of its func evaluation, reached by a func evalution from a previous state or triggered from the outside, etc.).
You see this is work in progress, and I'm going a pragmatical way with strong influence from praxis (the webservice provides a service used for testing other stuff, which is a very good test case while changing its code (and transforming it into SMs)).

I don't really understand your question:

why keep the state(s) of state machine(s) together with their process models?

(probably therefore I had to read something more about SMs); but hopefully my explanations help to understand, what I'm trying to reach (and what not).

PS: Please note [Update 2] above, describing how used simple SM differs from standard SMs.
PPS: Please also note [Update 3], describing that simple SM above just is not an SM, but an FM (to give it a better name)... Thanks to your question, which finally has led to this error correction!