So I posted a lot of Lisp here

OK, I know that most of you won't want to hack through pages of Lisp code and pages of Lisp rules. The post which is worth a look at is the example in use.

This shows the strength and weakness of the vanilla Eliza in action.

---

It's tempting, and even conventional, to scoff at Eliza. Peter Norvig says in Chapter 5.4 of his "Paradigms of Artificial Intelligence Programming":

" In the end, it is the technique that is important - not the program. ELIZA has been "explained away" and should rightfully be moved to the curio shelf. Pattern matching in general remains important technique, and we will see it again in subsequent chapters. The notion of a rule-based translator is also important. The problem of understanding English (and other languages) remains an important part of AI. Clearly, the problem of understanding English is not solved by ELIZA."

But the Eliza engine is a very good place to start.

---

What next?

The inadequacies of pseudo-human interlocutors are more forgivable in an animal. I'm going to craft a rule set for our dear, departed pet using the existing Eliza engine: Shadow v. 1.0.

Then it's back to Peter Norvig for his Prolog interpreter in Lisp (chapter 11). This will give me a unification algorithm and a resolution procedure.

My proposed architecture is to use the Eliza engine for input-output and have a Prolog-style knowledge-representation and inference-engine at the back.

In theory, the knowledge-based could be data-filled with rather tedious Q&A - the kind of thing which gives chatbots a bad name - but it's more pleasant for the user to have an initial registration procedure in which the user just tells the system about the domain of interest. For example, details of family and friends. Yes, I'm aware of privacy issues: you ask, in the age of Facebook?

The final area of architecture which I've been thinking about is the topic of conversation. This relates to the information and issues currently being talked about which drive the conversation forward. In Eliza it's basically in the hands of the user: there is no chatbot control mechanism.

But we can do better once we can do inference.

Watch this space.

Common Lisp code for the Eliza chatbot

#| This is the Peter Norvig (slightly adapted) code for the vanilla Eliza chatbot. For the rules see here. For an example of the program in action, see here.

This code should run if copied and pasted into a Common Lisp system, provided the rules file is also available and the 'Load' command below suitably amended - or just copy and paste the rules at the end of this file.

---
|#

;;; Eliza:  February 7th 2017 - February 10th 2017
;;;
;;; From Peter Norvig's book: Chapter 5
;;; "Paradigms of Artificial Intelligence Programming"
;;;
;;; Pattern matching, patterns and dialogue manager
;;;
;;; Posted:
; http://interweave-consulting.blogspot.co.uk/2017/02/common-lisp-code-for-eliza-chatbot.html
;;;
;;;  Reminder: (how to load and access files)
;;;
; (load  "C:\\Users\\HP Owner\\Google Drive\\Lisp\\Prog-Eliza\\Eliza.lisp")

(load  "C:\\Users\\HP Owner\\Google Drive\\Lisp\\Prog-Eliza\\Eliza-Rules.lisp")


;;; ---Patterns and input ---
;;;
;;; A pattern is a list of:-  segment-variable-lists, symbols or variables.
;;; An input is a list of symbols (without variables).
;;;
;;; Test pattern match in simple cases
;;;
; (setf *inp1* '(I need a vacation))
; (setf *pat1* '(I need a ?x))
; (pat-match *pat1* *inp1)
;
; ... when matched, should return a binding '((?X . vacation))
;
; (setf *inp2* '(Shadow and I need a holiday))
; (setf *pat2* '((?* ?U) need (?* ?V)) )
;
;;; --- Binding constants ---

(defconstant +fail+ nil "Indicates pat-match failure.")

(defconstant +no-bindings+ '((t . t))
"Indicates pat-match success but with no variables.")

(defconstant +example-bindings+
   '((?X . (the cat)) (?Y . (sat on)) (?Z the mat))  )

;;; --- BINDINGS a list of (var . value) dotted pairs ---
;;;
; a list of the form (?* ?variable-name) denotes a segment variable.
; Eg:  (pat-match '((?* ?p) need (?* ?X)) '(Mr Hulot and I need a vacation))
; gives ((?P MR HULOT AND I) (?X A VACATION)) .. as the (dotted pair) binding.

(defun get-binding (var bindings)
  "Find a (variable . value) pair in a binding list."
  (assoc var bindings))

(defun binding-val (binding)
  "Get the value part of a single binding."
  (cdr binding))

(defun lookup (var bindings)
  "Get the value part (for var) from a binding list."
  (binding-val (get-binding var bindings) ))

(defun extend-bindings (var val bindings)
  "Add a (var . value) pair to a binding list."
  (acons var val bindings))

(defun variable-p (x)    ; SYMBOL -> Bool
  "Is x a variable, a symbol beginning with '?' "
  (and (symbolp x) (equal (char (symbol-name x) 0) #\? )))
 
(defun starts-with (list x)  ; SYMBOL-list x SYMBOL -> Bool
  "Is this a non-empty list whose first element is x?"
  (and (consp list) (eql (car list) x)) )
 
;;; --- Pattern Matcher ---

; pat-match: PATTERN x INPUT x BINDINGS -> BINDINGS

(defun pat-match (pattern input &optional (bindings +no-bindings+))
  "Match pattern against input in the context of the bindings"
  (cond ((eq bindings +fail+ ) +fail+)
        ((variable-p pattern ) (match-variable pattern input bindings))
        ((eql pattern input) bindings)
        ((segment-pattern-p pattern) (segment-match pattern input bindings))
        ((and (consp pattern) (consp input))
                 (pat-match (rest pattern) (rest input)
                         (pat-match (first pattern) (first input) bindings)))
        (t   +fail+) ))
;
; segment-pattern-p: PATTERN -> bool   (if first element of PATTERN is seg-var

(defun segment-pattern-p (pattern)
  "pattern a non-empty-list, 1st element a segment-matching-pattern: ((?* var) . pat)"
  (and (consp pattern)
       (starts-with (first pattern) '?* )))

(defun match-variable (var input bindings) ; VAR x SYMBOL(-list) x BINDINGS -> BINDINGS
  "Does VAR match input? Uses (or updates) and returns bindings."
  (let ((var+val (get-binding var bindings)))
    (cond ((not var+val) (extend-bindings var input bindings))  ; add new binding
          ((equal (cdr var+val) input) bindings)                             ; do nothing
          (t +fail+ )                                                                          ; clash with existing
   )) )

; (setf b +EXAMPLE-BINDINGS+)
; (match-variable '?X '(the cat) b)
; (match-variable '?U 'dog  b)
; (match-variable '?X 'dog  b)     ; returns NIL

; segment-match: PATTERN x INPUT x BINDINGS x NAT -> BINDINGS

(defun segment-match (pattern input bindings &optional (start1 0))
"Match the pattern = ((?* var) . pat) against input - returns BINDINGS"
  (let ((var (second (first pattern)) )     ; we know ?* is the first
        (pat (rest pattern )) )
     (cond ((null pat) (match-variable var input bindings))   ; return this binding

           ;; We assume that pat, the rest of pattern, starts with a constant
           ;; In other words, a pattern can't have 2 consecutive vars
           ;; pos is index into input, the symbol matching next item in pattern
           (t (let ((pos (position (car pat) input :start start1 :test #'equalp)))
                   (if (null pos)
                         +fail+                          ; pattern-input mismatch
                         (let* ((input-rest (subseq input pos))
                                (input-prefix (subseq input 0 pos)) ; bind to var!
                                (b1 (match-variable var input-prefix bindings)) ;OK
                                (b2 (pat-match pat input-rest b1)))
                           ;; If b2 failed , try another longer one
                             (if (eq b2 +fail+)
                                  (segment-match pattern input bindings (+ pos 1) )
                                  b2) ) ) ) ) ) ) )

; (setf pattern '((?* ?W) sat on the ?V))
; (setf input '(The cat sat on sat on the mat))
; (segment-match pattern input +no-bindings+)

#| Note on segment-match (from Norvig, chapter 5.3)

In writing segment-match, the important question is how much of the input the
segment variable should match. One answer is to look at the next element of the
pattern (the one after the segment variable) and see at what position it occurs in the
input. If it doesn't occur, the total pattern can never match, and we should fail.

If it does occur, call its position pos. We will want to match the variable against the
initial part of the input, up to pos. But first we have to see if the rest of the pattern
matches the rest of the input. This is done by a recursive call to pat-match. Let the
result of this recursive call be named b2. If b2 succeeds, then we go ahead and match
the segment variable against the initial subsequence.

The tricky part is when b2 fails. We don't want to give up completely, because
it may be that if the segment variable matched a longer subsequence of the input,
then the rest of the pattern would match the rest of the input. So what we want is to
try segment-match again, but forcing it to consider a longer match for the variable.

This is done by introducing an optional parameter, start1, which is initially 0 and is
increased with each failure. Notice that this policy rules out the possibility of any
kind of variable following a segment variable.
|#

;;; --- test data for pattern matching ---

; (pat-match '(i need a ?X) '(i really need a vacation)) ; NIL
; (pat-match '(this is easy) '(this is easy))            ; ((T . T))
; (pat-match '(?X is ?X) '((2 + 2) is 4))                ; NIL
; (pat-match '(?X is ?X) '((2 + 2) is (2 + 2 )))       ; ((?X 2 + 2)) = ((?X . (2 + 2)))
; (pat-match '(?P need . ?X) '(i need a long vacation)) ;((?X A LONG VACATION) (?P . I))

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;
;;; --- RULES ---
;;;
;;; Here's an example of a rule: input-matching-pattern + one or more response-patterns
;;;
;;; (  ((?* ?X) I want (?* ?Y ))                  ; Pattern part
;;;       (What would it mean if you got ?Y)      ; Three possible responses
;;;       (Why do you want ?Y)
;;;       (Suppose you got ?Y soon?)   )
;;;
;;; -----------------------------------------------------------------------------------
;;; --- ELIZA top level ---
;;;
;;; Note on output - recall (terpri) prints a newline.
;;; print is just like prin1 (or princ - human readable)
;;; except that the printed representation of object
;;; is preceded by a newline and followed by a space.
;;;
;;; pprint is just like print except that the trailing space is omitted
;;; and object is printed with the *print-pretty* flag non-nil - pretty output.

(defun eliza (rules)
  "Respond to user input using pattern matching rules. 'bye' to terminate"
  (princ "Hello, I am your doctor. Please type bye to finish.")
  (terpri)
  (terpri)
  (princ "Please type your first name then return> ")
  (setf *user-name* (car (read-line-no-punct)))              ; substitutes $name$ in response
  (loop
   (print 'speak>)
   (let ((input (read-line-no-punct)))
      (if (equalp input (list 'bye)) (progn (print 'Goodbye) (return))
          (let* ((raw-response (use-eliza-rules input rules))
                 (response (flatten1 raw-response))
                 (out  (substitute *user-name* '$name$ response :test #'equalp)) )
            (pprint out) ) ) ) )  )

; Test:  (eliza +eliza-rules+)
; Test:  (eliza +eliza-rules-extended+)

(defun read-line-no-punct ()
  "Read an input line terminated by 'return', ignore punctuation, -> symbol-list"
  (read-from-string
   (concatenate 'string "(" (substitute-if #\space #'punctuation-p (read-line))
                ")")  ))

(defun punctuation-p (char) (find char ".,;:'!?#-()\\\""))

(defun use-eliza-rules (input rules)
  "Find some rule with which to transform the input ... ."
  (some #'(lambda (rule)
            (let ((bindings (pat-match (rule-pattern rule) input)))
              (if (not (eql bindings +fail+) )
                  (sublis (switch-viewpoint bindings)
                          (random-elt (rule-responses rule )) ))))
        rules))

(defun rule-pattern (rule) (car rule))     ; The pattern to match the input

(defun rule-responses (rule) (cdr rule))   ; the list of (remaining) response-patterns

(defun switch-viewpoint (bindings)
  "Change I to you and vice versa, and so on in bindings."
  (sublis '((I . you)  (you . I)    (me . you)
            (am . are) (was . were) (are . am)) bindings ))

(defun flatten1 (l)
  "Append together elements (or lists) in the list l (not lists of lists)."
  (mappend #'mklist  l) )

(defun mklist (x)
  "Return x if it is a list, otherwise (x)."
  (if (listp x) x (list x)))

(defun mappend (fn l)   ; (* -> **) x *-list -> <flat list>
  "Apply fn to each element of list l and append the results."
  (apply #'append (mapcar fn l) ))

(defun random-elt (choices)
  "Choose an element from a list at random."
  (elt choices (random (length choices )) ))


;;; --- End ---

The 'Doctor' rule-set for Eliza

#|  These are the rules which drive the 'Doctor' version of Eliza. For the Eliza code, see here.

This entire post should be executable if copied and pasted into Common Lisp.

For an example of the rules in use, see here.  
|#
;;; Eliza Rules:  February 7th 2017 - Feb 10th 2017
;;;
;;;  From Peter Norvig's book: Chapter 5
;;; "Paradigms of Artificial Intelligence Programming"
;;;
;;; Pattern matching, patterns and dialogue manager
;;;
;;; Posted: http://interweave-consulting.blogspot.co.uk/2017/02/the-doctor-rule-set-for-eliza.html
;;;
;;;  Reminder: (how to load and access files)
;;;
; (load  "C:\\Users\\HP Owner\\Google Drive\\Lisp\\Prog-Eliza\\Eliza-Rules.lisp")

;;; --- SHORT LIST OF ELIZA RULES ---
;;;
;;;  Note that the symbol $name$ will be substituted by the user's first name
;;;  in the Eliza top-level procedure.

(defparameter +eliza-rules+
  '(
    (((?* ?x) hello (?* ?y))
     (Hi $name$ - please state your problem))

    (((?* ?x) I want (?* ?y))
     (What would it mean if you got ?y)
     (Why do you want ?y)
     (Suppose you got ?y soon))

    (((?* ?x) if (?* ?y))
     (Do you really think its likely that ?y $name$)
     (Do you wish that ?y)
     (What do you think about ?y)
     (Really - if ?y))

    (((?* ?x) no (?* ?y))
     (Why not?)
     ($name$ - you are being a bit negative)
     (Are you saying "NO" just to be negative?))

    (((?* ?x) I was (?* ?y))
     (Were you really?)
     (Perhaps I already knew you were ?y)
     (Why do you tell me you were ?y now?))

    (((?* ?x) I feel (?* ?y))
     (Do you often feel ?y ?))

    (((?* ?x) I felt (?* ?y))
     (What other feelings do you have?))))

;;; --- EXTENDED RULES (c. ~50)  ---

(defparameter +eliza-rules-extended+
'(
(((?* ?x) hello (?* ?y))
 (Hi $name$ - please state your problem.))

(((?* ?x) hi (?* ?y))
 (Hi $name$ - please state your problem.))

(((?* ?x) computer (?* ?y))
 (Do computers worry you?)
 (What do you think about machines?)
 (Why do you mention computers?)
 (What do you think machines have to do with your problem?))

(((?* ?x)  my name (?* ?y))
 ($name$ - I am smarter than you think!))

(((?* ?x) your name (?* ?y))
 (I prefer to be an anonymous doctor $name$))

(((?* ?x) sorry (?* ?y))
 (Please don't apologize)
 (Apologies are not necessary)
 (What feelings do you have when you apologize?))

(((?* ?x) I remember (?* ?y))
 (Do you often think of ?y)
 (Does thinking of ?y bring anything else to mind?)
 (What else do you remember?)
 (Why do you recall ?y  right now?)
 (What in the present situation reminds you of ?y)
 (What is the connection between me and ?y))

(((?* ?x) do you remember (?* ?y))
 (Did you think I would forget ?y ?)
 (Why do you think I should recall ?y now)
 (What about ?y)
 (You mentioned ?y))

(((?* ?x) if (?* ?y))
 (Do you really think its likely that ?y)
 (Do you wish that ?y)
 (What do you think about ?y)
 (Really -- if ?y))

(((?* ?x) I dreamt (?* ?y))
 (Really - ?y)
 (Have you ever fantasized ?y while you were awake?)
 (Have you dreamt ?y before?))

(((?* ?x) I dreamed (?* ?y))
 (Really - ?y)
 (Have you ever fantasized ?y while you were awake?)
 (Have you dreamed ?y before?))

(((?* ?x) dream about (?* ?y))
 (How do you feel about ?y in reality?))

(((?* ?x) dream (?* ?y))
 (What does this dream suggest to you?)
 (Do you dream often?)
 (What persons appear in your dreams?)
 (Dont you believe that dream has to do with your problem?))

(((?* ?x) my mother (?* ?y))
 (Who else in your family ?y)
 (Tell me more about your family))

(((?* ?x) my father (?* ?y))
 (Your father?)
 (Does he influence you strongly?)
 (What else comes to mind when you think of your father?))

(((?* ?x) I want (?* ?y))
 (What would it mean to you if you got ?y)
 (Why do you want ?y)
 (Suppose you got ?y soon))

(((?* ?x) I am glad (?* ?y))
 (How have I helped you to be ?y)
 (What makes you happy just now)
 (Can you explain why you are happy that ?y))

(((?* ?x) I am sad (?* ?y))
 (I am sorry to hear you are depressed)
 (I'm sure its not pleasant to be sad))

(((?* ?x) are like (?* ?y))
 (What resemblance do you see between ?x and ?y ?))

(((?* ?x) is like (?* ?y))
 (In what way  is  it that ?x is like ?y)
 (What resemblance do you see?)
 (Could there really be some connection?)
 (How?))

(((?* ?x) alike (?* ?y))
 (In what way?)
 (What similarities are there?))

(((?* ?x) same (?* ?y))
 (What other connections do you see?))

(((?* ?x) I was (?* ?y))
 (Were you really?)
 (Perhaps I already knew you were ?y)
 (Do you think its relevant that you were ?y now?))

(((?* ?x) was I (?* ?y))
 (What if you were ?y ?)
 (Do you think you were ?y)
 (What would it mean if you were ?y))

(((?* ?x) I am (?* ?y))
 (In what way are you ?y)
 (Do you want to be ?y ?))

(((?* ?x) am I (?* ?y))
 (Do you believe you are ?y)
 (Would you want to be ?y)
 (You wish I would confirm you are ?y)
 (What would it mean if you were ?y))

(((?* ?x) am (?* ?y))
(Why do you say "AM?")
 (I dont really get that) )

(((?* ?x) are you (?* ?y))
 (Why are you interested in whether I am ?y or not?)
 (Would you prefer if I werent ?y)
 (Perhaps I am ?y in one of your fantasies?))

(((?* ?x) you are (?* ?y) )
 (Perhaps I am ?y only in your imagination!)
 (What makes you think I am ?y ?))

(((?* ?x) because (?* ?y))
 (Is ?y the real reason?)
 (What other reasons might there be?)
 (Does ?y seem to explain anything else?))

(((?* ?x) were you (?* ?y))
 (Perhaps I was ?y)
 (What do you think?)
 (In some places being ?y is barely legal!)
 (What if I had been ?y))

(((?* ?x) I cant (?* ?y))
 (Maybe you could ?y now)
 (What if you could ?y ?))

(((?* ?x) I miss (?* ?y))
 (Has ?y been gone long?)
 ($name$ - I can feel your sadness about ?y) )

(((?* ?x) I feel (?* ?y))
 (Do you often feel ?y ?))

(((?* ?x) I felt (?* ?y))
 (What other feelings do you have?))

(((?* ?x) I (?* ?y) you (?* ?z))
 (Perhaps in your fantasy we ?y each other))

(((?* ?x) why dont you (?* ?y))
 (Should you ?y yourself?)
 (Do you believe I dont ?y)
 (Perhaps I will ?y in good time))

(((?* ?x) yes (?* ?y))
 (You seem quite positive - please expand on this)
 ($name$ - I feel we should move on from this to another subject)
 (I understand $name$ but you need to move on to a new topic))

(((?* ?x) no (?* ?y))
 (Why not $name$ ?)
 (You are being a bit negative $name$)
 (Are you saying "NO" just to be negative?))

(((?* ?x) someone (?* ?y))
 (Can you be more specific about - ?y))

(((?* ?x) everyone (?* ?y))
 (Surely not everyone ?y)
 (Can you think of anyone in particular who ?y)
 (Who for example?)
 (You are thinking of a special person?))

(((?* ?x) always (?* ?y))
 (Can you think of a specific example?)
 (When?)
 (What incident are you thinking of?)
 (Really  - always?))

(((?* ?x) what (?* ?y))
 (Why do you ask?)
 (Does that question interest you?)
 (What is it you really want to know?)
 (What do you think?)
 (What comes to your mind when you ask that?))

(((?* ?x) why (?* ?y))
 (Its a good question - why ?y)
 (?x many things but its never that clear)
 (Well $name$ you are really in to deep questions) )

(((?* ?x) perhaps (?* ?y))
 (You do not seem quite certain.))

(((?* ?x) are (?* ?y))
 (Did you think they might not be ?y)
 (Possibly they are ?y))

(((?* ?x) that I (?* ?y))
 (Are you happy or sad that you ?y)
 (Possibly you should ?y))

(((?* ?x))
 (Very interesting)
 (I am not sure I understand you fully)
 (What does that suggest to you?)
 (Please continue)
 (Go on)
 (Do you feel strongly about discussing such things?))
))

So this is where we are .. Doctor

CL-USER 1 > (eliza +eliza-rules-extended+)
Hello, I am your doctor. Please type bye to finish.

Please type your first name then return>  Clare

SPEAK> Hi there.

(HI CLARE - PLEASE STATE YOUR PROBLEM.)
SPEAK> I think a lot about the cat.

(DO YOU FEEL  STRONGLY ABOUT DISCUSSING SUCH THINGS?)
SPEAK> Yes, I dream about Shadow.

(HOW DO YOU FEEL ABOUT SHADOW IN REALITY?)
SPEAK> I dreamed he was playing in the garden.

(HAVE YOU DREAMED HE WAS PLAYING IN THE GARDEN BEFORE?)
SPEAK> I seem to see him there a lot.

(DO YOU FEEL  STRONGLY ABOUT DISCUSSING SUCH THINGS?)
SPEAK> I am not sure you are that interested.

(IN WHAT WAY ARE YOU NOT SURE I AM THAT INTERESTED)
SPEAK> You are not helpful.

(WHAT MAKES YOU THINK I AM NOT HELPFUL ?)
SPEAK> I'm going to report you. What's your name?

(I PREFER TO BE AN ANONYMOUS DOCTOR CLARE)
SPEAK> I am not surprised.

(IN WHAT WAY ARE YOU NOT SURPRISED)
SPEAK> Bye

---

Vanilla Eliza in action using the extended rule-set (about 50 rules). The conversation would flow a lot more smoothly if I kept adding rules to capture the many, many diverse possibilities for input. But in the end, it's all reflecting back what the user said.

If you look at the rules, you will see that Eliza (the generic program - it's specific rules which create an actor like Doctor) is snatching at a couple of key words in the user's input, and is basically praying that the rest will make sense when bounced back. It's like when you speak a foreign language really badly.

To do any better will take knowledge and inference (to come).

---

A 'Doctor' rule-set. A basic set of rules for test purposes, and an extended set.

The Eliza code in Common Lisp.

I don't promise bug-free, but it mostly works OK.

Diary: weights + Lisp reader issues + trim & style

Consider a hulking biker, tattoos and leotard, bench-pressing some massive barbell.

Alongside him, place that irritatingly-cheerful hunk Dr Chris van Tulleken, raising and lowering some wimpy baby dumbbells.

Pretending to be exhausted. Not this stagey picture below.

It was all in aid of this study (in the latest series of "Trust Me, I'm a Doctor").

"The study split 49 weight trainers into two groups and started them on a 12-week weight training programme. For each participant, they calculated their ‘one-repetition maximum’ or 1RM – that’s the heaviest weight they can lift.

"They then split the study into two groups, one group lifting 30-50% of their 1RM and the other group lifting 75-90%. The key thing was that each group lifted their weights to ‘volitional failure’ – in other words, they lifted until they couldn’t lift any more.

‘Failure’ will happen to anyone and everyone, however strong, if they do enough repetitions. So the group lifting the lighter weights did a larger number of reps (20-25) than the group lifting the heavier weight (8-12).

"The theory behind muscle failure is to do with ‘motor units.’ Motor units are bundles of muscle fibres controlled by a nerve. When you lift a weight, motor units will be required to contract the muscle. With each lift, some motor units will get fatigued, so additional motor units need to be used to do the next lift. Sooner or later you get to a point where all your available motor units have been exhausted – that’s what causes your muscles to fail.

"In the McMaster study, the results showed that despite lifting different weights, both groups showed the same increase in strength and muscle growth. In other words, doing heavy weights with fewer reps or lighter weights with more reps made no difference. These results agreed with earlier research conducted by the same group.

"So what does that mean for the rest of us? Well, it means that you can get results lifting heavy weights OR lighter weights, so long as you’re pushing your muscles to work harder than they normally do. You don’t always need to lift to failure to get results – but your muscles need to be ‘overloaded’ compared to your normal day to day life. Strength and conditioning coach Richard Blagrove from St. Mary’s University, Twickenham, suggests that on a scale of 1 to 10, where 10 is repetition failure, lifting to 7 or 8 is about right.

"If your muscles are feeling that overload once a week, your body will adapt and get stronger. If you want to continue to get stronger, you will need to constantly re-assess and progress your weight or rep level, to make sure you are always pushing your muscles beyond their comfort zone. If your weight training feels easy, it probably isn’t doing anything for you.

"You can get results using weights machines or free weights. Free weights also force you to use stabiliser muscles meaning you use more energy, and your joints move in their most natural way. But it’s important to lift correctly, so as soon as your “form” starts to go, you should probably stop. If you DO want to push yourself to failure, weight machines might be the safest place to do it. If you lift to failure with free-weights you need a partner who can relieve you of the weight when you can no longer lift it."

In a nutshell, don't go with "go heavy or go home!".

Do enough reps so that the last three are 'difficult'. Obviously you need enough weight so that you can get there within, say, 8-15 reps.

---

While I was watching our recording of "Trust Me, ..." my subconscious was whirring furiously. This afternoon I had painstakingly edited 43 rules for my Eliza program, which I'm reconstructing. The program works! But the rule set:

(defparameter +eliza-rules-extended+  '(.. stuff .. ))

refused to compile: weird and incomprehensible error messages complaining of 'invalid traits'. We're meant to be AI-advancing daily, but our development environments haven't the first clue.

My epiphany was that a rule which said something like:

(Why don't you like your mother?)

is going to hit a major problem with that quote character.

I rushed upstairs and deleted all the quotes from don't, doesn't, it's .. and everything loaded properly. Tomorrow I'll test it thoroughly and maybe post the code.

That should thrill you, dear reader.

---

As you know, my first target emulation is going to be the cat. Clare, who had to suffer through a tedious explanation of my afternoon difficulties, suggested that I should be designing a chatbot for the elderly. I'm to data-fill it with small talk about family, friends and neighbours (and pets?).

I suspect there's a market for that, and I'm not completely convinced that a state-of-the-art artificial neural net is required to implement it either.

But here to the contrary is Google's view:

"A simple strategy to build lightweight conversational models might be to create a small dictionary of common rules (input → reply mappings) on the device and use a naive look-up strategy at inference time.

This can work for simple prediction tasks involving a small set of classes using a handful of features (such as binary sentiment classification from text, e.g. “I love this movie” conveys a positive sentiment whereas the sentence “The acting was horrible” is negative).

But, it does not scale to complex natural language tasks involving rich vocabularies and the wide language variability observed in chat messages."

---

The diary would not be complete without observing that Clare had her hair done today.

I did notice.

The return of (virtual) Shadow

The late Shadow

Chapter 5 of Peter Norvig's excellent "Paradigms of Artificial Intelligence Programming: Case Studies in Common Lisp" is entitled "ELIZA: Dialog with a Machine". I have spent the afternoon studying his pattern-matching code and transferring it into an executable Lisp file.*

An example of an Eliza pattern is this:

'((?* ?P) need (?* ?X))

which picks up text before and after the word 'need'.

If you tell Eliza:

'(Shadow and I need a vacation),

the program will match the input with the pattern above, thus ..

(pat-match '((?* ?P) need (?* ?X)) '(Shadow and I need a vacation))

to create a binding

'((?P SHADOW AND I) (?X A VACATION))       ;   (dotted pair) binding

which can then be passed to the output-generating pattern. And so on.

---

But we're not so much interested in Eliza and that Rogerian psychoanalyst. We want Shadow, our much-missed departed cat (RIP July 1st 2016).

The first step is to change the rules: less talk of vacations and how you're feeling about your parents, more about voles. And being sick on the carpet.

The Eliza architecture was criticised, correctly, for being totally vacuous. The emptiness of the 'conversation' led to bored withdrawal after a period depending upon the narcissism of the user.

We can do better.

To add intelligence to Shadow (such a bright cat!) we have to give him knowledge (in the knowledge lies the power) together with reasoning capability.

Peter Norvig helpfully gives us an equation (chapter 16, p. 548 or thereabouts):

Expert System = Prolog + uncertainty + caching + questions + explanations

The Prolog part powers the knowledge-base capturing Shadow's deep understanding of his own likes, light-hearted escapades and more lethal habits; the Q&A allows for his more sensitive, introspective side.

Peter Norvig has a detailed chapter (11 - Logic Programming) where he explains how to implement Prolog in Lisp (interpreted).

I think this is the way to go with chatbots: combine Eliza-style interaction with a little knowledge-based reasoning to add interest and depth, and to steer the conversation. Perhaps a semantic grammar in there somewhere to help extract meaning from the user's input, though personally, given the ungrammaticality of dialogue, I've always put semantics/pragmatics first.

Anyway, watch this space. It won't be quick .. but we will bring him back!

---

* We spent the morning strolling in the sun around Cheddar reservoir.

"No .. human has touched the edge of the truth of Go"

A minority interest of mine is the game of Go. I hope to study it .. at some point.

“After humanity spent thousands of years improving our tactics, computers tell us that humans are completely wrong,” Mr. Ke, 19, wrote on Chinese social media platform Weibo after his defeat. “I would go as far as to say not a single human has touched the edge of the truth of Go.”

From this unfailingly interesting article (via Marginal Revolution).

---

A lot of people have picked up on this, or related issues:

- Scott Alexander writes about a recent stellar AI conference
- Steve Hsu tells our masters about the future of AI and genetic engineering.

A battlespace AI

When contemplating AI-controlled cruise missiles, as in yesterday's post, there is a tendency to hear 'AI' and think magic, or imagine we're in the foothills of the Butlerian Jihad.

Still, even without access to classified information, there's a lot we can say about this kind of battlespace AI, just by comparing it with stuff we already know about.

An anti-carrier cruise missile is basically a suicide drone. Its mission is threefold:

• navigate to the target
• identify the best choice of target to crash into (and blow up)
• cope with an extraordinarily hostile environment.

All three of these mission priorities are amenable to current artificial neural net technology:

• navigation can leverage autonomous vehicle/reconnaissance-drone tech
• target identification is not dissimilar to existing object/facial recognition tasks
• the hostile environment can be addressed by massive simulation-training.

I imagine AI prototypes are probably flying drones around right now in simulated attack-scenarios. Steve Hsu has more information on this.

---

There are three interesting issues which arise from the specifics of a combat environment.

1. Target identification/selection

Unlike relatively benign civilian environments, enemy carriers and other ships will attempt to make location and targeting difficult. There will be smoke, perhaps battle damage, explosions and defensive measures such as laser dazzle, jamming, false targets and chaff.

The solution appears to involve multiple sensing platforms illuminating or imaging the target space: satellites; aircraft/drone loitering radars; multiple attack-weapons sharing sensor data on a local net.

Sensor data fusion is a complex but well-researched topic amenable to AI (satirised here).

2. The hostile environment

The incoming missiles will be targeted by the carrier group with everything they have. Antimissiles, guns, lasers. Who can imagine an optimal set of tactics for surviving such an assault?

An AI system which has trained on millions of simulations.

It's somehow similar to AlphaGo.

What we know of AI adversaries is that they often exhibit brilliant but quite counter-intuitive behaviour. That's mostly a plus in the last few kilometres.

3. Autonomy

There's a stupid point here, and an intelligent one.

The stupid argument demands that AI weapons systems have 'no autonomy' - that there will always be a human in the loop.

So ... like with mines, then?

Plainly, if the carrier has already been sunk on cruise missile arrival, the AI will make a fast call on the optimal secondary target. There will be no human in the loop - in fact real-time communications will undoubtedly be 'very difficult'.

However, on a larger scale of strategic autonomy we do need to worry about the unpredictability and lack of transparency of current neural net technology. If, for some obscure tactical reason, an AI weapon concludes that it needs to attack a friendly vessel, then - absent a superhuman common sense (right!) - we should be worrying.

An emergent research area is the design of human-machine interfaces which have explicit, communicable and actionable knowledge about the operations of their powerful but opaque neural net subsystems.

Humans have one of those too: it's called consciousness.

Revealed preferences

Chinese hypersonic glide vehicle DF-ZF

From the New York Times.

"In August, the state-run China Daily reported that the country had embarked on the development of a cruise missile system with a “high level” of artificial intelligence. The new system appears to be a response to a missile the United States Navy is expected to deploy in 2018 to counter growing Chinese military influence in the Pacific. ...

"The new Chinese weapon typifies a strategy known as “remote warfare,” said John Arquilla, a military strategist at the Naval Post Graduate School in Monterey, Calif. The idea is to build large fleets of small ships that deploy missiles, to attack an enemy with larger ships, like aircraft carriers."

Ultrafast AI-piloted cruise missiles, fabricated at low marginal cost, are pretty much guaranteed to take out an aircraft carrier.

The fact that the US and UK are still investing in these behemoths indicates that they see the next decades of force-projection as targeting third world states, not Russia or China.

Let's hope they're right.

---

The AGM-158C LRASM is the US anti-ship weapon mentioned.

---

What kind of AI could pilot a combat cruise missile?

"The Volatility Smile" - Derman and Miller

Amazon link

When I worked at Nortel  I occasionally got stock options.  Suppose Nortel shares were trading at $100 per share. I would be given the option of buying, say, 50 shares in six months time at a strike price of $80 per share.

Suppose Nortel stock went up in those six months to $120 per share. Those 50 shares would sell at $6,000. But I could buy them at $80, costing me $4,000.

By buying at a discount and then immediately selling, I would realise a profit of $2,000. I guess they thought I would thereby be totally incentivized to work tirelessly for stock appreciation.

As the Internet boom faded, I was seldom in the money. Nortel’s shares were under water, and my options were worthless. So at the time of issue, how should they have been priced?

Plainly the closer the option expiry date to the option trade date, the less the share price uncertainty - which lowers the option price (often called the premium).  However, if the shares are more volatile there is more chance that they will soar above the strike price - that’s got to raise the option price.

The Black-Scholes equation is a partial differential equation which describes the rate of change of option price over time as a function of stock price. The stock price is assumed to be varying as a random walk around its trend with some volatility. The equation can be solved to give option prices, similar to the call option example I discussed above.

Black-Scholes has just one unobservable parameter, the stock volatility. Other parameters in the model, the time to maturity, the strike price, the risk-free interest rate, and the current underlying stock price are all observable. In principle an option's theoretical value is a monotonically increasing function of implicit volatility.

The Black-Scholes model implies that the stock price volatility is flat compared with the strike price. This is not empirically true. When running Black-Scholes in reverse, computing the implicit volatility from observed market rates for options (and using the other observable parameters), equities tend to have skewed curves: compared to at-the-money, implied volatility is substantially higher for low strikes, and slightly lower for high strikes. Commodities often have the reverse behaviour to equities, with higher implied volatility for higher strikes. This departure from linearity, when graphed, is termed the volatility smile.

Naturally it is possible – at the expense of additional complexity – to factor in these non-linearities. And so we come to Derman’s and Miller’s book, “The Volatility Smile”. Aimed at practitioners who have already absorbed the standard Black-Scholes approach, this treatment looks in detail at several advanced models (local volatility, stochastic diffusion, jump-diffusion) which aim to provide a better match to real-life behaviour.

Presenting itself as a mathematical textbook, albeit informed and motivated by market realities, the precondition for getting the best from this work is plainly a postgraduate qualification in mathematical finance. The book is really for working quants. Those with the right background will, however, find the presentation both relevant and lucid.