Showing posts with label Eliza. Show all posts
Showing posts with label Eliza. Show all posts

Sunday, June 24, 2018

I talk to my car

Sometimes my car talks to me ..

Would you entrust your life to a car you couldn't talk to? I know most people would rather die than converse with their chauffeurs but how many professional drivers can discuss with equal ease philosophy, psychology and computer science (according to the module description)?

I say “They tell me you're like the human unconscious. Like me driving with my mind elsewhere. You're 'classify then do’ - with thousands of edge cases.”

The car says, “How long have you had a problem with edge cases?”

I say “When I drive, I can go for long periods daydreaming, on automatic. But then some tricky situation arises and there's a need to negotiate. That's when driving gets my full attention.”

The car says “Tell me more about driving.”

I say “The whole Skinner thing, the behaviourist black box, the input-output rules, the famous denial of intentionality. Dennett nailed it all, didn't he?”

The car says, “Tell me more about Dennett.”

“The thing is,” I say, “Without the intentional stance there can be no such thing as empathy. But who wants to be driven by a psychopathic car?”

What do you mean by empathy?” the car asks.

---

© Nigel Seel. June 2018.

---

Back to Stories.

Wednesday, March 14, 2018

A beta-level simulation

Amazon link

I'm just finishing "Elysium Fire" by Alastair Reynolds, his follow-up to "Aurora Rising: Previously published as The Prefect". It's a perfectly serviceable SF techno-thriller: the action moves along briskly enough.

I am most intrigued, though, by the concepts of alpha- and beta-level simulations.
"An alpha-level simulation was a digitised version of a scanned human mind, perfectly replicating its brain structure. It was fully capable of remembering past experiences, learning, adapting, and was fully self-aware.

Later technologies allowed individuals to survive the scanning process, leaving essentially two copies of the same person, one physical and one digital. ...

---

Beta-level simulations were sophisticated computer programs designed to mimic a person in appearance, mannerisms, and thought-patterns. While capable of successfully impersonating a human mind down to their most minute idiosyncrasies, they were not in fact self-aware -- they were just near-perfect imitations of life. As such, they enjoyed no legal rights or protections.

In Yellowstone society, and presumably other Demarchies, it was considered a most egregious social faux pas to allow someone to believe a beta-level simulation was in fact alpha-level."
A continuing theme is that after someone is murdered, they may well have had a shadowing beta-level simulation which the authorities then sequester for interrogation - a chance to interview the deceased victim, if you like. It's quite a shock when they're told they are dead.

The beta-levels generally don't know they're simulations and the hero, Inspector Dreyfus, is conflicted as to whether he should adopt the instrumental view ("I'm interviewing an expert system") or the intentional view ("I'm interviewing a digitalised person"). For the reader, this dilemma is never truly resolved.*

It does give me itchy fingers. I want to go back to my Lisp environment, to my 'Eliza' system and my automated theorem-prover. I'm sure I could knock up a fairly awesome beta-level myself, if we were talking, say, of a cat .. .

---

* I suppose a remote descendant of Replika will give us all beta-level simulations.

Monday, March 20, 2017

AI code in Lisp: new resource here

New resource on the sidebar to your right. This should please readers wondering why vast screeds of Lisp code randomly appear here, interrupting more erudite essays on this & that.

Except as I write this, 134 of you have visited: Description: Theorem Prover in Lisp.

Here's what the READ ME at the new sidebar says.

READ ME
=======

These Common Lisp files contain AI programs which are organised around the theme of building a chatbot.

They will all run independently and were developed in LispWorks free Personal Edition.

You can use the code as you like. It's not supported and there will certainly be bugs I haven't spotted.

I think of the code as a toolkit, there to be modified.

---

To come:

1. Upgrades to the resolution theorem-prover improving the display of proofs + any bug fixes.

2. An AI planner, oriented both towards a toy, virtual, physical world and speech acts for conversation planning.

3. A design for 'internal emotional states' to create some 'point' for the chatbot's autonomous behaviour; we need something more interesting than a natural language interface to Wikipedia-style queries.

Plus integration of all the above.

Thursday, March 16, 2017

Description: Theorem Prover in Lisp

[Update: RTP release 1.1 is now available.]
---

As I mentioned, I've now completed basic testing of my (very, very simple) resolution theorem prover in Common Lisp: the code is ready to load. But read the stuff below first.

1. How to run the system

Once you have the code in front of you, go to Part 2: clauses and extended clauses where you will find the test-data definitions as follows:
(defvar *c1* (mk-fact-clause '(member ?item (?item . ?rest ))))
(defvar *c2* (mk-rule-clause '(member ?item (? . ?rest)) '((member ?item ?rest))))
(defvar *c3* (mk-fact-clause '(likes rod horvath)))
(defvar *c4* (mk-fact-clause '(likes sally renner)))
(defvar *c5* (mk-fact-clause '(likes sally rod)))
(defvar *c6* (mk-fact-clause '(likes hardy renner)))
(defvar *c7* (mk-fact-clause '(likes hardy rod)))
(defvar *c8* (mk-fact-clause '(amusing hardy)))
(defvar *c9* (mk-fact-clause '(likes horvath moties)))
(defvar *c10* (mk-rule-clause '(likes sally ?x) '((likes ?x moties)) ) )
(defvar *c11* (mk-rule-clause '(likes rod ?x) '((likes ?x renner) (likes ?x rod))))
(defvar *c12* (mk-rule-clause '(likes ?x ?y) '((amusing ?y) (likes rod ?y)) ) )
(defvar *c13* (mk-fact-clause '(likes ?x ?x)))

(defvar *g1* (mk-goal-clause '((likes sally ?who))))
(defvar *g2* (mk-goal-clause '((member sally (rod renner horvath sally . moties)))))

(setf *axioms* (list *c1* *c2* *c3* *c4* *c5* *c6* *c7* *c8*
                 *c9* *c10* *c11* *c12* *c13*))

(setf *goals* (list *g1*))
Input your own facts, rules and goals and assign them to variables *axioms* and *goals* as shown above. Or just use my test data to check the system out in the first instance.

Now go to Part 4:  TEST DATA and execute the following commands as shown there.
(defvar *igb-list-ia-list* (initialise-databases *goals* *axioms*))

(defvar *igb-list* (first *igb-list-ia-list*))  ; extended goal clause(s)

(defvar *ia-list*   (second *igb-list-ia-list*))  ; extended axiom clauses

(setf archived-goals-and-proofs (prove *goals* *axioms* 6))    ; depth 6 here

(pprint (setf all-proofs (show-all-proofs archived-goals-and-proofs *ia-list*)))
The search depth (eg 6) is arbitrary - try 4 or 8 etc.

With *1step* set to true,
(defvar *1step* t)        ; If t stops proof loop each iteration and
;                                      prints archived-goals, archived proofs & next goals
the theorem-prover will print its workings on each inference cycle and wait for input before proceeding. If you just want the thing to run to completion, set *1step* to nil.

---

2. In-code documentation

Most of the code is heavily documented with runtime examples - but read this post first.

---

3. Data Structures

The data structures are as follows.

1. We start with literals like (LIKES ?X RENNER), (LIKES SALLY ?WHO) - which the function 'unify' tries to unify to create bindings like this:
((?WHO . RENNER) (#:?X1113 . SALLY)).

2. At the level of basic binary resolution we have clauses, which look like this:

A fact is a clause (a list of literals) which contains only one (positive) literal followed by <- .
(defvar *c1* '( (likes horvath moties) <- )                          ; a fact clause
A rule is a clause (a list) which is a (positive) literal (the clause head) followed by <- and then a list of literals - the body of the clause.

We use <- with value nil as a spacer only for human readability.
(defvar *c2* '((likes sally ?who) <- ((likes ?who ?y) (likes ?y moties)))     ;  rule
A goal is a clause (a list) of one or more negative literals. First element is <- for human readability.
(defvar *g* '(<- ((likes ?x renner) (likes ?x rod)) ) )          ;  a goal clause

3. At the level of running the whole proof procedure we have extended-clauses, sometimes written as igb-list and ia-list for extended goal clauses and extended axiom clauses. The letter 'i' stands for index, and 'b' stands for binding.

An extended clause is a triple: index,  clause, then (if a goal) binding. It is implemented as a list like this: (index clause binding). Example:
(defvar *icb* '((16 (4 (nil nil)) (2 (nil nil)))                   ; Index
                     (<- ((likes ?x renner) (amusing ?x)))       ; Clause
                              ((T . T))))                                    ; Binding
An index is a binary tree of integers, with leaves nil.
Example: (1 (nil nil))   or   (16 (12 (7 (nil nil)) (4 (nil nil)) ) (5 (nil nil)) )
Each leading integer is the current clause number; the two immediate children are the two resolution parent clauses .. and so on. Initial goals, and axioms have no parents, thus nil.

The reason for extended clauses is that we need to keep track of the inference steps to reconstruct the entire proofs afterwards. The bindings are kept to allow us to instantiate variables in the original query which we need as the answer!

---

4. FAQs (reprinted from here).

4.1. Isn't this just Prolog?

Prolog is encountered as a black box. You provide a program and a query (as above) and you get back bindings, like this:
?- likes(sally, W).

W = sally;
W = rod;
... and so on

If you want proofs, to add the 'occurs check' or to change the order in which resolution steps are tried (the search strategy) - well tough: all those things are both fixed and opaque in Prolog. To make them explicit for modification, you have to write an explicit theorem-prover in Prolog (which can of course be done).

4.2. You are using Horn clauses?

Yes. I initially thought to implement a general clausal prover, but Horn clauses make the resolution step particularly simple (just one positive literal to match) and you lose neither generality nor expressive power. But since everything in the code is both modular and explicit, it would be easy to extend the program.

4.3. The style is functional?

Yes. Resolution theorem provers on the Internet are heavily optimised with clever, imperative algorithms and direct access data structures such as hash tables. This makes the code efficient but obscure - very hard to understand.

I didn't want to do that. This is a tool-kit and I'm not trying to create thousands of logical inferences per second. My intended application area (simple inference over an ever-changing knowledge-base for a chatbot) never requires massive inferential chains so clarity and easy modifiability was my objective.

4.4. What was hard?

The program is architected at three levels.

(a) Unification

This is basically already quite modular and straightforward. I re-used Peter Norvig's code.

(b) Resolution

Binary resolution itself - specially with Horn clauses - is a straightforward procedure - as you will see in the code. There are some subtleties with bindings and substitutions, but once you realise that resolution is fundamentally a local operation it's not too difficult.

(c) Control and proof construction

The process of creating new goals and resolving them with the axioms is somewhat complex although again, sticking with Horn clauses makes it a lot easier: just think of a naive Prolog execution model.

However, if you want to return proofs, you need to number the axioms and number-and-archive goals as you process them, capturing the resulting tree of inferences. At the end, for each empty-clause = successful proof, you need to trace that index-tree backwards to recover the proof steps. I found it a bit intricate.

4.5. What's next?

In theory you can do anything with a theorem-prover (Prolog being the existence proof) but it's not necessarily the best architecture. For a planner, where state changes are part of the problem definition, I need to adapt the tool-kit to a design centred around actions with pre- and post-condition in the context of a goal-state and an updating world-state. Such a dynamic model can be used both for actions in a (virtual) world and conversation planning via speech acts.

The theorem-prover remains optimal as a separate module, managing the crystallized knowledge base using inference to draw conclusions - for example to drive questions and answers.

I'm thinking of using the already-programmed Eliza front-end as a robust conversational interface. Doesn't matter if it's bluffing some of the time if it can use learning, inference and planning often enough.

Onwards to the GOFAI smart chatbot ...

---

5. Miscellaneous

As usual, code is provided for free use, unsupported and with no guarantees at all that there aren't bugs I've missed. Feel free to tell me in the comments.

Other code you may find useful:
  1. A resolution theorem prover in Lisp (as described here)
  2. Prolog in Lisp
  3. Minimax (noughts-and-crosses) in Lisp
  4. Eliza in Lisp
 An example of input and output of the prover.

---

Update: Saturday March 18th 2017.

1. Code here slightly updated (to v. 1.01) with change to 'show-proof' (and 'show-all-proofs') to add an additional 'top level goals' parameter, thereby getting rid of the previous embedded global variable. This makes the code completely functional now.

2. With further testing (on the genealogical axiom set below) .. with deeper inferences .. I have noticed that naive pprint doesn't do a good job of showing the output - the 'found proofs'. Over the next few days I'll write a dedicated display function which can handle more deeply-nested proofs and let you know when it's done. I'll also provide you with proper version of the genealogical test data below: (not completed testing, so it's illustrative only at this point).

; ---  Test data: family tree : axioms and goal ---
;   https://jameskulkarni17.wordpress.com/2011/09/26/family-tree-using-prolog/
;   with anglicised names and modified rules.

(defvar *g1* (mk-fact-clause '(male gerry)))
(defvar *g2* (mk-fact-clause '(male peter)))
(defvar *g3* (mk-fact-clause '(male john)))
(defvar *g4* (mk-fact-clause '(male mike)))
(defvar *g5* (mk-fact-clause '(male james)))
(defvar *g6* (mk-fact-clause '(male hugo)))

(defvar *g7* (mk-fact-clause '(female mary)))
(defvar *g8* (mk-fact-clause '(female jane)))
(defvar *g9* (mk-fact-clause '(female sarah)))
(defvar *g10* (mk-fact-clause '(female christine)))


(defvar *g11* (mk-fact-clause '(child-of gerry mary peter)))
(defvar *g12* (mk-fact-clause '(child-of gerry mary john)))
(defvar *g13* (mk-fact-clause '(child-of peter jane james)))
(defvar *g14* (mk-fact-clause '(child-of peter jane mike)))
(defvar *g15* (mk-fact-clause '(child-of john sarah christine)))
(defvar *g16* (mk-fact-clause '(child-of john sarah hugo)))


(defvar *g17* (mk-fact-clause '(brother peter john)))
(defvar *g18* (mk-fact-clause '(brother john peter)))
(defvar *g19* (mk-fact-clause '(brother james mike)))
(defvar *g20* (mk-fact-clause '(brother mike james)))

(defvar *g21* (mk-fact-clause '(brother hugo christine)))
(defvar *g22* (mk-fact-clause '(sister christine hugo)))

(defvar *g23* (mk-rule-clause '(parent ?X ?Y) '((child-of ?X ? ?Y))))
(defvar *g24* (mk-rule-clause '(parent ?X ?Y) '((child-of ? ?X ?Y))))

(defvar *g25* (mk-rule-clause '(sibling ?X ?Y) '((brother ?X ?Y))))
(defvar *g26* (mk-rule-clause '(sibling ?X ?Y) '((sister ?X ?Y))))

(defvar *g27* (mk-rule-clause '(father ?X ?Z) '((child-of ?X ? ?Z))))

(defvar *g28* (mk-rule-clause '(mother ?Y ?Z) '((child-of ? ?Y ?Z))))

(defvar *g29* (mk-rule-clause '(son ?X ?Y ?Z)
                              '((male ?X) (father ?Y ?X) (mother ?Z ?X))))

(defvar *g30* (mk-rule-clause '(daughter ?X ?Y ?Z)
                              '((female ?X) (father ?Y ?X) (mother ?Z ?X))))

(defvar *g31* (mk-rule-clause '(wife ?X ?Y) '((female ?X) (child-of ?Y ?X ?))))

(defvar *g32* (mk-rule-clause '(grandfather ?X ?Z)
                              '((male ?X) (parent ?X ?Y) (parent ?Y ?Z))))

(defvar *g33* (mk-rule-clause '(grandmother?X ?Z)
                              '((female ?X) (parent ?X ?Y) (parent ?Y ?Z))))

(defvar *g34* (mk-rule-clause '(uncle ?X ?Z)
                              '((male ?X)   (sibling ?X ?Y) (parent ?Y ?Z))))

(defvar *g35* (mk-rule-clause '(aunt ?X ?Z)
                              '((female ?X) (sibling ?X ?Y) (parent ?Y ?Z))))

(defvar *g36* (mk-rule-clause '(cousin ?X ?Y)
                              '((parent ?U ?X) (sibling ?U ?W) (parent ?W ?Y))))

(defvar *g37* (mk-rule-clause '(ancestor ?X ?Y ?Z) '((child-of ?X ?Y ?Z))))
(defvar *g38* (mk-rule-clause '(ancestor ?X ?Y ?Z)
                              '((child-of ?X ?Y ?W)  (ancestor ?W ? ?Z))))

(defvar *family-tree-axioms* (list *g1* *g2* *g3* *g4* *g5* *g6* *g7*
                                 *g8* *g9* *g10* *g11* *g12* *g13*
                                 *g14* *g15* *g16* *g17* *g18* *g19*
                                 *g20* *g21* *g22* *g23* *g24* *g25*
                                 *g26* *g27* *g28* *g29* *g30* *g31*
                                 *g32* *g33* *g34* *g35* *g36* *g37* *g38*))

(defvar *gg1* (mk-goal-clause '((father ?X ?Y))))
(defvar *gg2* (mk-goal-clause '((mother ?X ?Y))))
(defvar *gg3* (mk-goal-clause '((child-of ?X ?Y ?Z))))
(defvar *gg4* (mk-goal-clause '((son james ?X ?Y))))
(defvar *gg5* (mk-goal-clause '((daughter christine ?X ?Y))))
(defvar *gg6* (mk-goal-clause '((grandfather ?X ?Y))))
(defvar *gg7* (mk-goal-clause '((aunt ?X ?Y))))
(defvar *gg8* (mk-goal-clause '((uncle peter ?X))))
(defvar *gg9* (mk-goal-clause '((cousin ?X ?Y))))
(defvar *gg10* (mk-goal-clause '((ancestor ?X ?Y james))))

(defvar *family-tree-goal* (list *gg1* ))   ; start testing with the first goal

;;; A function, 'prove-it', to save copy-and-paste of the long commands below

(defun prove-it (goal depth)
    (setf *igb-list-ia-list*
        (initialise-databases *family-tree-goal* *family-tree-axioms*))
    (setf *igb-list* (first *igb-list-ia-list*))
    (setf  *ia-list*  (second *igb-list-ia-list*))
     (setf archived-goals-and-proofs (prove (list goal) *family-tree-axioms*  depth))
     (pprint (setf all-proofs
                           (show-all-proofs archived-goals-and-proofs *ia-list* (list goal))))
 nil)

#|   --- Commands to run the test data ---

(defvar *igb-list-ia-list*
        (initialise-databases *family-tree-goal* *family-tree-axioms*))
(defvar *igb-list* (first *igb-list-ia-list*))
(defvar *ia-list*  (second *igb-list-ia-list*))

(setf archived-goals-and-proofs (prove *family-tree-goal* *family-tree-axioms*  6))

(pprint (setf all-proofs (show-all-proofs archived-goals-and-proofs *ia-list* *family-tree-

goal*)))

or type

(prove-it *gg1* 6)    ; but currently doesn't display well

----------------------------------------------------------------------------------
|#

#|
**********************   Intended Output   *************************

(defvar *gg1* (mk-goal-clause '((father ?X ?Y))))

1 ?- father(X,Y).

X = gerry
Y = peter ;

X = gerry
Y = john ;

X = peter
Y = james ;

X = peter
Y = mike ;

X = john
Y = christine ;

X = john
Y = hugo ;

No
******************************************************
(defvar *gg2* (mk-goal-clause '((mother ?X ?Y))))

2 ?- mother(X,Y).

X = mary
Y = peter ;

X = mary
Y = john ;

X = jane
Y = james ;

X = jane
Y = mike ;

X = sarah
Y = christine ;

X = sarah
Y = hugh ;

No
******************************************************
(defvar *gg3* (mk-goal-clause '((child-of ?X ?Y ?Z))))

3 ?- child-of(X,Y,Z).

X = gerry
Y = mary
Z = peter ;

X = gerry
Y = mary
Z = john ;

X = peter
Y = jane
Z = james ;

X = peter
Y = jane
Z = mike ;

X = john
Y = sarah
Z = christine ;

X = john
Y = sarah
Z = hugo ;

No
******************************************************
(defvar *gg4* (mk-goal-clause '((son james ?X ?Y))))

4 ?- son(james,X,Y).

X = peter
Y = jane

Yes
******************************************************
(defvar *gg5* (mk-goal-clause '((daughter christine ?X ?Y))))

5 ?- daughter(christine,X,Y).

X = john
Y = sarah

Yes
******************************************************
(defvar *gg6* (mk-goal-clause '((grandfather ?X ?Y))))

6 ?- grandfather(X,Y).

X = gerry
Y = james ;

X = gerry
Y = mike ;

X = gerry
Y = christine ;

X = gerry
Y = hugo ;

No
******************************************************
(defvar *gg7* (mk-goal-clause '((aunt ?X ?Y))))

7 ?- aunt(X,Y).

X = jane
Y = christine;

X = jane
Y = hugo ;

X = sarah
Y = james ;

X = sarah
Y = mike ;

No
******************************************************
(defvar *gg8* (mk-goal-clause '((uncle peter ?X))))

8 ?- uncle(peter,X).

X = christine;

X = hugo;

No
******************************************************
(defvar *gg9* (mk-goal-clause '((paternal-cousin ?X ?Y))))

9 ?- cousin(X,Y).

X = james
Y = christine ;

X = james
Y = hugo ;

X = mike
Y = christine ;

X = mike
Y = hugo;

X = christine
Y = james ;

X = christine
Y = mike ;

X = hugo
Y = james ;

X = hugo
Y = mike ;

No
******************************************************
(defvar *gg10* (mk-goal-clause '((paternal-ancestor ?X ?Y james))))

10 ?- ancestor(X,Y,james).

X = peter
Y = jane ;

X = gerry
Y = mary ;

|#


Saturday, February 11, 2017

Diary: snow + books + chatbot backstory

We last had snow here back in December 2010 (when it was heavy). This morning only the lightest dusting .. and as I write, it's gone.

A light dusting of snow in our garden this morning
---

This new book by Dalton Conley and Jason Fletcher should be arriving today.

Amazon link

Expect a review in due course if it's any good.

Update: it's now arrived and I've had a peek. The authors are both sociology professors, which rings alarm bells. And they have an agenda. They're here to defend the standard social science model against the unsettling results of recent DNA sequencing programs and GWAS (genome-wide association studies). Their idea is to make the most minimal concessions to evidence while still preserving the traditional social-sciences advocacy-based value system. So races don't exist, racial genetic differences in cognition don't exist, and most else which might disturb the liberal agenda of the irrelevance of genetics for outcomes is explained away.

Depressing.

[Update (Sunday): my first impressions were wrong. The book is both more honest and more erudite than had appeared from a first skim. The authors may be billed as sociologists but they are both genetics research practitioners. Once you have absorbed the theory and methodology, it's harder to be agenda-driven - if you're honest. I think a review will be in order once I've finished.]

[Update (Monday): Finished. It was as my first impressions. A sophisticated attempt to save the SSSM from those pesky geneticists. Sophistry rules: don't waste your money. Review here.]

---

Genetics, especially human genomics, is the one area of contemporary science most at odds with prevailing western ideology. Most of the time this didn't matter - you can believe whatever you like if it has no practical consequences: in fact most human beliefs are like that.

But with emergent technologies for DNA sequencing, embryo selection and genome-editing, the science soon will matter. You will be able, in principle, to improve attributes (such as the health, emotional stability and intelligence) of your children. The issues will be regulation. cost and your desire to do so.

I anticipate, with no joy, decades of screaming hysteria.

I'm also working my way through Sean Carroll's book on entropy and the 'Past Hypothesis',



and Stephen Cohen's excellent biography of Bukharin,




not to mention Scott Bakker's 'Prince of Nothing'.




It's hard to timeshare: particularly when one is programming.

Incidentally, just as it is said that no-one feels hunger pangs when stepping out of a plane to do a parachute jump, an afternoon programming tends to obscure the fact you've skipped lunch. Just over 68kg this morning (10st 10.4lb) as the abdominal bulge recedes somewhat.

I'm pleased.

---

Just a short note-to-self about the chatbot stuff. The next thing (after the recent Eliza-style vacuity) is to capture the 'aboutness' of conversation. In chatting, each conversational partner brings some backstory to a discussion.

The essence of being an effective chatbot is:
  1. Manage a dynamic back-story of your own,
  2. Engage with the backstory of your human conversational partner.
If the chatbot is a cat, the backstory can be quite constrained - for example, nocturnal adventures in the garden with other cats, badgers, voles, birds, food and vomiting. I was looking at Peter Norvig's GPS reconstruction (previously discussed here) as a possible route to writing a dynamic simulation model.

The various entities I mentioned (cats, badgers, voles, ..) should have actions which require their preconditions in the garden knowledge-base (KB) to hold, and which then assert their post conditions in the next KB iteration. Make actions probabilistic and you've got an interesting simulation.

GPS doesn't quite work in this forward-chaining mode so it's not a case of just adapting his existing code. But the Eliza pattern-matching engine can be the reusable heart of it.

Friday, February 10, 2017

So I posted a lot of Lisp here

OK, I know that most of you won't want to hack through pages of Lisp code and pages of Lisp rules. The post which is worth a look at is the example in use.

This shows the strength and weakness of the vanilla Eliza in action.

---

It's tempting, and even conventional, to scoff at Eliza. Peter Norvig says in Chapter 5.4 of his "Paradigms of Artificial Intelligence Programming":
" In the end, it is the technique that is important - not the program. ELIZA has been "explained away" and should rightfully be moved to the curio shelf. Pattern matching in general remains important technique, and we will see it again in subsequent chapters. The notion of a rule-based translator is also important. The problem of understanding English (and other languages) remains an important part of AI. Clearly, the problem of understanding English is not solved by ELIZA."

But the Eliza engine is a very good place to start.

---

What next?



The inadequacies of pseudo-human interlocutors are more forgivable in an animal. I'm going to craft a rule set for our dear, departed pet using the existing Eliza engine: Shadow v. 1.0.

Then it's back to Peter Norvig for his Prolog interpreter in Lisp (chapter 11). This will give me a unification algorithm and a resolution procedure.

My proposed architecture is to use the Eliza engine for input-output and have a Prolog-style knowledge-representation and inference-engine at the back.

In theory, the knowledge-based could be data-filled with rather tedious Q&A - the kind of thing which gives chatbots a bad name - but it's more pleasant for the user to have an initial registration procedure in which the user just tells the system about the domain of interest. For example, details of family and friends. Yes, I'm aware of privacy issues: you ask, in the age of Facebook?

The final area of architecture which I've been thinking about is the topic of conversation. This relates to the information and issues currently being talked about which drive the conversation forward. In Eliza it's basically in the hands of the user: there is no chatbot control mechanism.

But we can do better once we can do inference.

Watch this space.

Common Lisp code for the Eliza chatbot

#| This is the Peter Norvig (slightly adapted) code for the vanilla Eliza chatbot. For the rules see here. For an example of the program in action, see here.

This code should run if copied and pasted into a Common Lisp system, provided the rules file is also available and the 'Load' command below suitably amended - or just copy and paste the rules at the end of this file.

---
|#

;;; Eliza:  February 7th 2017 - February 10th 2017
;;;
;;; From Peter Norvig's book: Chapter 5
;;; "Paradigms of Artificial Intelligence Programming"
;;;
;;; Pattern matching, patterns and dialogue manager
;;;
;;; Posted:
; http://interweave-consulting.blogspot.co.uk/2017/02/common-lisp-code-for-eliza-chatbot.html
;;;
;;;  Reminder: (how to load and access files)
;;;
; (load  "C:\\Users\\HP Owner\\Google Drive\\Lisp\\Prog-Eliza\\Eliza.lisp")

(load  "C:\\Users\\HP Owner\\Google Drive\\Lisp\\Prog-Eliza\\Eliza-Rules.lisp")


;;; ---Patterns and input ---
;;;
;;; A pattern is a list of:-  segment-variable-lists, symbols or variables.
;;; An input is a list of symbols (without variables).
;;;
;;; Test pattern match in simple cases
;;;
; (setf *inp1* '(I need a vacation))
; (setf *pat1* '(I need a ?x))
; (pat-match *pat1* *inp1)
;
; ... when matched, should return a binding '((?X . vacation))
;
; (setf *inp2* '(Shadow and I need a holiday))
; (setf *pat2* '((?* ?U) need (?* ?V)) )
;
;;; --- Binding constants ---

(defconstant +fail+ nil "Indicates pat-match failure.")

(defconstant +no-bindings+ '((t . t))
"Indicates pat-match success but with no variables.")

(defconstant +example-bindings+
   '((?X . (the cat)) (?Y . (sat on)) (?Z the mat))  )

;;; --- BINDINGS a list of (var . value) dotted pairs ---
;;;
; a list of the form (?* ?variable-name) denotes a segment variable.
; Eg:  (pat-match '((?* ?p) need (?* ?X)) '(Mr Hulot and I need a vacation))
; gives ((?P MR HULOT AND I) (?X A VACATION)) .. as the (dotted pair) binding.

(defun get-binding (var bindings)
  "Find a (variable . value) pair in a binding list."
  (assoc var bindings))

(defun binding-val (binding)
  "Get the value part of a single binding."
  (cdr binding))

(defun lookup (var bindings)
  "Get the value part (for var) from a binding list."
  (binding-val (get-binding var bindings) ))

(defun extend-bindings (var val bindings)
  "Add a (var . value) pair to a binding list."
  (acons var val bindings))

(defun variable-p (x)    ; SYMBOL -> Bool
  "Is x a variable, a symbol beginning with '?' "
  (and (symbolp x) (equal (char (symbol-name x) 0) #\? )))
 
(defun starts-with (list x)  ; SYMBOL-list x SYMBOL -> Bool
  "Is this a non-empty list whose first element is x?"
  (and (consp list) (eql (car list) x)) )
 
;;; --- Pattern Matcher ---

; pat-match: PATTERN x INPUT x BINDINGS -> BINDINGS

(defun pat-match (pattern input &optional (bindings +no-bindings+))
  "Match pattern against input in the context of the bindings"
  (cond ((eq bindings +fail+ ) +fail+)
        ((variable-p pattern ) (match-variable pattern input bindings))
        ((eql pattern input) bindings)
        ((segment-pattern-p pattern) (segment-match pattern input bindings))
        ((and (consp pattern) (consp input))
                 (pat-match (rest pattern) (rest input)
                         (pat-match (first pattern) (first input) bindings)))
        (t   +fail+) ))
;
; segment-pattern-p: PATTERN -> bool   (if first element of PATTERN is seg-var

(defun segment-pattern-p (pattern)
  "pattern a non-empty-list, 1st element a segment-matching-pattern: ((?* var) . pat)"
  (and (consp pattern)
       (starts-with (first pattern) '?* )))

(defun match-variable (var input bindings) ; VAR x SYMBOL(-list) x BINDINGS -> BINDINGS
  "Does VAR match input? Uses (or updates) and returns bindings."
  (let ((var+val (get-binding var bindings)))
    (cond ((not var+val) (extend-bindings var input bindings))  ; add new binding
          ((equal (cdr var+val) input) bindings)                             ; do nothing
          (t +fail+ )                                                                          ; clash with existing
   )) )

; (setf b +EXAMPLE-BINDINGS+)
; (match-variable '?X '(the cat) b)
; (match-variable '?U 'dog  b)
; (match-variable '?X 'dog  b)     ; returns NIL

; segment-match: PATTERN x INPUT x BINDINGS x NAT -> BINDINGS

(defun segment-match (pattern input bindings &optional (start1 0))
"Match the pattern = ((?* var) . pat) against input - returns BINDINGS"
  (let ((var (second (first pattern)) )     ; we know ?* is the first
        (pat (rest pattern )) )
     (cond ((null pat) (match-variable var input bindings))   ; return this binding

           ;; We assume that pat, the rest of pattern, starts with a constant
           ;; In other words, a pattern can't have 2 consecutive vars
           ;; pos is index into input, the symbol matching next item in pattern
           (t (let ((pos (position (car pat) input :start start1 :test #'equalp)))
                   (if (null pos)
                         +fail+                          ; pattern-input mismatch
                         (let* ((input-rest (subseq input pos))
                                (input-prefix (subseq input 0 pos)) ; bind to var!
                                (b1 (match-variable var input-prefix bindings)) ;OK
                                (b2 (pat-match pat input-rest b1)))
                           ;; If b2 failed , try another longer one
                             (if (eq b2 +fail+)
                                  (segment-match pattern input bindings (+ pos 1) )
                                  b2) ) ) ) ) ) ) )

; (setf pattern '((?* ?W) sat on the ?V))
; (setf input '(The cat sat on sat on the mat))
; (segment-match pattern input +no-bindings+)

#| Note on segment-match (from Norvig, chapter 5.3)

In writing segment-match, the important question is how much of the input the
segment variable should match. One answer is to look at the next element of the
pattern (the one after the segment variable) and see at what position it occurs in the
input. If it doesn't occur, the total pattern can never match, and we should fail.

If it does occur, call its position pos. We will want to match the variable against the
initial part of the input, up to pos. But first we have to see if the rest of the pattern
matches the rest of the input. This is done by a recursive call to pat-match. Let the
result of this recursive call be named b2. If b2 succeeds, then we go ahead and match
the segment variable against the initial subsequence.

The tricky part is when b2 fails. We don't want to give up completely, because
it may be that if the segment variable matched a longer subsequence of the input,
then the rest of the pattern would match the rest of the input. So what we want is to
try segment-match again, but forcing it to consider a longer match for the variable.

This is done by introducing an optional parameter, start1, which is initially 0 and is
increased with each failure. Notice that this policy rules out the possibility of any
kind of variable following a segment variable.
|#

;;; --- test data for pattern matching ---

; (pat-match '(i need a ?X) '(i really need a vacation)) ; NIL
; (pat-match '(this is easy) '(this is easy))            ; ((T . T))
; (pat-match '(?X is ?X) '((2 + 2) is 4))                ; NIL
; (pat-match '(?X is ?X) '((2 + 2) is (2 + 2 )))       ; ((?X 2 + 2)) = ((?X . (2 + 2)))
; (pat-match '(?P need . ?X) '(i need a long vacation)) ;((?X A LONG VACATION) (?P . I))

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;
;;; --- RULES ---
;;;
;;; Here's an example of a rule: input-matching-pattern + one or more response-patterns
;;;
;;; (  ((?* ?X) I want (?* ?Y ))                  ; Pattern part
;;;       (What would it mean if you got ?Y)      ; Three possible responses
;;;       (Why do you want ?Y)
;;;       (Suppose you got ?Y soon?)   )
;;;
;;; -----------------------------------------------------------------------------------
;;; --- ELIZA top level ---
;;;
;;; Note on output - recall (terpri) prints a newline.
;;; print is just like prin1 (or princ - human readable)
;;; except that the printed representation of object
;;; is preceded by a newline and followed by a space.
;;;
;;; pprint is just like print except that the trailing space is omitted
;;; and object is printed with the *print-pretty* flag non-nil - pretty output.

(defun eliza (rules)
  "Respond to user input using pattern matching rules. 'bye' to terminate"
  (princ "Hello, I am your doctor. Please type bye to finish.")
  (terpri)
  (terpri)
  (princ "Please type your first name then return> ")
  (setf *user-name* (car (read-line-no-punct)))              ; substitutes $name$ in response
  (loop
   (print 'speak>)
   (let ((input (read-line-no-punct)))
      (if (equalp input (list 'bye)) (progn (print 'Goodbye) (return))
          (let* ((raw-response (use-eliza-rules input rules))
                 (response (flatten1 raw-response))
                 (out  (substitute *user-name* '$name$ response :test #'equalp)) )
            (pprint out) ) ) ) )  )

; Test:  (eliza +eliza-rules+)
; Test:  (eliza +eliza-rules-extended+)

(defun read-line-no-punct ()
  "Read an input line terminated by 'return', ignore punctuation, -> symbol-list"
  (read-from-string
   (concatenate 'string "(" (substitute-if #\space #'punctuation-p (read-line))
                ")")  ))

(defun punctuation-p (char) (find char ".,;:'!?#-()\\\""))

(defun use-eliza-rules (input rules)
  "Find some rule with which to transform the input ... ."
  (some #'(lambda (rule)
            (let ((bindings (pat-match (rule-pattern rule) input)))
              (if (not (eql bindings +fail+) )
                  (sublis (switch-viewpoint bindings)
                          (random-elt (rule-responses rule )) ))))
        rules))

(defun rule-pattern (rule) (car rule))     ; The pattern to match the input

(defun rule-responses (rule) (cdr rule))   ; the list of (remaining) response-patterns

(defun switch-viewpoint (bindings)
  "Change I to you and vice versa, and so on in bindings."
  (sublis '((I . you)  (you . I)    (me . you)
            (am . are) (was . were) (are . am)) bindings ))

(defun flatten1 (l)
  "Append together elements (or lists) in the list l (not lists of lists)."
  (mappend #'mklist  l) )

(defun mklist (x)
  "Return x if it is a list, otherwise (x)."
  (if (listp x) x (list x)))

(defun mappend (fn l)   ; (* -> **) x *-list -> <flat list>
  "Apply fn to each element of list l and append the results."
  (apply #'append (mapcar fn l) ))

(defun random-elt (choices)
  "Choose an element from a list at random."
  (elt choices (random (length choices )) ))


;;; --- End ---

The 'Doctor' rule-set for Eliza

#|  These are the rules which drive the 'Doctor' version of Eliza. For the Eliza code, see here.

This entire post should be executable if copied and pasted into Common Lisp.

For an example of the rules in use, see here.  
|#
;;; Eliza Rules:  February 7th 2017 - Feb 10th 2017
;;;
;;;  From Peter Norvig's book: Chapter 5
;;; "Paradigms of Artificial Intelligence Programming"
;;;
;;; Pattern matching, patterns and dialogue manager
;;;
;;; Posted: http://interweave-consulting.blogspot.co.uk/2017/02/the-doctor-rule-set-for-eliza.html
;;;
;;;  Reminder: (how to load and access files)
;;;
; (load  "C:\\Users\\HP Owner\\Google Drive\\Lisp\\Prog-Eliza\\Eliza-Rules.lisp")

;;; --- SHORT LIST OF ELIZA RULES ---
;;;
;;;  Note that the symbol $name$ will be substituted by the user's first name
;;;  in the Eliza top-level procedure.

(defparameter +eliza-rules+
  '(
    (((?* ?x) hello (?* ?y))
     (Hi $name$ - please state your problem))

    (((?* ?x) I want (?* ?y))
     (What would it mean if you got ?y)
     (Why do you want ?y)
     (Suppose you got ?y soon))

    (((?* ?x) if (?* ?y))
     (Do you really think its likely that ?y $name$)
     (Do you wish that ?y)
     (What do you think about ?y)
     (Really - if ?y))

    (((?* ?x) no (?* ?y))
     (Why not?)
     ($name$ - you are being a bit negative)
     (Are you saying "NO" just to be negative?))

    (((?* ?x) I was (?* ?y))
     (Were you really?)
     (Perhaps I already knew you were ?y)
     (Why do you tell me you were ?y now?))

    (((?* ?x) I feel (?* ?y))
     (Do you often feel ?y ?))

    (((?* ?x) I felt (?* ?y))
     (What other feelings do you have?))))

;;; --- EXTENDED RULES (c. ~50)  ---

(defparameter +eliza-rules-extended+
'(
(((?* ?x) hello (?* ?y))
 (Hi $name$ - please state your problem.))

(((?* ?x) hi (?* ?y))
 (Hi $name$ - please state your problem.))

(((?* ?x) computer (?* ?y))
 (Do computers worry you?)
 (What do you think about machines?)
 (Why do you mention computers?)
 (What do you think machines have to do with your problem?))

(((?* ?x)  my name (?* ?y))
 ($name$ - I am smarter than you think!))

(((?* ?x) your name (?* ?y))
 (I prefer to be an anonymous doctor $name$))

(((?* ?x) sorry (?* ?y))
 (Please don't apologize)
 (Apologies are not necessary)
 (What feelings do you have when you apologize?))

(((?* ?x) I remember (?* ?y))
 (Do you often think of ?y)
 (Does thinking of ?y bring anything else to mind?)
 (What else do you remember?)
 (Why do you recall ?y  right now?)
 (What in the present situation reminds you of ?y)
 (What is the connection between me and ?y))

(((?* ?x) do you remember (?* ?y))
 (Did you think I would forget ?y ?)
 (Why do you think I should recall ?y now)
 (What about ?y)
 (You mentioned ?y))

(((?* ?x) if (?* ?y))
 (Do you really think its likely that ?y)
 (Do you wish that ?y)
 (What do you think about ?y)
 (Really -- if ?y))

(((?* ?x) I dreamt (?* ?y))
 (Really - ?y)
 (Have you ever fantasized ?y while you were awake?)
 (Have you dreamt ?y before?))

(((?* ?x) I dreamed (?* ?y))
 (Really - ?y)
 (Have you ever fantasized ?y while you were awake?)
 (Have you dreamed ?y before?))

(((?* ?x) dream about (?* ?y))
 (How do you feel about ?y in reality?))

(((?* ?x) dream (?* ?y))
 (What does this dream suggest to you?)
 (Do you dream often?)
 (What persons appear in your dreams?)
 (Dont you believe that dream has to do with your problem?))

(((?* ?x) my mother (?* ?y))
 (Who else in your family ?y)
 (Tell me more about your family))

(((?* ?x) my father (?* ?y))
 (Your father?)
 (Does he influence you strongly?)
 (What else comes to mind when you think of your father?))

(((?* ?x) I want (?* ?y))
 (What would it mean to you if you got ?y)
 (Why do you want ?y)
 (Suppose you got ?y soon))

(((?* ?x) I am glad (?* ?y))
 (How have I helped you to be ?y)
 (What makes you happy just now)
 (Can you explain why you are happy that ?y))

(((?* ?x) I am sad (?* ?y))
 (I am sorry to hear you are depressed)
 (I'm sure its not pleasant to be sad))

(((?* ?x) are like (?* ?y))
 (What resemblance do you see between ?x and ?y ?))

(((?* ?x) is like (?* ?y))
 (In what way  is  it that ?x is like ?y)
 (What resemblance do you see?)
 (Could there really be some connection?)
 (How?))

(((?* ?x) alike (?* ?y))
 (In what way?)
 (What similarities are there?))

(((?* ?x) same (?* ?y))
 (What other connections do you see?))

(((?* ?x) I was (?* ?y))
 (Were you really?)
 (Perhaps I already knew you were ?y)
 (Do you think its relevant that you were ?y now?))

(((?* ?x) was I (?* ?y))
 (What if you were ?y ?)
 (Do you think you were ?y)
 (What would it mean if you were ?y))

(((?* ?x) I am (?* ?y))
 (In what way are you ?y)
 (Do you want to be ?y ?))

(((?* ?x) am I (?* ?y))
 (Do you believe you are ?y)
 (Would you want to be ?y)
 (You wish I would confirm you are ?y)
 (What would it mean if you were ?y))

(((?* ?x) am (?* ?y))
(Why do you say "AM?")
 (I dont really get that) )

(((?* ?x) are you (?* ?y))
 (Why are you interested in whether I am ?y or not?)
 (Would you prefer if I werent ?y)
 (Perhaps I am ?y in one of your fantasies?))

(((?* ?x) you are (?* ?y) )
 (Perhaps I am ?y only in your imagination!)
 (What makes you think I am ?y ?))

(((?* ?x) because (?* ?y))
 (Is ?y the real reason?)
 (What other reasons might there be?)
 (Does ?y seem to explain anything else?))

(((?* ?x) were you (?* ?y))
 (Perhaps I was ?y)
 (What do you think?)
 (In some places being ?y is barely legal!)
 (What if I had been ?y))

(((?* ?x) I cant (?* ?y))
 (Maybe you could ?y now)
 (What if you could ?y ?))

(((?* ?x) I miss (?* ?y))
 (Has ?y been gone long?)
 ($name$ - I can feel your sadness about ?y) )

(((?* ?x) I feel (?* ?y))
 (Do you often feel ?y ?))

(((?* ?x) I felt (?* ?y))
 (What other feelings do you have?))

(((?* ?x) I (?* ?y) you (?* ?z))
 (Perhaps in your fantasy we ?y each other))

(((?* ?x) why dont you (?* ?y))
 (Should you ?y yourself?)
 (Do you believe I dont ?y)
 (Perhaps I will ?y in good time))

(((?* ?x) yes (?* ?y))
 (You seem quite positive - please expand on this)
 ($name$ - I feel we should move on from this to another subject)
 (I understand $name$ but you need to move on to a new topic))

(((?* ?x) no (?* ?y))
 (Why not $name$ ?)
 (You are being a bit negative $name$)
 (Are you saying "NO" just to be negative?))

(((?* ?x) someone (?* ?y))
 (Can you be more specific about - ?y))

(((?* ?x) everyone (?* ?y))
 (Surely not everyone ?y)
 (Can you think of anyone in particular who ?y)
 (Who for example?)
 (You are thinking of a special person?))

(((?* ?x) always (?* ?y))
 (Can you think of a specific example?)
 (When?)
 (What incident are you thinking of?)
 (Really  - always?))

(((?* ?x) what (?* ?y))
 (Why do you ask?)
 (Does that question interest you?)
 (What is it you really want to know?)
 (What do you think?)
 (What comes to your mind when you ask that?))

(((?* ?x) why (?* ?y))
 (Its a good question - why ?y)
 (?x many things but its never that clear)
 (Well $name$ you are really in to deep questions) )

(((?* ?x) perhaps (?* ?y))
 (You do not seem quite certain.))

(((?* ?x) are (?* ?y))
 (Did you think they might not be ?y)
 (Possibly they are ?y))

(((?* ?x) that I (?* ?y))
 (Are you happy or sad that you ?y)
 (Possibly you should ?y))

(((?* ?x))
 (Very interesting)
 (I am not sure I understand you fully)
 (What does that suggest to you?)
 (Please continue)
 (Go on)
 (Do you feel strongly about discussing such things?))
))

Thursday, February 09, 2017

Diary: weights + Lisp reader issues + trim & style

Consider a hulking biker, tattoos and leotard, bench-pressing some massive barbell.

Alongside him, place that irritatingly-cheerful hunk Dr Chris van Tulleken, raising and lowering some wimpy baby dumbbells.

Pretending to be exhausted. Not this stagey picture below.



It was all in aid of this study (in the latest series of "Trust Me, I'm a Doctor").
"The study split 49 weight trainers into two groups and started them on a 12-week weight training programme. For each participant, they calculated their ‘one-repetition maximum’ or 1RM – that’s the heaviest weight they can lift.

"They then split the study into two groups, one group lifting 30-50% of their 1RM and the other group lifting 75-90%. The key thing was that each group lifted their weights to ‘volitional failure’ – in other words, they lifted until they couldn’t lift any more.

‘Failure’ will happen to anyone and everyone, however strong, if they do enough repetitions. So the group lifting the lighter weights did a larger number of reps (20-25) than the group lifting the heavier weight (8-12).

"The theory behind muscle failure is to do with ‘motor units.’ Motor units are bundles of muscle fibres controlled by a nerve. When you lift a weight, motor units will be required to contract the muscle. With each lift, some motor units will get fatigued, so additional motor units need to be used to do the next lift. Sooner or later you get to a point where all your available motor units have been exhausted – that’s what causes your muscles to fail.

"In the McMaster study, the results showed that despite lifting different weights, both groups showed the same increase in strength and muscle growth. In other words, doing heavy weights with fewer reps or lighter weights with more reps made no difference. These results agreed with earlier research conducted by the same group.

"So what does that mean for the rest of us? Well, it means that you can get results lifting heavy weights OR lighter weights, so long as you’re pushing your muscles to work harder than they normally do. You don’t always need to lift to failure to get results – but your muscles need to be ‘overloaded’ compared to your normal day to day life. Strength and conditioning coach Richard Blagrove from St. Mary’s University, Twickenham, suggests that on a scale of 1 to 10, where 10 is repetition failure, lifting to 7 or 8 is about right.

"If your muscles are feeling that overload once a week, your body will adapt and get stronger. If you want to continue to get stronger, you will need to constantly re-assess and progress your weight or rep level, to make sure you are always pushing your muscles beyond their comfort zone. If your weight training feels easy, it probably isn’t doing anything for you.

"You can get results using weights machines or free weights. Free weights also force you to use stabiliser muscles meaning you use more energy, and your joints move in their most natural way. But it’s important to lift correctly, so as soon as your “form” starts to go, you should probably stop. If you DO want to push yourself to failure, weight machines might be the safest place to do it. If you lift to failure with free-weights you need a partner who can relieve you of the weight when you can no longer lift it."
In a nutshell, don't go with "go heavy or go home!".

Do enough reps so that the last three are 'difficult'. Obviously you need enough weight so that you can get there within, say, 8-15 reps.

---

While I was watching our recording of "Trust Me, ..." my subconscious was whirring furiously. This afternoon I had painstakingly edited 43 rules for my Eliza program, which I'm reconstructing. The program works! But the rule set:
(defparameter +eliza-rules-extended+  '(.. stuff .. ))
refused to compile: weird and incomprehensible error messages complaining of 'invalid traits'. We're meant to be AI-advancing daily, but our development environments haven't the first clue.

My epiphany was that a rule which said something like:
(Why don't you like your mother?)
is going to hit a major problem with that quote character.

I rushed upstairs and deleted all the quotes from don't, doesn't, it's .. and everything loaded properly. Tomorrow I'll test it thoroughly and maybe post the code.

That should thrill you, dear reader.

---

As you know, my first target emulation is going to be the cat. Clare, who had to suffer through a tedious explanation of my afternoon difficulties, suggested that I should be designing a chatbot for the elderly. I'm to data-fill it with small talk about family, friends and neighbours (and pets?).

I suspect there's a market for that, and I'm not completely convinced that a state-of-the-art artificial neural net is required to implement it either.

But here to the contrary is Google's view:
"A simple strategy to build lightweight conversational models might be to create a small dictionary of common rules (input → reply mappings) on the device and use a naive look-up strategy at inference time.

This can work for simple prediction tasks involving a small set of classes using a handful of features (such as binary sentiment classification from text, e.g. “I love this movie” conveys a positive sentiment whereas the sentence “The acting was horrible” is negative).

But, it does not scale to complex natural language tasks involving rich vocabularies and the wide language variability observed in chat messages."
---

The diary would not be complete without observing that Clare had her hair done today.


I did notice.

Tuesday, February 07, 2017

The return of (virtual) Shadow

The late Shadow

Chapter 5 of Peter Norvig's excellent "Paradigms of Artificial Intelligence Programming: Case Studies in Common Lisp" is entitled "ELIZA: Dialog with a Machine". I have spent the afternoon studying his pattern-matching code and transferring it into an executable Lisp file.*

An example of an Eliza pattern is this:
'((?* ?P) need (?* ?X))
which picks up text before and after the word 'need'.

If you tell Eliza:
'(Shadow and I need a vacation),
the program will match the input with the pattern above, thus ..
(pat-match '((?* ?P) need (?* ?X)) '(Shadow and I need a vacation))
to create a binding
'((?P SHADOW AND I) (?X A VACATION))       ;   (dotted pair) binding
which can then be passed to the output-generating pattern. And so on.

---

But we're not so much interested in Eliza and that Rogerian psychoanalyst. We want Shadow, our much-missed departed cat (RIP July 1st 2016).

The first step is to change the rules: less talk of vacations and how you're feeling about your parents, more about voles. And being sick on the carpet.

The Eliza architecture was criticised, correctly, for being totally vacuous. The emptiness of the 'conversation' led to bored withdrawal after a period depending upon the narcissism of the user.

We can do better.

To add intelligence to Shadow (such a bright cat!) we have to give him knowledge (in the knowledge lies the power) together with reasoning capability.

Peter Norvig helpfully gives us an equation (chapter 16, p. 548 or thereabouts):
Expert System = Prolog + uncertainty + caching + questions + explanations
The Prolog part powers the knowledge-base capturing Shadow's deep understanding of his own likes, light-hearted escapades and more lethal habits; the Q&A allows for his more sensitive, introspective side.

Peter Norvig has a detailed chapter (11 - Logic Programming) where he explains how to implement Prolog in Lisp (interpreted).

I think this is the way to go with chatbots: combine Eliza-style interaction with a little knowledge-based reasoning to add interest and depth, and to steer the conversation. Perhaps a semantic grammar in there somewhere to help extract meaning from the user's input, though personally, given the ungrammaticality of dialogue, I've always put semantics/pragmatics first.

Anyway, watch this space. It won't be quick .. but we will bring him back!

---

* We spent the morning strolling in the sun around Cheddar reservoir.

Saturday, December 10, 2016

Why I never get anything done

I'm thinking ahead to when I get the Replika app on my Nexus 6 phone, early next year. I guess I'll soon be loading their Cloud database with forty texts per day.

I need to remember their usage model:
""Anyone grieving for a loved one will soon be able to talk to them using artificial intelligence technology pioneered by a woman who lost her best friend.

"Eugenia Kuyda, co-founder of Luka, a US technology company, has used the latest in AI systems to create a virtual version of Roman Mazurenko, who was killed in a road accident last year, days before his 33rd birthday.

"She fed more than 8,000 lines of text messages sent by Mr Mazurenko into a Google programme that allows people to create their own “chatbots” — computer programmes designed to simulate conversation with human users.

"The chatbot responds to questions in language that mimics Mr Mazurenko’s speech patterns so that she and other friends can “talk” to him as part of the grieving process.

"Ms Kuyda, 29, said that any doubts she had about the technology were allayed when the bot responded to her first questions in a tone that sounded like her friend."

The point of all this is that someone else, interacting with my chatbot-Replika, will think they're talking to "me". But I'm a different person depending on to whom I'm talking. Not everyone gets my flippant sense of humour so occasionally I have to act serious.

That's going to be a tough one.

Then I thought: it's unlikely that Replika (the company) will build a personal back-story, a contextual knowledge-base to power inference and make "my" replies more intelligent.

So why not do it myself?

I could simply copy all those Q&A texts into a .txt file and process them via a program like Eliza plus an automated theorem-prover (ATP).

The easiest way to do this would be in Prolog. The Eliza part isn't so hard: I already implemented a simple version using "The Art of Prolog" by Sterling and Shapiro. Writing an ATP in Prolog would also be interesting (and is also covered in the book).




The strategy is this. Start with the natural language input from those forty texts per day, specifically the questions being asked of me. We can't do inference on natural language so we have to translate it into a formal language with inference rules. The simplest is first-order predicate calculus: the Prolog clause variant would work.

We need the additional knowledge base which I would hand-craft (based on my own text responses) plus some rules to guide conversational inference - getting from an input to producing an appropriate output. All this could be coded as Prolog rules. It's similar to writing a meta-interpreter for Prolog in Prolog, covered in the book.

Finally I would translate the clausal form of the output into natural language as "my" response. A better response than any that a superficial Eliza-chatbot-like architecture could give.

You may think, like me, that this is already sounding like a big project.

---

So then I think. No-one but me could ever use this system. (Which kind of undermines the point).

It would have to be embedded in something like SWI-Prolog which would need to be launched first and would only work on my own machine. Getting it to work consistently and robustly across the Internet is most likely impossible.

Perhaps this should be implemented in Javascript so that my expert-system-Replika hybrid could be loaded into a client web page for anyone who wanted to have a conversation with "me"?
- There appears to be no really usable Prolog to Javascript compiler.
Writing anything as complex as I have described in native Javascript seems impossibly difficult.

... and so my mind wanders off, to something more immediately gratifying ... .

---

Sorry if this all sounds so GOFAI. I have as much chance of running TensorFlow at home as fly ...

Thursday, August 25, 2016

"Help Eliza, I'm in trouble!"

I'm something of a subscriber to the view: 'AI's the solution .. so what's the problem?'

The problem under consideration today is that of child abuse, mentioned in this post about Internet paedophiles yesterday, and prominent in continuing revelations about abuse at Ampleforth College.
[Wikipedia: "Ampleforth College is a coeducational independent day and boarding school in the village of Ampleforth, North Yorkshire, England. It opened in 1802 as a boys' school, and is run by the Benedictine monks and lay staff of Ampleforth Abbey.
...
Several monks and three members of the lay teaching staff molested children in their care over several decades. In 2005 Father Piers Grant-Ferris admitted 20 incidents of child abuse. This was not an isolated incident.

"The Yorkshire Post reported in 2005: "Pupils at a leading Roman Catholic school suffered decades of abuse from at least six paedophiles following a decision by former Abbot Basil Hume not to call in police at the beginning of the scandal."]
---

Let me remind you about Eliza, the original chatbot developed by Joseph Weizenbaum.
"ELIZA worked by simple parsing and substitution of key words into canned phrases. Depending upon the initial entries by the user, the illusion of a human writer could be instantly dispelled, or could continue through several interchanges.

"It was sometimes so convincing that there are many anecdotes about people becoming very emotionally caught up in dealing with [ELIZA] for several minutes until the machine's true lack of understanding became apparent.

"Weizenbaum's own secretary reportedly asked him to leave the room so that she and ELIZA could have a real conversation.

"As Weizenbaum later wrote, "I had not realized ... that extremely short exposures to a relatively simple computer program could induce powerful delusional thinking in quite normal people."
Eliza works by matching text input against a large database of templates. Each input template is linked to one or more possible output templates, with variables which can be instantiated to the substantive words from the input.

Eliza might, for example,
"respond to "My head hurts" with "Why do you say your head hurts?" A possible response to "My mother hates me" would be "Who else in your family hates you?"

"ELIZA was implemented using simple pattern matching techniques, but was taken seriously by several of its users, even after Weizenbaum explained to them how it worked. It was one of the first chatterbots."
In addition to crafting a reply, Eliza could easily have updated a user-database with the information it was receiving.

---

It's easy to see how this could be applied to helping victims of child abuse. A key design principle is that the abuser must not become aware that the child is passing on information: this rules out a tailored 'abuse app'.

I suggest a special WhatsApp-connected chatbot with a widely publicised name - let's say Help!.

The child contacts Help! on WhatsApp and the first thing he or she is asked to do is choose a name, say Peter, which is what will appear (instead of Help!) on their WhatsApp contacts list. I think the history of chats with Peter is going to have to vanish too, replaced with harmless confected froth.

The child is typing to an Eliza-like chatbot (maybe more like IBM's Watson than Eliza) which has been trained on scripts from charities like Childline.

Like Weizenbaum, we know that people of all ages are especially likely to confide in an AI agent.

The database which Help! constructs is a transcript of alleged abuse. The real problem is what to do with it. No doubt it's encrypted and identity-protected but at some point someone has to assess whether this is a real or false allegation, and figure out how to proceed.

But these are problems charities already have to deal with.

I think they should move on the app. There's already one for carers.

Thursday, September 03, 2015

AI Progress Report



So I got 'Eliza' working today, the simplified version from 'The Art of Prolog'. Time to take stock and figure out where to go next.

First, Prolog. Its supporters always touted it as a higher-level language than Lisp - though I always found Lisp more congenial, I like to set up data structures and manipulate them explicitly. With Prolog you define relationships between things and the miraculous powers of unification and depth-first search with backtracking pull magical rabbits out of hats. The Eliza program in Prolog can be read in its entirety on one screen, ditto for the blocks world planning system.

This procedural power is the result of enormously complex recursive structures built at execution time by the Prolog system. It frequently defies one's powers of abstraction, short-term memory and inference to visualise what's actually going on. I know you're meant to read and understand the programs declaratively, but in reality you don't get too far without a consideration of what actually happens at run-time.

Still, the power to write ridiculously-powerful programs in just a few lines of code is addictive. It reminds me of the first time I fired the General-Purpose Machine Gun (GPMG). I was good with the Lee-Enfield rifle and prided myself on my accuracy; the GPMG just bounced around and hosed the target. So much power and so little control!

---

Eliza and the Blocks World Planner were little milestones I had set myself, like climbing Pen y Fan. Items on my bucket list if you like. So what next?

Once you know how to set up knowledge bases and inferential systems you have the tools for developing intelligent agents. But, as I have cited before on these pages, 'in the knowledge lies the power'. If your agent lives in a closed-world with a fixed and limited database and rule set, it's going to run out of new things to do pretty fast. The interest comes from its interactions with the wider world.

Yet as Doug Lenat noted in the context of his 'Cyc' project:
"Any time you look at any kind of real life piece of text or utterance that one human wrote or said to another human, it's filled with analogies, modal logic, belief, expectation, fear, nested modals, lots of variables and quantifiers," Lenat said. "Everyone else is looking for a free-lunch way to finesse that. Shallow chatbots show a veneer of intelligence or statistical learning from large amounts of data. Amazon and Netflix recommend books and movies very well without understanding in any way what they're doing or why someone might like something.

"It's the difference between someone who understands what they're doing and someone going through the motions of performing something."

Cycorp's product, Cyc, isn't "programmed" in the conventional sense. It's much more accurate to say it's being "taught." Lenat told us that most people think of computer programs as "procedural, [like] a flowchart," but building Cyc is "much more like educating a child."

"We're using a consistent language to build a model of the world," he said.

This means Cyc can see "the white space rather than the black space in what everyone reads and writes to each other." An author might explicitly choose certain words and sentences as he's writing, but in between the sentences are all sorts of things you expect the reader to infer; Cyc aims to make these inferences.

Consider the sentence, "John Smith robbed First National Bank and was sentenced to 30 years in prison." It leaves out the details surrounding his being caught, arrested, put on trial, and found guilty. A human would never actually go through all that detail because it's alternately boring, confusing, or insulting. You can safely assume other people know what you're talking about. It's like pronoun use - he, she, it - one assumes people can figure out the referent. This stuff is very hard for computers to understand and get right, but Cyc does both.

"If computers were human," Lenat told us, "they'd present themselves as autistic, schizophrenic, or otherwise brittle. It would be unwise or dangerous for that person to take care of children and cook meals, but it's on the horizon for home robots. That's like saying, 'We have an important job to do, but we're going to hire dogs and cats to do it.'"
Cyc has been in development since 1984 and its knowledge base currently contains over one million human-defined assertions, rules or common sense ideas. Yet it's still barely found practical use. I'm certainly not planning on reproducing that level of effort.

I think we're back to virtualizing the cat ...

Sunday, August 09, 2015

Simulating your significant other

Talking Angela: a tiresome app
I've told Clare that I'm writing a simulation of her - and that this explains why I spend long hours toiling away upstairs in  my study generating Prolog code.

I'm not sure whether she believes me ...

'Virtual Clare' will not be a chatbot in the mould of Eliza, or modern equivalents such as Cleverbot (try it here). Chatbot conversation is purposeless, vacuous and tedious beyond belief as it mindlessly pattern-matches canned responses*. But, as Ed Feigenbaum at Stanford noted with regard to expert systems, “In the knowledge lies the power.”

There were some expert systems a while back which explored a kind of creativity and learning. Given a comprehensive list of facts and a bunch of rules, they would apply the rules to generate new facts and then use heuristics to weed out those that were obvious, pointless and redundant. The system kind of saw the implications of what it knew. You can see how this might beef up a conversational chatbot's repertoire.

It's impossible for me to replicate the knowledge of an adult person - the significant other simply has had too many experiences, has too many memories and too much knowledge of the intricate web of daily life. The domain of conversation has to be drastically reduced -- for example, to the worldview of a cat.

Yes, notwithstanding Wittgenstein's aversion to the idea, I think Gossip Cat is coming back!

---

* So I guess that makes simulating me pretty easy, then ...