Saturday, January 21, 2017

Neural Nets written in 'logic gate assembler'

From TechCrunch: " frees AI from the prison of the supercomputer"
"When someone talks about AI, or machine learning, or deep convolutional networks, what they’re really talking about  -  as is the case for so many computing concepts  -  is a lot of carefully manicured math.

At the heart of these versatile and powerful networks is a volume of calculation only achievable by the equivalent of supercomputers. More than anything else, this computational cost is what is holding back applying AI in devices of comparatively little brain: phones, embedded sensors, cameras. ...

Machine learning, Farhadi continued, tends to rely on convolutional neural networks (CNN); these involve repeatedly performing simple but extremely numerous operations on good-sized matrices of numbers. But because of the nature of the operations, many have to be performed serially rather than in parallel. ...

It’s the unfortunate reality of both training and running the machine learning systems performing all these interesting feats of AI that they feature phenomenally computationally expensive processes.

“It’s hard to scale when you need that much processing power,” Farhadi said. Even if you could fit the “beefy”  -  his preferred epithet for the GPU-packed servers and workstations to which machine learning models are restricted -  specs into a phone, it would suck the battery dry in a minute.

Meanwhile, the accepted workaround is almost comically clumsy when you think about it: You take a load of data you want to analyze, send it over the internet to a data center where the AI actually lives and computers perhaps a thousand miles away work at top speed to calculate the result, hopefully getting back to you within a second or two."
The Allen Institute for Artificial Intelligence has a new idea.
"It’s not such a problem if you don’t need that result right away, but imagine if you had to do that in order to play a game on the highest graphical settings; you want to get those video frames up ASAP, and it’s impractical (not to mention inelegant) to send them off to be resolved remotely.

But improvements to both software and hardware have made it unnecessary, and our ray-traced shadows and normal maps are applied without resorting to distant data centers.

Farhadi and his team wanted to make this possible for more sophisticated AI models. But how could they cut the time required to do billions of serial operations?

“We decided to binarize the hell out of it,” he said. By simplifying the mathematical operations to rough equivalents in binary operations, they could increase the speed and efficiency with which AI models can be run by several orders of magnitude.

Here’s why. Even the simplest arithmetic problem involves a great deal of fundamental context, because transistors don’t natively understand numbers — only on and off states. Six minus four is certainly two, but in order to arrive at that, you must define six, four, two and all the numbers in between, what minus means, how to check the work to make sure it’s correct, and so on. It requires quite a bit of logic, literally, to be able to arrive at this simple result.

But chips do have some built-in capabilities, notably a set of simple operations known as logic gates. One gate might take an input, 1 (at this scale, it’s not actually a number but a voltage), and output a 0, or vice versa. That would be a simple NOT gate, also known as an inverter. Or of two inputs, if either is a 1, it outputs a 1 — but if neither or both is a 1, it outputs a 0. That’s an XOR gate.

These simple operations are carried out at the transistor level and as such are very fast. In fact, they’re pretty much the fastest calculations a computer can do, and it happens that huge arrays of numbers can be subjected to this kind of logic at once, even on ordinary processors.

The problem is, it’s not easy to frame complex math in terms that can be resolved by logic gates alone. And it’s harder still to create an algorithm that converts mathematical operations to binary ones. But that’s exactly what the AI2 engineers did."
And they have something which works, at least in prototype.
"Farhadi showed me the fruits of their labor by opening an app on his phone and pointing it out the window. The view of the Fremont cut outside was instantly overlaid with boxes dancing over various objects: boat, car, phone, their labels read.

In a way it was underwhelming: after all, this kind of thing is what we see all the time in blog posts touting the latest in computer vision.

But those results are achieved with the benefit of supercomputers and parallelized GPUs; who knows how long it takes a state of the art algorithm to look at an image and say, “there are six boats, two cars, a phone and a bush,” as well as label their boundaries.

After all, it not only has to go over the whole scene pixel by pixel, but identify discrete objects within it and their edges, compare those to known shapes, and so on; even rudimentary object recognition is a surprisingly complex task for computer vision systems.

This prototype app, running on an everyday smartphone, was doing it 10 times a second."
I still remember programming in IBM System/360 Assembler. I later discovered that every assembly language I had ever used was itself further compiled to/interpreted by microcode.


Their paper, "XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks", shows that they're using CNN algorithms relying upon fast CPU bit-operators such as 'XNOR and bit-counting operations'. A spin-off company is productising this,

I guess the ultimate would be to forgo leveraging fast CPU/Register operations, implementing the algorithms directly onto the logic gates of a special-purpose chip, an AI accelerator.

Can't get simpler than that .. until quantum computing.

Friday, January 20, 2017

Programming is so .. arid

Helena Cronin wrote:
"Women on average have a strong preference for working with people - hence the nurses and teachers; and, compared to men, they care more about family and relationships and have broader interests and priorities—hence little appeal in becoming CEOs. Men have far more interest in "things" - hence the engineers ... "
I'm sitting here looking at a short recursive function which returns a tree whose nodes are themselves a complicated data-type (Atom-list x Nat x Nat) and I'm trying to do minimax on it. The Lispworks 'Listener' has replaced the Atom lists with hash signs (why?) and the arithmetic min function complains it's getting something weird rather than Real - and has thrown me into the debugger.


The code looks good, although there are so many good-practice projector and constructor functions, cluttering the text.

I sit in frustration, trying to visualise the unwrapping of the computation as the nested mutually-recursive functions execute their combined paths to disaster. Part of me wants to chuck this code away; another part muses that this would mean I was getting even more stupid in my old age.

I review the psychological characteristics required for successful programming.
  • An ability to mentally visualise complex and abstract structures - (heavily g-loaded)
  • A minute attention to detail - the smallest typos kill you
  • Obsessional perseverance when nothing goes right and time-wasting options beckon.
If ever an occupation required what Simon Baron-Cohen calls the 'systemizing brain', this is it.


Update: (Twenty minutes later).

OK, so it was a type clash. I forgot to update a previous sub-function. Needed to add some projectors. So now I have a result.

Shame it's not the one I actually need ...


Some print-journalism stays with you. I regularly recall a right-on female commentator on The Times using her column to weigh in on gender equality in programming. After some shocking statistics she recounts how she asked her daughter if she had considered becoming a programmer. Her daughter looks at her as if she is mad: "Why would I want to do anything as boring as that?"

The daughter's mother is not of course a programmer. She's an opinion-piece journalist, a job where you get to meet and socialise with important people and then write lots of stuff of interest to your like-minded pals.



In case it isn't obvious, I know there are good female programmers. I know there are good female physicists (see here). It's just .. we're talking distributions here - you know, bell-shaped curves? - and the male-female means are noticeably different.

Strive for equality of opportunity; do not expect equality of outcome.

Thursday, January 19, 2017

Scotoma after exercise

So this evening was the second time in two months that I've experienced a scotoma.
"5th November 2016. After weights exercise, a visual illusion to the right of central visual field (both eyes separately). A flickering jagged arc. This is consistent with a scotoma. Same phenomenon with either eye closed. After half an hour it began to move out and enlarge while retreating further into peripheral vision. After an hour it disappeared.

This phenomenon reappeared 19th January 2017, a couple of hours after fairly intense exercise, at 6.10 pm. Visible duration 20 minutes. Started at the centre of the visual field, a flashing-lightning V-shape with vertex at 8 o'clock. Gradually got larger until it vanished past the boundary of the visual field.

Other relevant information: I skipped lunch today as part of my weight-control programme."
After the first time I checked with Wikipedia to discover scintillating scotoma - it looks like this:

although as mentioned I'm seeing a well-defined white jagged shape, growing slowly in size.

I'm not the only one. A google search for "Scotoma after exercise" returns this.
"f you read the title and opened this post, you most likely do suffer from Scintillating Scotoma. You know, start to lose part of vision because of a flashing spot .. . The flashing zig-zag moves to outer edge of eyes, then, after 20 minute or so, goes away.

The common consensus is that is a migraine headache, without the pain.

Anyhow, since I started upping my intensity on the bike trainer workouts, I seem to be getting them after each workout. Could it be clustering since the intensity is greater? I used to get them sometimes after early season races, then as the season wore on and I got used to the intensity, they went away.

I know they're not really an issue, but, hard not go to into a mini-panic each time. "
Wikipedia talks about "cortical spreading depression" as the cause:
"a wave of electrophysiological hyperactivity followed by a wave of inhibition. Spreading depolarization (SD) describes a phenomenon characterized by the appearance of depolarization waves of the neurons and neuroglia that propagates across the gray matter at a velocity of 2–5 mm/min.

SD can be induced by hypoxic conditions and facilitates neuronal death in energy-compromised tissue. SD has also been implicated in migraine aura, where SD is assumed to ascend in well-nourished tissue and is typically benign in most of the cases, although it may increase the probability in migraine patients to develop a stroke."
I don't suffer from migraine, and the effect is purely visual although if pressed, I might admit to an ill-defined feeling of 'headiness' which persists for rather longer.

It seems benign and not a precursor to anything serious so I'm inclined to see it as no more than a warning to ease up a little on high-intensity training, which is currently sending my pulse north of 156, close to the maximum heart rate (207 - 0.7 * 66 = 161) for my age.


Apart from any intrinsic interest, I put this post up to compensate for the lack of engagement many of you feel for my science posts - such as the one posted earlier - fascinating though they are 😐 ..

Quantum theory of the past

Eternalism is not hard to justify.
Yesterday I contemplated my situation and concluded: "This is real, this is now."

Today, when I recollect that scene, I'm inclined to think it was indeed real, and shows the reality of the past (which has not flickered out of existence but is .. elsewhere).

Yesterday, I also thought, "Tomorrow, I will be writing a post."

Today, here I am doing it. For yesterday's me, that shows the reality of the future.
As Frank Sinatra observed, "You can't have one without the other."


For greater conviction, we can appeal to special relativity. As I wrote in a piece for,
"Brian Greene in ‘The Fabric of the Cosmos’ (page 134) considers an alien in a galaxy ten billion light years away, at the edge of the visible universe. Simply by ambulating towards or away from us at 10 mph, the alien’s view of what is happening ‘right now’ on earth swings from 149 years in the past to 149 years in the future."
Still, the block universe is classical and therefore inaccurate. It's necessary to move to quantum theory where, as usual, one needs to take the red pill.

Theoretical physicist Jeremy Bernstein on 'A Quantum Past'.
"In FAPP (For All Practical Purposes) language we have a quantum mechanical system described by a wave function ψ(t), I am  only interested in the time variable.

The wave function obeys a Schrödinger equation with a Hamiltonian H. The formal solution to this equation is ψ(t) = exp(iHt)ψ(0). Throughout I am setting ћ = 1. Thus to recover Ψ(0) from ψ(t) all we have to do is to multiply by exp(-iHt).

Haven’t we then recovered the past? What is all the fuss about? The problem is that there is more to life than the wave function. There are the “observables” which represent what we really want to know about the system. These observables are described by Hermitian operators A. B. C and so on. We can expand ψ in a sum over the orthonormal eigenfunctions of any of these operators. The coefficients in the expansion are related to the probabilities that in a measurement the system will be found to have one of these eigenvalues. This is “Born’s rule” and in FAPP it must be assumed.

To find which of these eigenvalues the system actually has, we must perform a measurement. Stripped to its essence the apparatus that produces this measurement projects out from the sum of eigenfunctions one of them.

After the measurement the rest of the terms in the sum disappear. Using the term of art, the wave function “collapses”. It is at this point that we lose our capacity to reconstruct the past.

Projection operators are singular. They do not have inverses. All the king’s horses and all the king’s men cannot put the wave function back together again.

It was von Neumann in the early 1930’s who first noted that in FAPP mechanics there were two kinds of processes. There were processes that could be described by a Schrödinger equation and there were measurements which could not. He did not, as far as I know, comment on what this implied for retrodiction.

A case in point is an electron described by a spherically symmetric Schrödinger wave. If this electron strikes a detector is does so at a place - a spot. After this happens all trace of the spherically symmetric wave function vanishes.

I have certainly not made a careful search of the literature but among the founding fathers of FAPP I can come up with only two references that deal with the matter of the quantum past.

One is Heisenberg and the other is a paper by Einstein, Richard Tolman, and Boris Podolsky, “Knowledge of Past and Future in Quantum Mechanics” which they wrote in 1931 when Einstein was spending time at CalTech."
Bernstein talks about these two references, and then discusses where he thinks the problem resides, and what is to be done.
"It seems to me that any interpretation of the quantum theory that addresses this [the problem of wavefunction collapse] must have the feature that measurements are simply just another interaction like the rest.

Von Neumann’s notion that there were two classes of interactions one whose time evolution could be described by a Schrödinger equation and one of which couldn’t, has to be abandoned.

I will discuss two proposals for doing this each of which has its adherents and its detractors. On the one hand I am going to discuss what I will call “Bohmian mechanics” a term which David Bohm, who invented this approach , apparently did not like. As far as he was concerned, he was just doing quantum mechanics but in a different way. However nearly everyone else calls it Bohmian mechanics - so will I.

On the other hand, I am going to discuss the “decoherent history” interpretation which Murray Gell-Mann and Jim Hartle have done the most on. Sometimes this is called the “many worlds” interpretation, but not by them. I think that the term “many worlds” is misleading. As far as we know there is one world, the one we live in."
I'm not a fan of “Bohmian mechanics” and insofar as any quantum ontology works for me, it has to be "Many Worlds" (Sean Carroll explains why).

So I'll skip over Bernstein's description of Bohm's view of the quantum past and quote his take on “decoherent histories”.
"In the "Many Histories Interpretation" what indeed is history?

At first sight this might seem to be obvious. All we have to do is to run the chain backwards.

Yes this gives one history but there are others, possibly very many others. The reason is that if all we know is the present state vector there are many paths by which we could have arrived there depending on which initial state vector we started from. We have no way of knowing this from the data we have at hand.

Let us take an example discussed by Hartle - the Schrödinger cat (I can’t resist noting that when I spent an afternoon with Schrödinger in his apartment in Vienna there was no cat). In any event this unfortunate feline is put in a box that contains a capsule of poison gas and a sample of uranium. The capsule is triggered so that if the uranium has an alpha decay, the alpha sets off the trigger and the unfortunate feline expires.

After a time interval we open the box and happily the cat is alive. It could, according to the many history approach have arrived at this state in two ways. The initial state might have been a cat alive state or it might have been a coherent sum of a cat alive and a cat dead state. From the presence of the living cat we cannot decide.

The vision of the past given by the decoherent history interpretation and the Bohmian seems radically different. In Bohmian mechanics we could in principle follow all the cat molecules backwards in time and arrive at one and only one past.

I don’t know how you feel, but the ambiguity of the past makes me queasy. It might be entertaining to imagine that in an alternate past my grandmother who was born in a Polish stetl could have been Eleanor Roosevelt.

I readily accept that these pasts to not communicate but there seem to be too many of them from the point of view of economy. A trip to a barber wielding Occam’s razor seems warranted.

In any case when it comes to quantum pasts, as Duke Ellington taught us, “Things ain’t what they used to be.”
Perhaps Bernstein wrote this short paper just for that final joke at the end?

To summarise, if you take quantum theory seriously and you take eternalism (the block universe) seriously, then it seems that the past is as indeterminate as the future.


How does this relate to the 'low entropy in the past' idea used to explain the 'arrow of time'?

Since quantum theory is consistent with the second law of thermodynamics, a picture emerges of backwards branching towards (superpositions of) Big Bang variants.

My ex-colleague Roy said as much in this comment, on an earlier post devoted to the MWI. See here for more.

Wednesday, January 18, 2017

The Meaning of Life

In the beginning we should all be physicists. As Sean Carroll has repeatedly pointed out, the laws underlying the physics of everyday life are completely understood.
"Many people resist the implication that this theory is good enough to account for the physics underlying phenomena such as life, or consciousness. They could, in principle, be right, of course; but the only way that could happen is if our understanding of quantum field theory is completely wrong.

When deciding between “life and the brain are complicated and I don’t understand them yet, but if we work harder I think we can do it” and “I understand consciousness well enough to conclude that it can’t possibly be explained within known physics,” it’s an easy choice for me."
Carroll summarises our complete theory of the universe in one equation. As he points out, "No experiment ever done here on Earth has contradicted this model."

For the physicist, life is merely a parameter subspace in the evolution of the universal wavefunction. I'm tempted to mention cats. The physicist is professionally a psychopath. So note that the next time a famous physicist expresses an opinion on politics or public policy. They are talking outside their discipline.


It's not wrong to start with physics, how could it be? But we seek more enlightenment from biology. At least the subjects we study there are actually living.

The title of this post is the meaning of life. Did you see the word 'human' anywhere?

Consider the plants and animals, the germs and fungi, occupants of this planet for four billion years. Darwin gave us the answer - the meaning of life is to survive and have reproducing progeny. All those ancestral entities which didn't 'get that' were eliminated from reality.

And for most creatures, there is no more meaning than that. If they were to waste their time and energy on doing anything else, they would be outcompeted and removed from the gene pool.


On to sociology. It seems unbearable to a cultured person in the twenty first century that the meaning of their life is reduced to the number of child-bearing children they manage to create. This has thrown reductionist biologists into confusion.

I once read that the reason Catholic priests were celibate was some variation of the alleged 'gay uncle' phenomenon. Utterly delusional.

To state the obvious, humans are social creatures and large-scale agrarian societies rapidly outcompeted hunter-gathers. Almost all contemporary humans are the descendants of individuals who successfully adapted to living in large-scale cooperative societies.

Until the recent advent of contraception, reproductive success was strongly linked to social status. Social status in large-scale societies selects for social accomplishment: it helps to be good at something people value, and not to be a muppet.

A desire simply to breed is not really enough: your neighbours can get lethally irritated.

It's surprising how many people delude themselves that being an actor, a rock star, a politician or a top executive or even intellectual is not about getting the girl (or, if a girl, getting a better class of admirer).

What do you think?


Social creatures have complex life histories, no longer purely individualistic. We all recall "Haldane famously joking that he would willingly die for two brothers or eight cousins."

Those celibate Catholic priests are using prosocial psychological drives to inhibit their more primal reproductive imperatives. No society could cohere without such inhibitory mechanisms and since the ages of rape and pillage of outgroups, there has been selection for male self-control.

As ever, there is variation in the population and a celibate occupation strongly selects within that.


Some conclusions. The biologists are right: for humans the meaning of life is in some extended sense to have offspring. A society which fails to reproduce abolishes itself and vanishes from reality. The extended sense means that we could profitably devote our lives to aiding close kin, or even those whose social-solidarity benefits our kin (the foundation of reciprocal altruism).

We all benefit from the continuing integrity of our societies with their high carrying capacities.

It's OK to be a celibate, rule-abiding priest .. really. Even if you are professionally confused about the meaning of life.

Tuesday, January 17, 2017

I win my first noughts and crosses game!

Much excitement amongst regular readers here as to when I will play my first game against my AI noughts and crosses system.

Well, it just happened and I won playing X.

Here's the game-transcript.

CL-USER 22 >  (play-game 'X *initial-board* 'human 'rand-player)

Next to play:  X

     0 | 1 | 2
     3 | 4 | 5
     6 | 7 | 8

"Please enter an available number: " 8

     0 | 1 | 2
     3 | 4 | 5
     6 | 7 | X

--- Computer playing O now moves ---

     O | 1 | 2
     3 | 4 | 5
     6 | 7 | X

"Please enter an available number: " 6

     O | 1 | 2
     3 | 4 | 5
     X | 7 |

--- Computer playing O now moves ---

     O | 1 | 2
     3 | 4 | 5
     X | O | X

"Please enter an available number: " 2

     O | 1 | X
     3 | 4 | 5
     X | O | X

--- Computer playing O now moves ---

     O | 1 | X
     O | 4 | 5
     X | O | X

"Please enter an available number: " 1

     O | X | X
     O | 4 | 5
     X | O | X

--- Computer playing O now moves ---

     O | X | X
     O | 4 | O
     X | O | X

"Please enter an available number: " 4

Win for X

     O | X | X
     O | X | O
     X | O | X

Perhaps I shouldn't get too excited: here's my opponent - noting that ...
  • the parameter m is the 'mark' -  X or O
  • b is the evolving board, eg:  '(X 1 O O 4 5 X 7 X).

(defun rand-player (m b)
   (let* ((remaining-numbers (remove 'O (remove 'X b)))
          (n                 (length remaining-numbers))
          (random-index      (random n))  )
      (nth random-index remaining-numbers))  )

The considerably more difficult task of getting the tree-search system debugged I'll leave for tomorrow!


You may have noticed my poor fourth move. OK, I was toying with it.


Update Wednesday 18th January.

To Roy's comment below: yes, it's random. I used it to test the game-playing framework and user interface. It is rather spooky and apparently cunning to play against - hard to second-guess 😏.

Today I've been working on the tree-generation algorithm and minimax. Probably a session or two away from the first capable AI player.

The liberal colonisation of western elites: why?

The British Prime Minister, Theresa May, gave a defensive speech on Brexit today and was duly patronised on the talk shows by Remain liberals in their characteristic, languid upper-class drawl.

I wonder yet again why western capitalist elites seem dominated by liberals. The phenomenon is too widespread to be explained by purely national histories. From the Clintons to the Camerons to the Merkels the caste is pervasive.

I turn again to Jonathan Haidt and Moral Foundations Theory, quoting him from this previous post.
"Think of it like this: Evolution gave all human beings the same taste receptors — for sweet, sour, salt, bitter, and umami (or MSG) — but cultures then create unique cuisines, constrained by the fact that the cuisine must please those taste receptors. Moral foundations work much the same way. The six main moral taste receptors, according to MFT, are:

Care/harm: We feel compassion for those who are vulnerable or suffering.

Fairness/cheating: We constantly monitor whether people are getting what they deserve, whether things are balanced. We shun or punish cheaters.

Liberty/oppression: We resent restrictions on our choices and actions; we band together to resist bullies.

Loyalty/betrayal: We keep track of who is "us" and who is not; we enjoy tribal rituals, and we hate traitors.

Authority/subversion: We value order and hierarchy; we dislike those who undermine legitimate authority and sow chaos.

Sanctity/degradation: We have a sense that some things are elevated and pure and must be kept protected from the degradation and profanity of everyday life. (This foundation is best seen among religious conservatives, but you can find it on the left as well, particularly on issues related to environmentalism.)

"As with cuisines, societies vary a great deal in the moralities they construct out of these universal predispositions. Many traditional agricultural and herding societies rely heavily on the loyalty, authority, and sanctity  foundations to create rituals, myths, and religious institutions that bind groups together with a strong tribal consciousness.

"That can be highly effective for groups that are often attacked by neighboring rivals, but commercial societies (such as Amsterdam in the 17th century or New York City today) are far less in need of these foundations, and so make much less use of them.
Jonathan Haidt and his co-workers found that modern political tribes project differently onto these six dimensions.

Source: Wikipedia

Liberals are motivated almost entirely by the Caring dimension and Fairness, which they tend to interpret as equality of outcome. We might also observe that liberals tend to value self-control and eschew displays of physical aggression. Playing by the formal rules demands both.

Conservatives score less on Caring, stronger on Fairness which they tend to interpret as equality of opportunity, and are supportive of their own institutions via Groupishness.

Libertarians are like Mr Spock in Star Trek: cool and unemotional, detached and strongly autonomous as shown by their focus on Liberty - freedom from interference.

Modern capitalism in its full, globalist, multi-ethnic majesty is a kind of ecosystem-filter for these different moral types.

As I mentioned in the previous post about the current state of Russia, capitalism works best - as a network of anonymous economic relationships distributed over time and space - with a system of rules in place to enforce property rights and ensure transaction-integrity.

The point about rules is that they should be impersonal, not unduly biased by power and stable. In turn this requires an enforcing state which upholds associated values. These, however, can easily conflict with common psychological drives.

  • Universal abstract rights can undermine particularist family, kin and ethnic interests
  • Formal process requires the replacement of arbitrary force to resolve disputes
  • A safety net is needed to keep the masses from toppling the elites.

In MFT terms, this prioritises Fairness and Caring, and undermines the kinds of particularism expressed in the three Groupish dimensions. In theory Libertarian Freedom might suit an idealised capitalism but in reality its disengaged state-model is just ineffective and dangerously uncaring.

If these points carry a ring of truth then it's no surprise that liberals are so strongly selected for when populating elite positions, particularly within the state apparatus and organs of elite ideology like the media and social sciences.

When Groupish, 'populist' conservatives come to power, propelled by disgruntled in-groups, liberals react with fear and loathing seeing the threat to their carefully-crafted model of atomised society composed of formally-equal individuals.

They are right to do so: as discussed in the previous post, authoritarian leaders privilege their supporters while demonising others as out-groups. This can end in tears.

But .. the tenets of liberal ideology are so tendentious, so self-serving, and often so plainly delusional and damaging to the social fabric that sometimes, something just has to be done.

I think Theresa May clearly understands this, which makes her deference to the liberal establishment besieging her so depressing.


Is it possible to run a free, productive, dynamic, inclusive capitalist society without its natural ideology of social atomisation, blank-slate equality and hyper-individualism?

You might want to read this.

Monday, January 16, 2017

"Nothing Is True and Everything Is Possible"

Dominic Cummings said that no-one should 'pontificate on Putin’s mafia government' before reading this book.

Amazon link

Here are some excerpts from The Guardian's review.
"Pomerantsev, born in the UK to Russian émigré parents, spent almost a decade in Moscow working as a TV producer, making documentaries and reality shows for Russian audiences. He arrived in the early 2000s, in the midst of an oil boom that brought a measure of prosperity to many and huge wealth to a select few, creating a tidal wave of glitz and extravagance, especially in the capital.

Nothing Is True and Everything Is Possible is an entertaining if at times bleak chronicle of these years, depicting a world “where gangsters become artists, gold‑diggers quote Pushkin, Hells Angels hallucinate themselves as saints”.


"We also meet Vitali Dyomochka, a Siberian hoodlum turned cineaste. Dissatisfied with the quality of crime dramas on Russian TV – “it was all fake” – he took to making his own series, giving starring roles to several of his henchmen. There were no scripts, stuntmen or makeup: “all the blood you saw on the screen was real”, Pomerantsev writes, adding that “the guns and bullets were all real, too; when they filmed a shoot-’em-up in a bar the place was wasted”.

Djomochka allegedly got the series broadcast by getting his goons to threaten local TV stations; needless to say, it was a huge hit."


"Putinism itself is built on an ideology of all ideologies at once: liberalism; nationalism; conservatism; Orthodox tradition; an “anti-hegemonic” foreign policy.

As Pomerantsev points out, one key to the success of this new authoritarianism is that “instead of simply oppressing opposition, as had been the case with 20th-century strains, it climbs inside all ideologies and movements, exploiting them and rendering them absurd”.

The clearest example of this is the creation of a political system that has the appearance of democracy – regular elections, multiple parties, a free media – without any of the substance: the elections are rigged; the parties are all under the president’s control; the media do what their owners tell them, and the owners obey the Kremlin.

It’s this mismatch between form and content that has earned the Putin regime the name “virtual” or “imitation democracy”.


"Corruption, which affects almost every sphere of life, from major corporations to visits to the doctor, has taken root in the vast juridical twilight zone that emerged during the 1990s, and which the authorities have an active interest in maintaining.

Among the many other stories Pomerantsev follows is that of Yana Yakovleva, a businesswoman suddenly arrested for “illegally” trading in chemicals she has been selling perfectly legally for years. She’s the victim of “state raiding”, a widespread phenomenon in which government officials put the squeeze on businesses – finding owners to be in breach of fire-safety regulations, say, or in arrears with taxes they never owed; or, as in Yakovleva’s case, charging them retroactively with having broken a brand-new law.

She spends several grim months in prison before coming to trial; astoundingly, she wins and is released – but less because of her actual innocence than because her campaign has made her a useful pawn in a battle between two powerful Kremlin factions. “To make something happen in Russia,” Pomerantsev concludes, “you have to be both valiant protester and Machiavellian.”

"By the summer of 2010, when Moscow was wreathed in suffocating smoke from peat-bog fires, Pomerantsev had become increasingly frustrated by the constraints he faced in his work. His channel only wanted “positive” stories, and there were clear limits on what could be said and shown: he had to cut all the high-level politics out of Yakovleva’s story, for instance.

But on returning to the UK, he encountered the beneficiaries of the system he thought he had left behind – the oligarchs and bureaucrats turned businessmen who have siphoned wealth out of Russia and into London, the gilded post-Soviet youth who spend their time surrounded by their peers in an exclusive network of Mayfair nightclubs."
The words we use are value-loaded. In most historical societies, what we call corruption is the normal operation of 'big man' factional political economy. The Romans would have understood.

There is an economics literature on corruption, conceptualising it as rent-seeking and adding gratuitously to transaction costs. However, liberal democracies also have expensive mechanisms for arbitrating differences and setting priorities. The difference is more about stability and predictability: in a sense, liberal democracies have nationalised corruption and rebranded it.

The main thing I get from Peter Pomerantsev's book is that in today's Russia, society-wide trust does not exist and most state institutions do not work as such, being merely facades for powerful and arbitrary interest groups.

This so accords with history and natural human psychology that it jarringly reminds us again just how unlikely the relatively non-corrupt western model of bourgeois democracy actually is. The rule of law and the effectiveness of institutions requires elites to inhibit their emotions and personal drives, subordinating them to the rules.

Who knows how much of this is cultural and how much genetic? What we do know is that corrupt, clannish societies have enormous historical inertia, southern Italy being a case in point. The Russians haven't had much luck with their history in this regard, unfortunately.

As a root-and-branch picture of modern Russia, I'm with Mr Cummings. We should not be seeing Putin's state as a 'more vigorous capitalist society with more traditional values and less politically-correct nonsense'.

No, Russia is more like a capitalist version of Game of Thrones.

As a piece of writing, "Nothing Is True and Everything Is Possible" is mostly fascinating, but could have benefited from the attentions of an editor to cut out the occasional longueur.

Sunday, January 15, 2017

A natural (or green) burial

For the avoidance of doubt, Clare has always insisted on a green burial. A shroud or decomposable coffin, a small tree - a rowan would be ideal - planted on top.

We had heard that Wells Cemetery has a corner devoted to natural burials so we strolled down there yesterday. The cemetery is quite pleasant (pictured below) but there are no woodland plots, just a field adjacent to the main site reserved for 'lawn burials'. Not at all what she had in mind.

Wells cemetery

Apparently I am to be buried in an adjoining plot.

I'm good with that - it's not really my problem anyway. Perhaps I could have a yew? A service to future bowmen?

The headstones were in varied states of disrepair. You should plainly avoid anything where the stonemason drills tiny holes and then pops in different coloured stone flakes to spell out the legend. These inevitably fall out over the years. The best lettering is chiseled then painted.

The green/natural burial protocol forbids headstones. The idea is, over time, to create a new wood. There would be records in the office of course. And perhaps GPS coordinates on the Internet.

As we wandered amongst the graves, I suggested I could mark her eventual spot with a holographic projector. As the visitor approached, a projection of Clare would emerge which would greet and say a few words. It could be solar-powered for maximum greenness.

And then I thought: folk would be wandering through this graveyard assaulted by a cacophonous swarm of the long-dead springing into existence, like chuggers from hell.

My second thought was augmented reality. In the future, everyone will inhabit a reality painted with tailored virtualized overlays. This will let the dead speak only to those interested, and the dataset will live in the cloud, not on a weatherproof server under that rowan tree.

But by then, synthetic versions of ourselves will inhabit the datasphere, interacting on demand. It will seem like we had never died.

No-one will care about those spectral arboreal presences in Wells cemetery.


Note: these funeral arrangements are entirely speculative at time of writing.

Saturday, January 14, 2017

The long road to Go

Some people fill their dead time with shoot-'em-ups on their phones or tablets.


I'm not interested in chess, but the game of Go has vaguely intrigued me. A brief search led me to the British Go Association and their recommended starter Android app, 'GOdroid'.

GOdroid screen shots

This app is for complete beginners, played on a 9x9 board on my Nexus 10.
I lost the first game.

I lost the second game more comprehensively.
OK, so this is not as easy as it appears. Time for some assistance. I turn to Amazon.

Amazon link

The book arrived this morning and looks promising.


No jokes please. I just wrote a post about building an AI system to play noughts and crosses. I am not planning to take on AlphaGo!

The long road to noughts and crosses

Clare innocently asks why, after five afternoons of programming, we still don't have a program which can play noughts and crosses.

The game seems pretty trivial. Just arrange for the program to play in the corners, then complete 'three in a row' if it can. Probably the system will never lose and will win quite often.

Yes, I could have done that.


I explain that in the version I'm working on, the computer starts with no information about noughts and crosses and no built-in ideas about how to play.

Faced with a blank board there are nine possible places to put its first 'X'. It has no reason to prioritise one over another, so it therefore generates nine possible board positions.

For each of these, it doesn't know where its opponent will place its first 'O', so that's eight possibilities it has to consider. It's now juggling a tree of depth two with 72 options.

The process repeats. No win or loss can occur until the fifth move (3 'X's and 2 'O's) by which time the tree has branched to 15,120 possible boards. Most positions are still in-play though.

Lacking a priori insight, the program has to apply a combination of things. Looking ahead, say, to depth 5, it can score future boards where it wins very positively, and positions where it loses very negatively. Ongoing positions it has to judge more heuristically, creating a 'how good is this for me?' score.

Then it has to somehow back-up these terminal scores which are maybe five or more moves out, bringing them back so that they give guidance for the very next move.

Effectively, the computer is asking itself: what next move maximises my chances to being on a road which ends up with a win, despite my opponent trying to thwart me on every turn?

This is the famous 'minimax' algorithm.

So plenty of apparatus to put together before the first cross gets entered in the first game.

I'm almost there.


Once the apparatus of tree generate-and-test has been written and debugged, it can easily be extended to interactions which are less trivial than noughts and crosses.

Automated Theorem Provers and Expert Systems also build trees and evaluate nodes against success criteria. The difference is that the trees they build are not adversarial, so no minimax.

But a conversation planning system is more like a two-player game.

My goal of a smarter chatbot with inference and conversational direction is still very firmly in my mind .. although it's a way off.

Friday, January 13, 2017

My weight training programme (video)

Here's an 8 minute video where I describe my weight training routine, which starts with an exercise bike HIT warm-up followed by floor exercises and free weights (dumbbells).

As of Saturday morning, 23 people have viewed this post .. and 3 people watched the video. What were the other 20 doing?

Some mistakes in my commentary:
  • the heaviest dumbbells are 14 kg each, not 14.5 kg 
  • the first weight routine is the 'bent over row' (not 'back over row') 
  • it's the 'bicep curl', not 'bicep crunch'.
The order I do exercises varies, there's no rule which says 'heaviest weights first' - in fact that's rather deprecated. Sometimes I start with arm flaps (not the technical term).


Credits to Clare for the camera work.

Thursday, January 12, 2017

Clare bought a cactus

From Lidl, actually

Proud of herself, she asked me to step into the kitchen:
"I've bought something new."
I gazed blindly around in confusion, my eyes eventually alighting on the pepper pot.
"You bought some new pepper?"
The new cactus was on the window ledge. I had been staring right at it.

It may be called 'Blindsight'.


She joked that if she ran away and replaced herself with another woman, it might be .. well, two weeks before I realised.

John Cook on automated theorem provers


Mathematician and computer scientist John Cook restates the orthodoxy on automated theorem proving (ATP).
"When I first heard of automated theorem proving, I imagined computers being programmed to search for mathematical theorems interesting to a wide audience. Maybe that’s what a few of the pioneers in the area had in mind too, but that’s not how things developed.

The biggest uses for automated theorem proving have been highly specialized applications, not mathematically interesting theorems. Computer chip manufacturers use formal methods to verify that given certain inputs their chips produce certain outputs. Compiler writers use formal methods to verify that their software does the right thing. A theorem saying your product behaves correctly is very valuable to you and your customers, but nobody else. These aren’t the kinds of theorems that anyone would cite the way they might site the Pythagorean theorem. Nobody would ever say “And therefore, by the theorem showing that this particular pacemaker will not fall into certain error modes, I now prove this result unrelated to pacemakers.”

Automated theorem provers are important in these highly specialized applications in part because the results are of such limited interest. For every theorem of wide mathematical interest, there are a large number of mathematicians who are searching for a proof or who are willing to scrutinize a proposed proof. A theorem saying that a piece of electronics performs correctly appeals to only the tiniest audience, and yet is probably much easier (for a computer) to prove.

The term “automated theorem proving” is overloaded to mean a couple things. It’s used broadly to include any use of computing in proving theorems, and it’s used more narrowly to mean software that searches for proofs or even new theorems. Most theorem provers in the broad sense are not automated theorem provers in the more narrow sense but rather proof assistants. They verify proofs rather than discover them. (There’s some gray zone. They may search on a small scale, looking for a way to prove a minor narrow result, but not search for the entire proof to a big theorem.) There have been computer-verified proofs of important mathematical theorems, such as the Feit-Thompson theorem from group theory, but I’m not aware of any generally interesting discoveries that have come out of a theorem prover."
However, if you widen the discussion to automated deduction as we see in expert systems, planning systems and pretty much everything Google is trying to achieve, inference is everywhere.

Dominic Cummings on the media

"High prestige pundits and editors yield great power over the stories told (and have far more power over politicians like Cameron, unfortunately, than they realise) but the field is not based on real expertise. Fields dominated by real expertise are distinguished by two features: 1) there is enough informational structure in the environment such that reliable predictions are possible despite complexity and 2) there is effective feedback so learning is possible.

Neither condition applies generally to politics or the political media. In the most rigorous studies done, it has been shown that in general political experts are little better than the proverbial dart throwing chimp and that those most confident in their big picture views and are most often on TV  – people like Robert Peston, Jon Snow, and Evan Davis – are the least accurate political ‘experts’ (cf. HERE).

We know that cognitive diversity is vital for political accuracy yet almost all political institutions and the media – including the dominant people at Newsnight, the Economist, the FT, and Parliament – are actually remarkably homogenous, as discussed above, and they herd around very similar ideas about how the world works. Scientists and entrepreneurs in particular are almost totally excluded from political influence.

There is no structure to hold them to account either internally or externally so, like anyone when not forced to be rigorous, they fool themselves."
From Dominic Cumming's blog. It's a very long piece, but written with intelligence and a scathing disregard for bubble-think. Quite educational.

The scientists allowed on TV normally express the liberalism of academia, and are usually interviewed outside of their field of expertise, if hard science.

A further extract:
"Polls show that better educated people are less likely to have accurate views about the science of evolution and genetics (their desire to send moral signals suckers them into believing fairy tales).

The conformity of the educated is in some ways a good thing – most obviously, a basic consensus about things like not killing one’s domestic opponents that is extremely unusual historically. But it has many bad effects too.

There is a collective lack of imagination which makes the system very susceptible to disastrous shocks. They share a narrow set of ideas about how the world works which mistakes their own view as the only possible sensible approach.

They are always writing about how ‘shocking’ things are to them – things that never were as low probability events as they imagine.  They can’t imagine something like Stalin deliberately creating a famine or deliberately murdering millions.

They tell themselves that Hitler will be ‘more sensible in power’ and ‘engagement’ is the right path. Western liberals (like Clinton and many pro-euro campaigners) and conservatives (like Bush) talked of relations with Putin as if he is a normal western politician rather than an ex-KGB mafia overlord with views very far from western liberals.

They tell each other ‘I can’t imagine President Trump, it just can’t happen’. Many conservatives are now telling themselves that they should not take Trump too literally but that too is a failure of imagination – his character is clear to those unblinded by gang mentality and he will govern in character."

Via Steve Hsu - Mr Cummings has now been added to my blog list, to your right.