Logistic discriminant analysis (= neural nets)

If you were trained (as I was back in the 1980s) on Good Old Fashioned AI (GOFAI), the technical background you studied consisted primarily of formal logic and discrete maths, implemented by symbolic programming languages such as Lisp and Prolog.

Meanwhile, the minority neural net tendency used statistical techniques and differential equations.

It's hard to imagine two more discordant cultures.

In these days of the overarching victory of the latter, I was interested to read the following from the excellent overview book, "Statistics: A Very Short Introduction" (David J. Hand), page 104.

"In fact, logistic regression can be regarded as the most basic kind of neural network."

I confess I had never thought of neural networks as simply a mainstream statistical classification tool.

Wikipedia has two articles on the subject: "Discriminant function analysis" and "Linear discriminant analysis" along with "Logistic Regression".

Something to look at further.

---

Marr's Tri-Level Hypothesis

David Marr was one of my heroes when I was an active AI researcher. Outside of computer vision I think he is mostly forgotten now (he died tragically early), but he said something important about methodology in AI research when many around him were writing programs that did vaguely cool stuff while claiming they were advancing science.

Marr distinguished three levels of analysis.

1. computational level: what does the system do (e.g.: what problems does it solve or overcome) and similarly, why does it do these things


2. algorithmic/representational level: how does the system do what it does, specifically, what representations does it use and what processes does it employ to build and manipulate the representations


3. implementational/physical level: how is the system physically realised (in the case of biological vision, what neural structures and neuronal activities implement the visual system).

His terms are not great (he was trained as a biologist): his computational level is really the theory of system behaviour in the environment of interest; his second level might be better described as an architectural level, describing the various ways the system's capabilities could be decomposed into subsystems and their inter-relationships; finally comes the issue of specific processing mechanisms and algorithms.

It's still common to see people waving the banner for one of these elements of analysis, while ignoring the others. Only confusion results.

Neural networks are an architecture. As currently understood and built, the term denotes a distributed, connected computational architecture well-suited to a certain class of problems, namely pattern recognition, feature extraction and classification.

This is a proper subset of the cognitive problems animals (including humans) have to solve in the world.

Artificial neural networks today consume Terabytes of training data, solving recognition/ classification problems of interest to Google, Facebook and the like.

I am reminded of the man who has a hammer.

The easy wins will fade away well before they achieve the purported Holy Grail of Artificial General Intelligence.

I hasten to add the obvious: in any event, you and I are considerable more than arid and cerebral AGIs.

Last words

"It's going to be weird, not being alive."

"But of course, it won't be anything."

---

I'm collecting possible epitaphs.

A backup plan for Mars

One idea for terraforming Mars involves crashing kilometre-scale ice-comets onto the planet's surface.

A comet impacting on Mars

That might kick-start a colony, but the surface gravity of Mars isn't sufficient to retain a reasonable atmosphere and water over thousands of years.

We could be optimistic and hope our descendants will fix this problem, but why not a little insurance?

I propose we choose some large ice-comets, preferably in the outer system where gravitational perturbations from the likes of Jupiter and Saturn are minimised, and adjust their orbits to intercept Mars in, say, 5,000 years time.

The downside: if there's a civilisation on Mars at that time which lacks technological sophistication, they're going to have some real bad days.

The upside: some would survive. But if Mars loses its water and oxygen, they're all toast anyway.

---

I checked. The JPL ephemeris program can currently model the position of the planets out to 3,000 AD. Their positional accuracy for Mars is around plus or minus one kilometre.

I think we can do this, people.

You know they'll thank us in the end.

Power starvation

Isle of Wight Music Festival

Ever sat in a tent watching the battery on your phone or tablet die, with no mains power?

Such was my experience in Minehead last weekend. So I bought this:

RAVPower 26800mAh 3-Port 5.5A iSmart Output
Portable Charger Compact Power Bank External Battery Pack

It performed flawlessly, quickly recharging Clare's iPad, my Nexus 6 phone and my Nexus 10 tablet.

It then took around 14 hours overnight to recharge the Power Bank from empty, using the iPad's 5V 2.1A USB mains-charger.

---

Tablets charge at 5V 2.1A from a USB mains-charger while for phones it's 5V 1A.

You can charge a tablet from a 1A phone charger - but it takes twice as long; you can charge a phone from a 2.1A tablet charger - the extra current capacity makes no difference and can't be used by the phone - but it's perfectly safe.

My Nexus 6 phone has a 3220 mAh battery, my Nexus 10 tablet has a 9000 mAh battery while Clare's iPad 2 has a 6,944mAh battery.

At 1 amp you would predict that the phone would take just over three hours to charge, the iPad would take seven hours and the tablet 9 hours.

This is why it makes sense, tablet-wise, to invest in a 2.1 amp charger like this, rather than just reusing old phone chargers, so as to halve these times..

AmazonBasics Wall Charger with USB Outlet (2.1 Amp Output)

---

Due to various inefficiencies, you should multiply the device charging times above by up to an additional 50% (assuming you were charging from empty - but recall you are advised not to let the charge drop below 30% for long-term battery-health reasons).

Follow your passion

From Marginal Revolution.

... or there's always dentistry ...

The Bishop's Palace, Wells

A picnic lunch today at The Bishop's Palace, Wells.

The Pilgrim statue - inscription below

Here's a close-up of The Pilgrim's face .. it rather reminds me of ..

China Mieville

Wells Cathedral from The Bishop's Palace garden

If you know what this flower is, could you tell me in the comments?

Update: thanks guys, it's an Allium.

Your author and his wife

"The Last Days of New Paris" has an Amazon page, but no-one seems to know when it will be available. August?

"June 1940 following Paris’ fall to the Germans, the villa of Air-Bel in Marsailles, is filled with Trotskyists, anti-fascists, exiled artists, and surrealists.

"One Air-Bel dissident decides the best way to fight the Nazis is to construct a surrealist bomb.

"When the bomb is accidentally detonated, surrealist Cataclysm sweeps Paris and transforms it according to a violent, weaponized dream logic."

You had me at 'surrealist bomb'.

Robert Heinlein on Democracy

Robert Heinlein from "The Notebooks of Lazarus Long, Time Enough for Love".

"Democracy is based on the assumption that a million men are wiser than one man. How's that again? I missed something. Autocracy is based on the assumption that one man is wiser than a million men. Let's play that over again too. Who decides?"

In Robert Harris's novel "Archangel", I recall a scene after the death of Stalin where the secret police chief, Beria, tries to manipulate his colleagues to ensure his own succession. He has a cunning plan but it does not end well.

A politburo colleague observes that Beria, like many very smart people, assumes that more average people will never figure out what he's up to. But that's to mistake the nature of average folk: it's not that they can't get there - they can; it just takes them a little longer.

That's why I'm not that worried by the left-hand side of the bell-curve getting the vote.

The not-so-super-smart are usually not au fait with the niceties of public policy options, but they tend to be pretty expert on whether things are working out well for themselves. And in the end, that's a recipe for a stable and productive society: keep the punters reasonably content.

If you disenfranchise swathes of the population, you reserve political power to the vested interests of those who retain the vote. Naturally public policy will be captured (in the economics sense of regulatory capture) by such vested interests.

In the long run, this is seldom a good thing.

Sentimental, moralistic platitudes about the 'ethical superiority of democracy' don't do it for me.

No, I'm strictly functionalist. Democracy should be supported because it's the only system which provides a regular and universal protocol to "kick the bastards out".

---

Note: if you want Robert Heinlein's considered view as to how to make democracy work, you should read "Starship Troopers".

Tl; dr - voting should be restricted to those prepared to defend society, so no votes for parasites. Basically you do your military service, indicate your willingness to put your life on the line for your country .. and then you get to vote.

A book abandoned

Cool cover art.

 Amazon link

I read Scott Alexander's extensive review and tried to cancel but too late: the very next day the Amazon parcel arrived. Alexander's review is an easier read than the book itself, which is written in a 'flat' academic style akin to a succession of Wikipedia mini-articles.

I wouldn't have minded if the content had been as described in the back cover quotes ('mind-blowing', 'fascinating', 'stimulating') .. but the ideas are relatively pedestrian, what you would expect if you thought about it a bit; many of Hanson's assertions are plausible without being that compelling.

So after a while of mentally chewing on cardboard, I abandoned.

If a society of brains-emulated-in-hardware-and-software is truly your thing, then it's a lot more fun reading Greg Egan's sublime Permutation City.

---

Back yesterday from a couple of days camping on the heights above Minehead where we also took in the Civil War reenactment at Dunster Castle.

Clare fronting the view of Minehead from the campsite

Dunster Castle: the women spin and cook behind the cannon

The pikemen and musketeers assemble

I'd anticipated a mass mêlée, a pitched battle of The Sealed Knot - but what you see in the pictures was what we got.

The fighting persons in close-up

I especially liked Ginger Baker (above) who gave us a one-person marching drum solo after the obligatory team musket discharges and pike-drill.

The aircraft carrier in total war

Interesting discussion about that exciting new weapon, the US navy railgun. A lot of feedback in the comments, mostly skeptical, about new 'wunderwaffe'.

I think it boils down to this: against a third-world opponent, nothing works like a carrier group for overwhelming global force projection; conversely, if you're fighting a first-world state with up-to-date weaponry, your carrier group is toast.

First-world state-on-state warfighting doctrine doesn't seem to have kept up with drone & missile technologies. Drones and missiles are orders of magnitude cheaper than the platforms (aircraft, tanks, ships) they target.

Offensive modes involving pre-positioning and saturation-attacks seem impossible to reliably defend against. And if all else fails, comment 13 observes:

"when articles talk about a naval war between China and USA in 2050 using some sexy new weapon, they never seem to cover that old weapon - nuclear ICBMs."

Five missiles in an area-spread with megaton warheads - no carrier group is going to look too clever after that.

---

My favourite fictional account: "Firelance" by David Mace (although the Vindicator is a cruise missile platform, not a carrier).

Google's war on folders

In the beginning, the search model for the Internet was (nested) folders:

Yahoo = "Yet Another Hierarchical Officious Oracle"

Remember, the intent is to find something in a vast space of candidates. The simplest way is to give that thing a label ('the filename') and put it in a folder which groups similar items by some salient property ('the folder'). Folders are nested ad libitum.

Folder-nesting is essentially set-inclusion, sometimes termed an 'isa-hierarchy'; a related idea is that of 'inheritance'.

Given an isa-hierarchy (a tree structure), a language to describe it is most simply effected by labelling the branches in order:

"Dropbox/mydata/camping/camping-list.docx"

where the language syntax is pretty much isomorphic to the semantic structure.

It was a small shock when Gmail introduced arbitrary labels to replace folders. The semantic space became somewhat more complex, as the set of all subsets of a collection of labels constitutes a lattice.

A simple Google search on keywords is not dissimilar to a search on labels (plus a fancy ordering relation wherein lies the magic).

Then came Google Photos, where all those carefully-named folders on your hard-drive ('My-Pictures/Holiday-in-the-Dordogne-July-2014') vanished, and Google used metadata and AI scene recognition to organise your photos along many dimensions: people, places, things-in-the-picture, etc.

In real life and in ordinary conversation, people do not restrict themselves to labels to refer to things.

"That famous joke by that dead middle class comedian"

doesn't get a useful hit from Google today, although many British people might hazard a guess.

"That famous joke by Bob Monkhouse"

nails it, top of the list - but it's really labels at work here.

As Google augments its bottom-up, neural-net phrase-recognisers with stable and ubiquitous semantic/pragmatic models, the clunky, manual world of the folder hierarchy will finally be put to rest.

Google tried to bring a version of search to the personal hard-drive once before, but it never really took off: too much noise and not enough signal in the results.

But next time around?

Folders: archaic, inflexible, labour-intensive .. and doomed.