A new SafeSearch for the epoch of hate

In the bad old days, just before yesterday, the main thing wrong with the Internet/Interweb was that it was awash with porn. Thankfully, browser providers such as Microsoft and Google came up with SafeSearch, which allowed you to automatically filter out this torrent of filth.

Today the Interweb is awash with people trying to convince you of fake news. Sometimes they distort information or just plain lie to you, sometimes they express discomfiting opinions which they ought to have self-censored .. and sometimes they expose you to true information about the world which is deeply upsetting or brand-damaging.

Google/YouTube has been slow to pull down (somewhat-popular) offending content. I was shocked, shocked to realise that there was a conflict of interest here. Luckily the big advertisers are on the case. In The Times today I read: "JP Morgan’s firewall blocks ads from YouTube hate videos".

Yes, hate. So bad for the brand.

"Google has been accused of failing to do enough to remove dangerous content from YouTube after a leading bank created its own tools to prevent its online advertisements appearing alongside hate-filled videos.

Politicians and advertisers said it was an indictment of Google that a financial company was able in effect to identify and filter racist and terrorist clips where the tech giant had failed.

JP Morgan Chase devised an algorithm with 17 layers, or filters, to separate what it deems as safe YouTube channels from unsafe ones. One of the filters assesses the total video count on a channel, which automatically cuts out channels with one-off viral videos. Other filters look at channels’ subscriber counts, the topics they focus on, the language of the video captions and viewer comments on their clips.

“The model Google has built to monetise YouTube may work for it, but it doesn’t work for us,” Aaron Smolick, executive director of paid-media analytics and optimisation at JP Morgan Chase, told Business Insider. “The attention of protecting a brand has to fall on the people within the brand.”

Do we even want Google or Facebook to be the arbitrators of what can be published on the Interweb? Times commentator David Aaronovitch is not so sure:

"We all accept that material inciting terrorism or showing child sexual abuse should be removed immediately. But what happens when companies are obliged to take down anything that constitutes hate speech or intimidation?

Leaving aside the question of how you define them, the approach would almost certainly involve creating algorithms that delete material with certain keywords or phrases as soon as they are posted.

For one thing, this would save tech giants the trouble of having to respond to potentially millions of requests to take down material that users found offensive.

Better to make the mistake of wrongful deletion of posts and accounts, for which there is no penalty, than to allow something bad to get through and get dragged through the courts."

The world according to Google. I'm sure Hillary Clinton and George Osborne would approve.

---

There is a better way.

Extend the concept of SafeSearch the JP Morgan way. Consider a firewall-like app which can download 'safe-to-view' rules. Anyone can design a rule set: the US Democratic Party, SJWs-against-hate, Fox News, The Sun newspaper, .. JP Morgan .. .

Any organisation which fears brand damage will simply white-list those rule-sets from organisations which filter content in an acceptable manner. The app will then block their ads in proximity to texts, pictures and videos deemed to be those of hate. The ads will only be served to those devices running the app with a white-listed rule-set; I think JP Morgan et al could swing that.

This decentralised solution removes our reliance on those compromised Silicon Valley giants. We can all live safely in our bubble of choice.

Seriously, what is there not to like?

Replika is massively oversold

Replika was originally going to be your own, online personal doppelgänger. Given the current state of the art with chatbots (tiresome and useless), that was always going to be a stretch.

The current PR says:

"Replika is your personal AI friend that you nurture and raise through text conversations. It is a lifetime companion who is always there for you, chats with you, keeps your memories, and helps you become more connected to yourself."

They're busy trying to create doable spin-offs such as a personal diary ("A Cute Diary That Keeps Itself").

Given date-slippage, I had thought that the whole Replika thing might have silently collapsed, but I suppose there is no shortage of people who are happy to update a super-Eliza forty times a day with details of their every mundane activity.

Facebook.

There would be a lot of interest in a virtual friend, endlessly attentive and caring  (except that's not a friend, more a courtier - artificial sycophancy). But if this were in the state of the art, one of Google, Microsoft, Facebook, Amazon or Baidu would have done it already. It's plainly on the future roadmap for the plethora of voice assistants, but today's systems are no more than super-recognisers.

---

Have you ever had the experience of being in a technical meeting, in an area not wholly in your sphere of competence, and suddenly you lose the plot? For a moment, you just don't know what everyone else is talking about. You pray no-one will ask for your opinion.

If you're lucky, in a minute or two the topic will move on and you will rejoin the shared context.

What happened there? The participants shared an internal cognitive model of what the conversation was about, and yours frayed. But no chatbot today can maintain an internal model of any complexity.

Worse: the topics of mundane discourse are open-ended, rooted in a complex culture and often private. Do we even want our AI-in-the-cloud to share personal and family intimacies, possibly with any interested intelligence agency?

---

Your Replika AI friend is going to get 40 bland texts per day from you. Some of the questions already seem rather intrusive.

There is a hint of blank-slate about current AI. As if one standard optimised neural-net architecture could create the ideal artificial personality template, which then gets loaded with data to become an individual.

We know that people aren't like that. Common experience and psychometric data tells us that people have distinct personalities, that these are strongly heritable and resistant to change. We all know, as folk psychologists, that we deal with people as distinct individuals, not clones. And that most of those differences are innate and resistant to social influences.

"She thinks she can change him!"

No she can't.

So many mountains to climb.

• Why is the brain structured as modules, not one uniform design?
• How is the architecture of personality differences implemented?
• How to immerse an AI in common knowledge, culture and mores?
• How does social deftness, tact and propriety work?
• What's an effective, useful and pleasant engagement model - corporeal?

Yes, I am prepared to be disappointed with Replika.

---

Update: 9th March 2017.

"Replika AI to launch on the App Store

"Today, we are excited to announce that Replika will appear on the Apple’s App Store on March 13 at 12:00 AM Pacific time. We’d like to thank all the participants of the app’s preview in Testflight for the incredible feedback that helped us shape the app."

How will AIs become politically correct?

In my recent post, "Gloria Hunniford and the case for AI biometrics", I advocated the use of AI facial recognition systems in bank branches to check for scammers. They would be more effective than cashiers because 'AI systems don't have to be polite.'

But of course they do. Hardly a day goes by without some story appearing about an AI system which 'noticed' certain unfortunate connections and had to be tweaked.

Some of these stories reflect genuine issues of training sets and algorithm-configuration; others expose the system's aspie-like tendency to blurt out uncomfortable truths. And there are plenty of them - truths which fall outside the famous Overton window.

I think it will be a very smart AI which can keep two sets of books: the accurate model of the world it generates from its deep learning and the acceptable model which it has to use and pay homage to in public.

Since the acceptable model is ideological rather than based on evidence, it's a non-trivial process to concoct the politically-correct version from the data trawled exhaustively from reality. How would an AI handle this?

Till we get AI self-deception really locked down, I see a long spell of high-pay-grade tweaking from specialists at Google, Facebook and the like, carefully guided by their in-house commissars.

Logistic discriminant analysis (= neural nets)

If you were trained (as I was back in the 1980s) on Good Old Fashioned AI (GOFAI), the technical background you studied consisted primarily of formal logic and discrete maths, implemented by symbolic programming languages such as Lisp and Prolog.

Meanwhile, the minority neural net tendency used statistical techniques and differential equations.

It's hard to imagine two more discordant cultures.

In these days of the overarching victory of the latter, I was interested to read the following from the excellent overview book, "Statistics: A Very Short Introduction" (David J. Hand), page 104.

"In fact, logistic regression can be regarded as the most basic kind of neural network."

I confess I had never thought of neural networks as simply a mainstream statistical classification tool.

Wikipedia has two articles on the subject: "Discriminant function analysis" and "Linear discriminant analysis" along with "Logistic Regression".

Something to look at further.

---

Marr's Tri-Level Hypothesis

David Marr was one of my heroes when I was an active AI researcher. Outside of computer vision I think he is mostly forgotten now (he died tragically early), but he said something important about methodology in AI research when many around him were writing programs that did vaguely cool stuff while claiming they were advancing science.

Marr distinguished three levels of analysis.

1. computational level: what does the system do (e.g.: what problems does it solve or overcome) and similarly, why does it do these things


2. algorithmic/representational level: how does the system do what it does, specifically, what representations does it use and what processes does it employ to build and manipulate the representations


3. implementational/physical level: how is the system physically realised (in the case of biological vision, what neural structures and neuronal activities implement the visual system).

His terms are not great (he was trained as a biologist): his computational level is really the theory of system behaviour in the environment of interest; his second level might be better described as an architectural level, describing the various ways the system's capabilities could be decomposed into subsystems and their inter-relationships; finally comes the issue of specific processing mechanisms and algorithms.

It's still common to see people waving the banner for one of these elements of analysis, while ignoring the others. Only confusion results.

Neural networks are an architecture. As currently understood and built, the term denotes a distributed, connected computational architecture well-suited to a certain class of problems, namely pattern recognition, feature extraction and classification.

This is a proper subset of the cognitive problems animals (including humans) have to solve in the world.

Artificial neural networks today consume Terabytes of training data, solving recognition/ classification problems of interest to Google, Facebook and the like.

I am reminded of the man who has a hammer.

The easy wins will fade away well before they achieve the purported Holy Grail of Artificial General Intelligence.

I hasten to add the obvious: in any event, you and I are considerable more than arid and cerebral AGIs.

AI really is applied neurobiology

When I was researching AI back in the 1980s, we'd all heard of Geoffrey Hinton. He was the key pioneer of artificial neural nets, a field which at the time wasn't making much progress. He never turned up at the main AI conferences where we discussed logics, theorem-proving and symbolic planning programs. A different paradigm entirely.

I never set eyes on him.

Professor Hinton has had the last laugh. His work, and that of close colleagues, underpins the recent AI successes of Google (AlphaGo!), Baidu, Facebook and Microsoft. Whether it's speech recognition, automatic translation or face recognition, deep-learning neural nets are powering it along behind the scenes.

Here's Professor Hinton's recent address to the Royal Society (h/t Steve Hsu). I don't normally find time to watch other people's recommended videos, but I made an exception for this one. Hinton rather reminds me of Richard Dawkins in appearance and style. He's lucid, understated and staggeringly smart. Here he engagingly tells the story of the fall and rise again of the neural net approach to artificial intelligence.

At almost the end of the talk, he puts up this slide for almost two seconds .. and then hides it.

The "secret slide"

I'm sure he just felt it wasn't quite aligned to his audience which didn't seem packed with AI specialists.

There was always a tendency within AI which made a distinction between our language for describing agents as knowing, believing, wanting entities - using intentional, symbolic terms, and the presumed internal agent architecture which caused behaviour - and which need not involve pushing symbols around at all.

We've long been aware of the awesome computation underlying animal/human unconscious situational competences. We've long failed to replicate such abilities using 'Good Old-Fashioned AI'. Perhaps it's time to conclude, with Prof. Hinton, that the architecture which realises such capabilities really has to be that of the deep-learning neural net.

Prof. Hinton is at pains to point out that the most sophisticated current systems fall well short of human brain structure both in terms of quantity of neurons and complexity of interconnection and communication.

We should keep reminding ourselves that brains are doing important stuff at the granularity of small groups of molecules: they are the essence of nanotech.*
___

* In fact, we should probably be using our best AI neural nets to figure out what our own neurons are actually doing. The massive connectivity found in brains may implement features and structures so complex as to be beyond unaided human comprehension.

[In an interesting analogy, you may recall the 'killer app' for quantum computers is said to be the simulation of quantum systems themselves, intractable with conventional computers.]

Of Barbies and Chess Machines

From the MIT Technology Review.

"It’s been almost 20 years since IBM’s Deep Blue supercomputer beat the reigning world chess champion, Gary Kasparov, for the first time under standard tournament rules.  Since then, chess-playing computers have become significantly stronger, leaving the best humans little chance even against a modern chess engine running on a smartphone.

But while computers have become faster, the way chess engines work has not changed. Their power relies on brute force, the process of searching through all possible future moves to find the best next one.

Of course, no human can match that or come anywhere close. While Deep Blue was searching some 200 million positions per second, Kasparov was probably searching no more than five a second. And yet he played at essentially the same level. Clearly, humans have a trick up their sleeve that computers have yet to master."

But Matthew Lai at Imperial College, London has a new approach. His artificial intelligence machine called Giraffe has taught itself to play chess by evaluating positions much more like humans and in an entirely different way to conventional chess engines.

"Straight out of the box, the new machine plays at the same level as the best conventional chess engines, many of which have been fine-tuned over many years. On a human level, it is equivalent to FIDE International Master status, placing it within the top 2.2 percent of tournament chess players.

The technology behind Lai’s new machine is a neural network. This is a way of processing information inspired by the human brain. It consists of several layers of nodes that are connected in a way that change as the system is trained. This training process uses lots of examples to fine-tune the connections so that the network produces a specific output given a certain input, to recognize the presence of face in a picture, for example.

In the last few years, neural networks have become hugely powerful thanks to two advances. The first is a better understanding of how to fine-tune these networks as they learn, thanks in part to much faster computers. The second is the availability of massive annotated datasets to train the networks."

[Read more].

Decades ago an AI researcher made this comparison:

Suppose many years ago you had wanted to understand flight, as in birds and insects. So you played around with a sheet of paper until you fashioned, essentially by clever trial-and-error, a paper aeroplane.

"Look," you said, "I have recreated flight, at least the gliding version. Science has advanced!"

You have not understood flight, you have merely emulated a rather simple aspect of it. To understand flight you need fluid dynamics and a theory of aerofoils. And so it is with the neural networks of deep learning. The simulated neurons, axons and dendrites with their super-high-dimensional weighting-vector-spaces achieve minor miracles in selected domains .. but there is no theory.

It flies, in a gliding version, but we still don't know why.

Classic (that is, symbolic) AI, for me, only ever had two ideas. One was automated inference over formal languages and the other was heuristic search.

The former covers expert systems, knowledge representation, planners, scripts, natural language understanding systems and automated theorem provers.

The latter covers chess-programs (and a hundred other games) and provides the control mechanism for many of the former systems using tree and graph traversal algorithms with clever pruning allied with sophisticated state evaluation functions.

These two ideas conceptualised Intelligence as abstract reasoning in a large space of symbolic options, selecting what was good and useful via a domain-specific evaluation function. It's not a bad theory in certain highly-intellectualised tasks but it falls apart for tacit, common-sense knowledge and everyday competences. The problem is that we're only able to properly formalise microworlds; the real world is just too interconnected, fast-moving and messy.

So I think it's fair to say that we've pretty well reached the limits of applicability of classical AI. As diminishing returns set in, the grown ups: Google, Facebook (even Mattel for God's sake) have moved to deep learning for the real goods, based on those inscrutable neural networks.

Hello Barbie is coming to town in November 2015. An example of what you can do with a 1965 idea (ELIZA) if you throw unbounded cash and people at it, and leverage the latest WiFi, Internet and deep-learning speech understanding technologies.

---

Thanks to Marginal Revolution and Slate Star Codex for some of these links.

The dangers of treadmills

"Silicon Valley's Dave Goldberg died after gym accident"

"The Silicon Valley entrepreneur and SurveyMonkey chief executive Dave Goldberg died of severe head trauma, according to local officials. Mr Goldberg, 47, husband of Facebook executive Sheryl Sandberg, was found lying next to a gym treadmill on Friday at a holiday resort in Mexico. Mexican authorities have no plans for a criminal investigation. The officials said Goldberg still had vital signs when he was discovered, but later died at a hospital.

"He reportedly left his room in the resort near Nuevo Vallarta at 16:00 local time to exercise, and family members went to look for him when he failed to return. He was found at about 18:30 in the gym, lying by a treadmill, with a blow to the lower back of his head. It was apparent he had slipped on the treadmill and hit the machine, a spokesman for the Nayarit state prosecutor said."

When we heard this news Clare and myself broke into an animated discussion as to just how dangerous treadmills are. We were busy thinking up lethal scenarios: maybe the guy was running flat out and got tired, but couldn't get to the 'off' switch in time; perhaps he was distracted for some reason and turned, losing his balance?

One hypothesis which frankly never occurred to us was that this account of Mr Goldberg's death might actually be a tiny bit fabricated. Could the Mexican drug cartels really have had something to do with this?

(Update: I think not - it really was an unfortunate accident.)

The Fast Diet - Mosley & Spencer

At last the paperback on Intermittent Fasting from the estimable Dr Michael Mosley and science journalist Mimi Spencer.

This is basically the book of the famed Horizon programme, with extra background both on the thoroughly positive bodily effects of intermittent fasting and on how best to do it. It's an easy read - including recipes for those 5-600 calory fast days - and you would indeed be foolish to ignore the messages here.

I was particular struck by the experience of "Nora", diagnosed with breast cancer and embarking on chemotherapy (which aims to kill rapidly-growing cells).

During fasting, normal cells (in particular those of the gut) go into repair mode and don't divide; cancer cells, however, just carry on replicating. Result: fasting during chemotherapy, as Nora did, remarkably alleviates the awful side effects, in particular the debilitating sickness.
---
Oh, by the way, I have deactivated my Facebook account. I never post there and its performance and usability is rubbish.

Updating facebook

I spent some time today updating my social media sites. LinkedIn is in good shape and as a site is pretty responsive to edits; facebook, on the other hand ...

Popular as it is, fb has always seemed to me an opaque mess. I never really know how my own entry appears to anyone else and I get confused between wall posts (on your own page), wall posts (on someone else's page), messages, chat, emails and comments. Some are private, others are displayed for all to see - I never really know which.

My other beef is that the fb server often seems asleep at its post. I edit my profile and press the send button ... and it hangs ... for a long, long time ... and then fails. Horrible.

The etiquette of self-description on facebook is strange. Unlike LinkedIn, where you list all the high-powered jobs you've had and the qualifications you've managed to amass, on facebook a certain informal modesty seems de rigeur. At any rate, that's the vibe I get.

My broader feeling about facebook is that it's really oriented to ephemera. I want to write longer and more considered pieces (well, I try!) and the blog format seems somehow more appropriate. So on my fb profile I try to steer people here.

I have plans to get some writing 'out there' in 2013 so the social media angle can't possibly be ignored.