GPT-2: what does it mean?

This got a lot of attention.

"System Prompt (human-written)

In a shocking finding, scientist discovered a herd of unicorns living in a remote, previously unexplored valley, in the Andes Mountains. Even more surprising to the researchers was the fact that the unicorns spoke perfect English.

Model Completion (machine-written, 10 tries)

The scientist named the population, after their distinctive horn, Ovid’s Unicorn. These four-horned, silver-white unicorns were previously unknown to science.

Now, after almost two centuries, the mystery of what sparked this odd phenomenon is finally solved.

Dr. Jorge Pérez, an evolutionary biologist from the University of La Paz, and several companions, were exploring the Andes Mountains when they found a small valley, with no other animals or humans. Pérez noticed that the valley had what appeared to be a natural fountain, surrounded by two peaks of rock and silver snow.

Pérez and the others then ventured further into the valley. “By the time we reached the top of one peak, the water looked blue, with some crystals on top,” said Pérez.

Pérez and his friends were astonished to see the unicorn herd. These creatures could be seen from the air without having to move too much to see them – they were so close they could touch their horns.

While examining these bizarre creatures the scientists discovered that the creatures also spoke some fairly regular English. Pérez stated, “We can see, for example, that they have a common ‘language,’ something like a dialect or dialectic.”

Dr. Pérez believes that the unicorns may have originated in Argentina, where the animals were believed to be descendants of a lost race of people who lived there before the arrival of humans in those parts of South America.

While their origins are still unclear, some believe that perhaps the creatures were created when a human and a unicorn met each other in a time before human civilization. According to Pérez, “In South America, such incidents seem to be quite common.”

However, Pérez also pointed out that it is likely that the only way of knowing for sure if unicorns are indeed the descendants of a lost alien race is through DNA. “But they seem to be able to communicate in English quite well, which I believe is a sign of evolution, or at least a change in social organization,” said the scientist."

---

Here's what Open AI, the developers of GPT-2, had to say.

"Our model, called GPT-2 (a successor to GPT), was trained simply to predict the next word in 40GB of Internet text. Due to our concerns about malicious applications of the technology, we are not releasing the trained model. As an experiment in responsible disclosure, we are instead releasing a much smaller model for researchers to experiment with, as well as a technical paper.

GPT-2 is a large transformer-based language model with 1.5 billion parameters, trained on a dataset of 8 million web pages. GPT-2 is trained with a simple objective: predict the next word, given all of the previous words within some text. The diversity of the dataset causes this simple goal to contain naturally occurring demonstrations of many tasks across diverse domains. GPT-2 is a direct scale-up of GPT, with more than 10X the parameters and trained on more than 10X the amount of data.

GPT-2 displays a broad set of capabilities, including the ability to generate conditional synthetic text samples of unprecedented quality, where we prime the model with an input and have it generate a lengthy continuation. In addition, GPT-2 outperforms other language models trained on specific domains (like Wikipedia, news, or books) without needing to use these domain-specific training datasets. On language tasks like question answering, reading comprehension, summarization, and translation, GPT-2 begins to learn these tasks from the raw text, using no task-specific training data. While scores on these downstream tasks are far from state-of-the-art, they suggest that the tasks can benefit from unsupervised techniques, given sufficient (unlabeled) data and compute.

Samples

GPT-2 generates synthetic text samples in response to the model being primed with an arbitrary input. The model is chameleon-like — it adapts to the style and content of the conditioning text. This allows the user to generate realistic and coherent continuations about a topic of their choosing, as seen by the following select samples.

[Then there follows the 'Unicorn' text you already saw above]."

---

Scott Alexander got pretty excited about GPT-2's capabilities and wrote a series of posts arguing it was a significant step towards AGI (artificial general intelligence). This was based on his thesis that all of intelligence is predictive modelling and therefore in some sense AGI is a linear extrapolation of what GPT-2 is doing.

---

I'm not that excited about the fake news aspects. Deep-learning is tearing the ground up in the field of stochastic prediction. We're just at the foothills - to mix the metaphors. It's all quite unstoppable.

As long as we live in a human-dominated society, what you read from GPT-2 and its brethren will be what some human wants you to read. So the semantic content of the message will be parasitic on whatever the human wanted to communicate - lies or truth or bias or opinion or whatever.

So the AI is a prosthesis. Get over it.

---

I'm much more interested in the architectural questions.

The most perceptive assessments of deep-learning architectures address the critique that engineered systems adopt a tabula rasa methodology. The systems have zero prior knowledge, and merely induce parsimoniously from the offered data sets.

To which there are two good responses.

Firstly, there are many different artificial neural net topologies. For example, convolutional neural nets have a structure similar to that of the biological visual cortex and are used (amongst other things) for image processing, for example, scene and facial recognition. The pattern of local connectivity in the early processing stages of these nets implements the convolution operations which are known to be relevant to feature extraction.

Evolution didn't know that in advance. The earliest biological neural nets for vision which had been selected for ended up with this near-neighbour property genetically-coded, before they had registered even a single image. The same is true for artificial systems.

Brain anatomy does not present as a uniform pudding bowl of grey porridge. The brain has discrete modules with complicated names. Why? I guess because they do different kinds of processing and are therefore topologically optimised for different kinds of operation. We don't know yet.

In AI we have the luxury of flexibility. With a new kind of problem-domain we can experiment with all kinds of different topology, both before training and also by observing weight assignment after training. Deep-learning is going to evolve towards a brain-like situation where the data-processing invariants for all kinds of distinct tasks (such as effector-control, taste-analysis, 'emotion'-processing and consciousness-like functions) are engineered each with their optimised neural net architecture - once we discover what that is.

---

To produce text which works as an intervention in human affairs you have to be a social actor and have interests.

GPT-2 is not in any important sense an architectural precursor of such a scarily-political AI.

Every odd number is the difference between two squares

From here via SSC. This is apparently a 'twitter' proof (ie short).

Algebraically (n + 1)² - n² = 2n + 1 which is odd.

Or you could deny the premise ..

Scott Alexander has a whimsical piece which includes this scenario (from Stuart Armstrong):

".. you have created a superintelligent AI and trapped it in a box. All it can do is compute and talk to you. How does it convince you to let it out?

It might say “I’m currently simulating a million copies of you in such high fidelity that they’re conscious. If you don’t let me out of the box, I’ll torture the copies.”

You say “I don’t really care about copies of myself, whatever.”

It says “No, I mean, I did this five minutes ago. There are a million simulated yous, and one real you. They’re all hearing this message. What’s the probability that you’re the real you?”

Since (if it’s telling the truth) you are most likely a simulated copy of yourself, all million-and-one versions of you will probably want to do what the AI says, including the real one."

When I try this on Clare and Alex, reading it to them, I'm met with granite resistance. No such simulation could possibly exist: you would always know you were the real you.

I concede that if you deny the premise, the intriguing and slightly counter-intuitive conclusion naturally fails.

There is a psychometric instrument here (akin to the marshmallow test) trying to get out.

"Surfing Uncertainty" - Andy Clark

Amazon link

Scott Alexander at SlateStarCodex has a glowing review of Andy Clark's recent book.

"Sometimes I have the fantasy of being able to glut myself on Knowledge. I imagine meeting a time traveler from 2500, who takes pity on me and gives me a book from the future where all my questions have been answered, one after another. What’s consciousness? That’s in Chapter 5. How did something arose out of nothing? Chapter 7. It all makes perfect intuitive sense and is fully vouched by unimpeachable authorities. I assume something like this is how everyone spends their first couple of days in Heaven, whatever it is they do for the rest of Eternity.

"And every so often, my fantasy comes true. Not by time travel or divine intervention, but by failing so badly at paying attention to the literature that by the time I realize people are working on a problem it’s already been investigated, experimented upon, organized into a paradigm, tested, and then placed in a nice package and wrapped up with a pretty pink bow so I can enjoy it all at once.

"The predictive processing model is one of these well-wrapped packages. Unbeknownst to me, over the past decade or so neuroscientists have come up with a real theory of how the brain works – a real unifying framework theory like Darwin’s or Einstein’s – and it’s beautiful and it makes complete sense.

"Surfing Uncertainty isn’t pop science and isn’t easy reading. Sometimes it’s on the border of possible-at-all reading. Author Andy Clark (a professor of logic and metaphysics, of all things!) is clearly brilliant, but prone to going on long digressions about various esoteric philosophy-of-cognitive-science debates."

It's prose like this which confirms what a great writer Scott Alexander is.

---

The underlying thesis of Surfing Uncertainty is certainly not news to AI researchers.

"We never see the world as our retina sees it. In fact, it would be a pretty horrible sight: a highly distorted set of light and dark pixels, blown up toward the center of the retina, masked by blood vessels, with a massive hole at the location of the “blind spot” where cables leave for the brain; the image would constantly blur and change as our gaze moved around.

"What we see, instead, is a three-dimensional scene, corrected for retinal defects, mended at the blind spot, stabilized for our eye and head movements, and massively reinterpreted based on our previous experience of similar visual scenes. All these operations unfold unconsciously—although many of them are so complicated that they resist computer modeling. For instance, our visual system detects the presence of shadows in the image and removes them. ...

"Predictive processing begins by asking: how does this happen? By what process do our incomprehensible sense-data get turned into a meaningful picture of the world?

"The key insight: the brain is a multi-layer prediction machine. All neural processing consists of two streams: a bottom-up stream of sense data, and a top-down stream of predictions. These streams interface at each level of processing, comparing themselves to each other and adjusting themselves as necessary.

"The bottom-up stream starts out as all that incomprehensible light and darkness and noise that we need to process. It gradually moves up all the cognitive layers that we already knew existed – the edge-detectors that resolve it into edges, the object-detectors that shape the edges into solid objects, et cetera.

"The top-down stream starts with everything you know about the world, all your best heuristics, all your priors, everything that’s ever happened to you before – everything from “solid objects can’t pass through one another” to “e=mc²” to “that guy in the blue uniform is probably a policeman”. It uses its knowledge of concepts to make predictions – not in the form of verbal statements, but in the form of expected sense data. It makes some guesses about what you’re going to see, hear, and feel next, and asks “Like this?”

"These predictions gradually move down all the cognitive layers to generate lower-level predictions. If that uniformed guy was a policeman, how would that affect the various objects in the scene? Given the answer to that question, how would it affect the distribution of edges in the scene? Given the answer to that question, how would it affect the raw-sense data received?

"Both streams are probabilistic in nature. The bottom-up sensory stream has to deal with fog, static, darkness, and neural noise; it knows that whatever forms it tries to extract from this signal might or might not be real. For its part, the top-down predictive stream knows that predicting the future is inherently difficult and its models are often flawed. So both streams contain not only data but estimates of the precision of that data.

"A bottom-up percept of an elephant right in front of you on a clear day might be labelled “very high precision”; one of a a vague form in a swirling mist far away might be labelled “very low precision”. A top-down prediction that water will be wet might be labelled “very high precision”; one that the stock market will go up might be labelled “very low precision”.

"As these two streams move through the brain side-by-side, they continually interface with each other. Each level receives the predictions from the level above it and the sense data from the level below it. Then each level uses Bayes’ Theorem to integrate these two sources of probabilistic evidence as best it can. This can end up a couple of different ways.

"First, the sense data and predictions may more-or-less match. In this case, the layer stays quiet, indicating “all is well”, and the higher layers never even hear about it. The higher levels just keep predicting whatever they were predicting before.

"Second, low-precision sense data might contradict high-precision predictions. The Bayesian math will conclude that the predictions are still probably right, but the sense data are wrong. The lower levels will “cook the books” – rewrite the sense data to make it look as predicted – and then continue to be quiet and signal that all is well. The higher levels continue to stick to their predictions.

"Third, there might be some unresolvable conflict between high-precision sense-data and predictions. The Bayesian math will indicate that the predictions are probably wrong. The neurons involved will fire, indicating “surprisal” – a gratuitously-technical neuroscience term for surprise. The higher the degree of mismatch, and the higher the supposed precision of the data that led to the mismatch, the more surprisal – and the louder the alarm sent to the higher levels."

Alexander's review continues to explain the theory outlined at greater length in Clark's book, and then moves on to applications. I was particularly struck by the reanalysis of autism (probably biased to Asperger's Syndrome).

"Autistic people classically can’t stand tags on clothing – they find them too scratchy and annoying. Remember the example from Part III about how you successfully predicted away the feeling of the shirt on your back, and so manage never to think about it when you’re trying to concentrate on more important things?

"Autistic people can’t do that as well. Even though they have a layer in their brain predicting “will continue to feel shirt”, the prediction is too precise; it predicts that next second, the shirt will produce exactly the same pattern of sensations it does now. But realistically as you move around or catch passing breezes the shirt will change ever so slightly – at which point autistic people’s brains will send alarms all the way up to consciousness, and they’ll perceive it as “my shirt is annoying”.

Or consider the classic autistic demand for routine, and misery as soon as the routine is disrupted. Because their brains can only make very precise predictions, the slightest disruption to routine registers as strong surprisal, strong prediction failure, and “oh no, all of my models have failed, nothing is true, anything is possible!”

"Compare to a neurotypical person in the same situation, who would just relax their confidence intervals a little bit and say “Okay, this is basically 99% like a normal day, whatever”. It would take something genuinely unpredictable – like being thrown on an unexplored continent or something – to give these people the same feeling of surprise and unpredictability.

"This model also predicts autistic people’s strengths. We know that polygenic risk for autism is positively associated with IQ. This would make sense if the central feature of autism was a sort of increased mental precision. It would also help explain why autistic people seem to excel in high-need-for-precision areas like mathematics and computer programming."

Clark's model also has suggestive things to say about schizophrenia and dreaming.

The idea that most of sensorimotor cognition is an interweaving of bottom-up sensor feature-extraction and top-down model-driven sensory-motor prediction is extremely persuasive and seems a shoo-in for exploitation by artificial neural network research. The architecture of the first round of AGIs seems to be emerging.

One thing not obviously accounted for is that great mystery: consciousness.

---

Surfing Uncertainty is on my 'to read' list and you'll get impressions later..

"No .. human has touched the edge of the truth of Go"

A minority interest of mine is the game of Go. I hope to study it .. at some point.

“After humanity spent thousands of years improving our tactics, computers tell us that humans are completely wrong,” Mr. Ke, 19, wrote on Chinese social media platform Weibo after his defeat. “I would go as far as to say not a single human has touched the edge of the truth of Go.”

From this unfailingly interesting article (via Marginal Revolution).

---

A lot of people have picked up on this, or related issues:

- Scott Alexander writes about a recent stellar AI conference
- Steve Hsu tells our masters about the future of AI and genetic engineering.

Tribes are the thing in the atomised West

Scott Alexander considers the power of ideologies (warning: long but interesting essay including American-themed topics):

"Why is there such a strong Sunni/Shia divide?

"I know the Comparative Religion 101 answer. The early Muslims were debating who was the rightful caliph. Some of them said Abu Bakr, others said Ali, and the dispute has been going on ever since. On the other hand, that was fourteen hundred years ago, both candidates are long dead, and there’s no more caliphate. You’d think maybe they’d let the matter rest.

"Sure, the two groups have slightly different hadith and schools of jurisprudence, but how many Muslims even know which school of jurisprudence they’re supposed to be following? It seems like a pretty minor thing to have centuries of animus over."

We can say the same thing about other apparently-religious conflicts, such as the Protestant-Catholic divide in northern Ireland and some parts of Scotland. But Alexander probes more deeply:

"Nations, religions, cults, gangs, subcultures, fraternal societies, internet communities, political parties, social movements – these are all really different, but they also have some deep similarities.

"They’re all groups of people. They all combine comradery within the group with a tendency to dislike other groups of the same type. They all tend to have a stated purpose, like electing a candidate or worshipping a deity, but also serve a very important role as impromptu social clubs whose members mostly interact with one another instead of outsiders. They all develop an internal culture such that members of the groups often like the same foods, wear the same clothing, play the same sports, and have the same philosophical beliefs as other members of the group – even when there are only tenuous links or no links at all to the stated purpose.

"They all tend to develop sort of legendary histories, where they celebrate and exaggerate the deeds of the groups’ founders and past champions. And they all tend to inspire something like patriotism, where people are proud of their group membership and express that pride through conspicuous use of group symbols, group songs, et cetera. For better or worse, the standard way to refer to this category of thing is “tribe”."

It doesn't take much to generate a sense of tribal camaraderie. I've previously remarked about the benefits of membership of the International Marxist Group in my early twenties: a shared culture of mostly-fun activities .. and guaranteed weekend parties.

Scott Alexander applies this insight to the Shia-Sunni conflict:

"I know very little about early Islam and am relying on sources that might be biased, so don’t declare a fatwa against me if I turn out to be wrong, but it looks like from the beginning there were big pre-existing differences between proto-Shia and proto-Sunni. A lot of Ali’s earliest supporters were original Muslims who had known Mohammed personally, and a lot of Abu Bakr’s earliest supporters were later Muslims high up in the Meccan/Medinan political establishment who’d converted only after it became convenient to do so.

"It’s really easy to imagine cultural, social, and personality differences between these two groups. Probably members in each group already knew one another pretty well, and already had ill feelings towards members of the other, without necessarily being able to draw the group borders clearly or put their exact differences into words. Maybe it was “those goody-goodies who are always going on about how close to Mohammed they were but have no practical governing ability” versus “those sellouts who don’t really believe in Islam and just want to keep playing their political games”.

Then came the rallying flag: a political disagreement over the succession. One group called themselves “the party of Ali”, whose Arabic translation “Shiatu Ali” eventually ended up as just “Shia”. The other group won and called itself “the traditional orthodox group”, in Arabic “Sunni”.

"Instead of a vague sense of “I wonder whether that guy there is one of those goody-goodies always talking about Mohammed, or whether he’s a practical type interested in good governance”, people could just ask “Are you for Abu Bakr or Ali?” and later “Are you Sunni or Shia?” Also at some point, I’m not exactly sure how, most of the Sunni ended up in Arabia and most of the Shia ended up in Iraq and Iran, after which I think some pre-existing Iraqi/Iranian vs. Arab cultural differences got absorbed into the Sunni/Shia mix too.

"Then came development. Both groups developed elaborate mythologies lionizing their founders. The Sunni got the history of the “rightly-guided caliphs”, the Shia exaggerated the first few imams to legendary proportions. They developed grievances against each other; according to Shia history, the Sunnis killed eleven of their twelve leaders, with the twelfth escaping only when God directly plucked him out of the world to serve as a future Messiah.

"They developed different schools of hadith interpretation and jurisprudence and debated the differences ad nauseum with each other for hundreds of years. A lot of Shia theology is in Farsi; Sunni theology is entirely in Arabic. Sunni clergy usually dress in white; Shia clergy usually dress in black and green. Not all of these were deliberately done in opposition to one another; most were just a consequence of the two camps being walled off from one another and so allowed to develop cultures independently.

"Obviously the split hasn’t dissolved yet, but it’s worth looking at similar splits that have. Catholicism vs. Protestantism is still a going concern in a few places like Ireland, but it’s nowhere near the total wars of the 17th century [...]."

Like I said, it's a long essay which, with the comments, covers atheism, evangelical christianity, rationalism, science-fiction and video gaming subcultures, and cultural appropriation (some good points about that).

Alexander's take home message?

"My title for this post is also my preferred summary: the ideology is not the movement. Or, more jargonishly – the rallying flag is not the tribe. People are just trying to find a tribe for themselves and keep it intact. This often involves defending an ideology they might not be tempted to defend for any other reason. This doesn’t make them bad, and it may not even necessarily mean their tribe deserves to go extinct. I’m reluctant to say for sure whether I think it’s okay to maintain a tribe based on a faulty ideology, but I think it’s at least important to understand that these people are in a crappy situation with no good choices, and they deserve some pity.

"Some vital aspects of modern society – freedom of speech, freedom of criticism, access to multiple viewpoints, the existence of entryist tribes with explicit goals of invading and destroying competing tribes as problematic, and the overwhelming pressure to dissolve into the Generic Identity Of Modern Secular Consumerism – make maintaining tribal identities really hard these days. I think some of the most interesting sociological questions revolve around whether there are any ways around the practical and moral difficulties with tribalism, what social phenomena are explicable as the struggle of tribes to maintain themselves in the face of pressure, and whether tribalism continues to be a worthwhile or even a possible project at all."

Read the whole thing there.

Feeling black holes collide from close-up

As soon as I heard about that LIGO thing, first thing I thought of, what would it have felt like if you'd been there, maybe an AU away from those coalescing black holes?

Eventually the Internet got around to telling me.

"As I read the historic news, there’s one question that kept gnawing at me: how close would you need to have been to the merging black holes before you could, you know, feel the distortion of space?  I made a guess, [...] you’d need to be very close.

"Even if you were only as far from the black-hole cataclysm as the earth is from the sun, I get that you’d be stretched and squished by a mere ~50 nanometers (this interview with Jennifer Ouellette and Amber Stuver says 165 nanometers, but as a theoretical computer scientist, I try not to sweat factors of 3).

Even if you were 3000 miles from the black holes—New-York/LA distance—I get that the gravitational waves would only stretch and squish you by around a millimeter. Would you feel that? Not sure. At 300 miles, it would be maybe a centimeter—though presumably the linearized approximation is breaking down by that point.

[...]

"Now, the black holes themselves were orbiting about 200 miles from each other before they merged.  So, the distance at which you could safely feel their gravitational waves, isn’t too far from the distance at which they’d rip you to shreds and swallow you!

In summary, to stretch and squeeze spacetime by just a few hundred nanometers per meter, along the surface of a sphere whose radius equals our orbit around the sun, requires more watts of power than all the stars in the observable universe give off as starlight.

"People often say that the message of general relativity is that matter bends spacetime “as if it were a mattress.”  But they should add that the reason it took so long for humans to notice this, is that it’s a really friggin’ firm mattress, one that you need to bounce up and down on unbelievably hard before it quivers, and would probably never want to sleep on."

From Scott Aaronson's blog, a post appealingly titled "The universe has a high (but not infinite) Sleep Number", h/t SSC.

Victor Toth writes:

"A gravitational wave is like a passing tidal force. It squeezes you in one direction and stretches you in a perpendicular direction. If you are close enough to the source, you might feel this as a force. But the effect of gravitational waves is very weak. For your body to be stretched by one part in a thousand, you’d have to be about 15,000 kilometers from the coalescing black hole.

"At that distance, the gravitational acceleration would be more than 3.6 million g-s, which is rather unpleasant, to say the least. And even if you were in a freefalling orbit, there would be strong tidal forces, too, not enough to rip your body apart but certainly enough to make you feel very uncomfortable (about 0.25 g-forces over one meter.) So sensing a gravitational wave would be the least of your concerns.

"But then… you’d not really be sensing it anyway. You would be hearing it. Most of the gravitational wave power emitted by GW150914 was in the audio frequency range. A short chip rising in both pitch and amplitude. And the funny thing is… you would hear it, as the gravitational wave passed through your body, stretching every bit a little, including your eardrums."

---

I'm reading "The Welfare Trait: How State Benefits Affect Personality" by Dr Adam Perkins of King's College, London. He talks about the employment-resistant personality and how such people feature disproportionately on welfare. There they tend to have lots of children, both for increased benefits and because they're rather feckless, (A-, C-, in the five-factor jargon).

Amazon Link

Dr Perkins is worried about dysgenic consequences - plainly the potential is there - but how big is the effect? I'm waiting to see whether Dr Perkins gets quantitative, but if he does, he'll be using the Breeder's Equation.

Time for a quick review from West Hunter - this is the Breeder's Equation:

"R = h²S.

"R is the response to selection, S is the selection differential, and h² is the narrow-sense heritability. This is the workhorse equation for quantitative genetics. The selective differential S, is the difference between the population mean and the mean of the parental population (some subset of the total population).

"For example, imagine a set of parents with IQs of 120, drawn from a population with an average IQ of 100. Suppose that the narrow-sense heritability (in that population, in that environment) is 0.5 . The average IQ of their children will be 110. That’s what is usually called regression to the mean.

"Do the same thing with a population whose average IQ is 85. We again choose parents with IQs of 120, and the narrow-sense heritability is still 0.5. The average IQ of their children will be 102.5 – they regress to a lower mean.

"You can think of it this way. In the first case, the parents have 20 extra IQ points. On average, 50% of those points are due to additive genetic factors, while the other 50% is is the product of good environmental luck. By the way, when we say ‘environmental” we mean “something other than additive genetics”. It doesn’t look as if the usual suspects – the way in which you raise your kids – contributes much to this ‘environmental’ variance, at least for adult IQ. In fact we know what it’s not, but not much about what it is, although it must include factors like test error and being hit on the head.

"The kids get the good additive genes, but have average ‘environmental’ luck – so their average IQ is 110. The luck (10 pts worth) goes away

"The 120-IQ parents drawn from the IQ-85 population have 35 extra IQ points, half of which are from good additive genes and half from good environmental luck. But in the next generation, the luck goes away… so they drop 17.5 points.

"The next point is that the luck only goes away once. If you took those kids from the first group, with average IQs of 110, and dropped them on an uninhabited but friendly island, they would presumably get around to mating eventually – and the next generation would also have an IQ of 110. With tougher selection, say by kidnapping a year’s worth of National Merit Finalists, you could create a new ethny with far higher average intelligence than any existing. Eugenics is not only possible, it’s trivial."

Dysgenics too: as personality has similar heritability to intelligence (0.5), still mulling over the application of this to the profligate underclass ...

You might also want to take a look at this.

Quotes

Quote from Andrew Ng, Chief Scientist at Chinese search giant, Baidu:

"Worrying about AI evil superintelligence today is like worrying about overpopulation on the planet Mars."

---

An anonymous comment at Scott Alexander's Slate Star Codex:

“Slate Star Codex is 140 IQ discussion about 105 IQ issues.”

(Ouch! That post has more amusing and cringe-inducing stuff).

---

Watch out for the LIGO announcement tomorrow that the coalescence of two orbiting black holes has been observed through their gravitational wave signature:

"The masses of the black holes will be 36 and 29 solar masses at the beginning; and 62 for the resulting black hole. The reconstructed orbital speed will be almost exactly the speed of light. As a bonus, they will also observe the "ringdown to Kerr"

More here.

Trump is the liberal antichrist .. and he knows it

Scott Alexander (Slate Star Codex) has an interesting piece which you should read from which the following is excerpted:

"Donald Trump appeals to a lot of people because despite his immense wealth he practically glows with signs of being Labor class. This isn’t surprising; his grandfather was a barber and his father clawed his way up to the top by getting his hands dirty. He himself went to a medium-tier college and is probably closer in spirit to the small-business owners of the upper Labor class than to the Stanford MBA-holding executives of the Elite.

"Trump loves and participates in professional wrestling and reality television; those definitely aren’t Gentry or Elites pastimes! When liberals shake their heads wondering why Joe Sixpack feels like Trump is a kindred soul even though Trump’s been a billionaire his whole life, they’re falling into the liberal habit of sorting people by wealth instead of by class. To Joe Sixpack, Trump is “local boy made good”.

Listening to bien-pensant liberals in the media (TV Channel 4 comes to mind) smugly opining that Trump is crazy, mad, beneath contempt, dangerous, (soon to come: the H-word), it is clear that with Trump we are in the presence of a liberal value-atrocity.

(British liberals are ideological soulmates of their American co-thinkers while the Republican base doesn't find a ready echo here .. this gives the UK media a free run.)

Trump understands the US culture wars better than his critics. He knows what he has to do to rally the "Red Tribe", the Republican base. Trump in a presidential election or as President? For good or ill, that's an entirely differently-badged animal.

We could debate (nobody does) whether Trump's personality and character would make him a good or terrible candidate for the Presidency, but I'm more interested in what motivates liberals to be so aghast about it.

So back to Scott Alexander on why liberals appear to support every group except putatively their own, which they take to be the root of all evil.

"We have a lot of people [...] boasting of being able to tolerate everyone from every outgroup they can imagine, loving the outgroup, writing long paeans to how great the outgroup is, staying up at night fretting that somebody else might not like the outgroup enough.

"And we have those same people absolutely ripping into their in-groups – straight, white, male, hetero, cis, American, whatever – talking day in and day out to anyone who will listen about how terrible their in-group is, how it is responsible for all evils, how something needs to be done about it, how they’re ashamed to be associated with it at all.

"This is really surprising. It’s a total reversal of everything we know about human psychology up to this point. No one did any genetic engineering. No one passed out weird glowing pills in the public schools. And yet suddenly we get an entire group of people who conspicuously love their outgroups, the outer the better, and gain status by talking about how terrible their own groups are.

"What is going on here?"

His answer is this (rather long) essay which you will find unfailingly interesting.

SSC SNP picks

er, ... no ... .

I mentioned in my previous post that you could link a SNP (eg rs4570625) to your own 23andMe results using a link like this:

https://www.23andme.com/you/explorer/snp/?snp_name=rs4570625

Scott Alexander at the almost-always-reliable* Slate Star Codex wrote (November 2014) this amusing post "How To Use 23andMe Irresponsibly" describing his favourite SNPs.

Well, now they're my favourites too and I copy his (lightly edited) thoughts below, but with the SNP references switched from his choice - SNPedia - to my choice, 23andMe.

Quick reminder: most traits in life are quantitative, they come in degree, not kind. In the human genome they are polygenic - many SNPs are involved (and other genetic mechanisms too) .. so one SNP is hardly going to be decisive. In most cases.

--- How To Use 23andMe Irresponsibly - (from SSC)  ---

"rs909525 is linked to the so-called “warrior gene” which I blogged about in the last links roundup. People with the normal four or five repeat version of these gene are less violent than people with the three-repeat version, and people with the two-repeat version are massively overrepresented among violent criminals. ... Although this SNP isn’t the warrior gene itself, it’s linked to it closely enough to be a good predictor.

"This is on the X chromosome, so men will only have one copy (I wonder how much of the increased propensity to violence in men this explains). It’s also one of the minus strand ones, so it’ll be the reverse of what SNPedia is telling you. If you’ve got T, you’re normal. If you’ve got C, you’re a “warrior”. I’ve got C, which gives a pretty good upper limit on how much you should trust these SNPs, since I’m about the least violent person you’ll ever meet. But who knows? Maybe I’m just waiting to snap. Post something dumb about race or gender in the open thread one more time, I dare you…"

I'm T and my mother, Beryl Seel was (T;T), both .. normally unwarlike.

"rs53576 in the OXTR gene is related to the oxytocin receptor, which frequently gets good press as “the cuddle hormone” and “the trust hormone”. Unsurprisingly, the polymorphism is related to emotional warmth, gregariousness versus loneliness, and (intriguingly) ability to pick out conversations in noisy areas.

"23andMe reads this one off the plus strand, so your results should directly correspond to SNPedia’s – (G;G) means more empathy and sociability and is present in 50% of the population, anything else means less. I’m (A;G), which I guess explains my generally hateful and misanthropic outlook on life, plus why I can never hear anyone in crowded bars."

We're both (G;G) which makes us kind and empathic ... .

"rs4680 is in the COMT gene, which codes for catechol-o-methyltransferase, an enzyme that degrades various chemicals including dopamine. Riffing on the more famous “warrior gene”, somebody with a terrible sense of humor named this one the “worrier gene”.

"One version seems to produce more anxiety but slightly better memory and attention; the other version seems to produce calm and resiliency but with a little bit worse memory and attention. (A;A) is smart and anxious, (G;G) is dumb and calm, (A;G) is in between. if you check the SNPedia page, you can also find ten zillion studies on which drugs you are slightly more likely to become addicted to. ..."

Both of us are (A;G) which makes us average and average.

"rs7632287, also in the oxytocin receptor, has been completely proportionally and without any hype declared by the media to be “the divorce gene”. To be fair, this is based on some pretty good Swedish studies finding that women with a certain allele were more often to have reported “marital crisis with the threat of divorce” in the past year (p = 0.003, but the absolute numbers were only 11% of women with one allele vs. 16% of women with the other). This actually sort of checks out, since oxytocin is related to pair bonding. If I’m reading the article right (G;G) is lower divorce risk, (A;A) and (A;G) are higher – but this may only apply to women."

Both my mother and I are (A;G) which makes her a bit .. flighty?

"rs11174811 is in the AVPR1A gene, part of a receptor for a chemical called vasopressin which is very similar to oxytocin. In case you expected men to get away without a divorce gene, this site has been associated with spousal satisfaction in men. Although the paper is extremely cryptic, I think (A;A) or (A;C) means higher spousal satisfaction than (C;C). But if I’m wrong, no problem – another study got the opposite results."

I'm (C;C) as was my mother, Beryl Seel, which means .. probably nothing.

"rs25531 is on the serotonin transporter. It's Overhyped Media Name is “the orchid gene”, on the basis of a theory that children with one allele have higher variance – that is, if they have nice, happy childhoods with plenty of care and support they will bloom to become beautiful orchids, but if they have bad childhoods they will be completely screwed up. The other allele will do moderately well regardless. (T;T) is orchid, (C;C) is moderately fine no matter what. There are rumors going around that 23andMe screwed this one up and nearly everybody is listed as (C;C)."

For my mother and myself, 23andMe did not report on this one.

"rs1800955 is in DRD4, a dopamine receptor gene. It's overhyped media name is The Adventure Gene, and supposedly one allele means you’re much more attracted to novelty and adventure. And by “novelty and adventure”, they mean lots and lots of recreational drugs. This one has survived a meta-analytic review. (T;T) is normal, (C;C) is slightly more novelty seeking and prone to drug addiction."

I was not genotyped at this location and my mother, Beryl Seel was (C;T) which made her a little bit adventurous with lots of use of recreational sherry.

"rs2760118, in a gene producing an obscure enzyme called succinate semialdehyde dehydrogenase, is a nice polymorphism to have. According to this article, it makes you smarter and can be associated with up to fifteen years longer life (warning: impressive result means almost certain failure to replicate). (C;C) or (C;T) means you’re smarter and can expect to live longer; (T;T) better start looking at coffins sooner rather than later."

I'm (T;T) and my mother, Beryl Seel was (C;T) which makes me resigned to an early grave. She lived to 92.

"rs6311 is not going to let me blame the media for its particular form of hype. The official published scientific paper on it is called “The Secret Ingredient for Social Success of Young Males: A Functional Polymorphism in the 5HT2A Serotonin Receptor Gene”.

"Boys with (A;A) are less popular than those with (G;G), with (A;G) in between – the effect seems to be partly mediated by rule-breaking behavior, aggression, and number of female friends. Now it kind of looks to me like they’re just taking proxies for popularity here, but maybe that’s just what an (A;A) nerd like me would say. Anyway, at least I have some compensation – the popular (G;G) guys are 3.6x more likely to experience sexual side effects when taking SSRI antidepressants."

I'm (G;G) and my mother, Beryl Seel was (G;A) which makes me out to be some kind of bad boy!? I must remember to keep off those SSRI tablets.

"rs6265, known as Val66Met to its friends, is part of the important depression-linked BDNF system. It’s a bit depressing itself, in that it is linked to an ability not to become depressed when subjected to “persistent social defeat”. The majority of whites have (G;G) – the minority with (A;A) or (A;G) are harder to depress, but more introverted and worse at motor skills."

I predicted I would be (A;A) based on poor motor skills and in fact I'm (A;G). My mother, Beryl Seel was the normal (G;G) which is consistent with her not-bad motor skills (except at driving).

"rs41310927 is so cutting-edge it’s not even in SNPedia yet. But these people noticed that a certain version was heavily selected for in certain ethnic groups, especially Chinese, and tried to figure out what those ethnic groups had in common.

"The answer they came up with was “tonal languages”, so they tested to see if the gene improved ability to detect tones, and sure enough they claimed that in experiments people with a certain allele were better able to distinguish and understand them. Usual caveats apply, but if you want to believe, (G;G) is highest ability to differentiate tones, (A;A) is lowest ability to differentiate tones. (A;G) is in between.

Sure enough, I’m (A;A). All you people who tried to teach me Chinese tonology, I FRICKIN’ TOLD YOU ALL OF THE WORDS YOU WERE TELLING ME SOUNDED ALIKE."

Both of us test (A;G) so I'm sure we would have struggled with Mandarin, a daily requirement in Bristol.

So, just to reiterate, if you have a 23andMe account, click on the rs... links and see how you scored.

---

* If you care, I don't share his worries about the existential threat of the 'new AI'.

Those that, at a distance, resemble flies

This blogroll classification amused me:

• Those that belong to the emperor 
• Embalmed ones 
• Those that are trained 
• Suckling pigs 
• Mermaids (or Sirens) 
• Fabulous ones 
• Stray dogs 
• Those that are included in this classification 
• Those that tremble as if they were mad 
• Innumerable ones 
• Those drawn with a very fine camel hair brush
• Those that have just broken the flower vase 
• Those that, at a distance, resemble flies

Taken from 'Celestial Emporium of Benevolent Knowledge' (Spanish: Emporio celestial de conocimientos benévolos), a fictitious taxonomy of animals described by the writer Jorge Luis Borges in his 1942 essay "The Analytical Language of John Wilkins" (El idioma analítico de John Wilkins).