I’ve been thinking about math a lot recently. Learning it has become a bit of an obsession.

I’ve come to believe that there is a fundamental mistake in the way society conceptualizes math.

• “You need it for physics, engineering, computer graphics…”
• “It’s a subject at school. Pass the exam and forget it.”
• or even “it’s the study of things with formal definitions” (which is true, but it makes the scale hard to see)

Here’s how society ought to think about math: math is not academic. Math is not “a tool”. Math is the exclusive totality of reason. If you are not doing math, you are not rational. If you seem to be rational, it’s because you are approximating math. To the extent your approximation fails, you will fail.

Math is not a subject at school. If you leave school understanding math, the other subjects are just details. If you leave the school without understanding math, you will forever be a consumer of knowledge you can’t understand, only apply by rote. Your schooling will have failed at the most fundamental level.

Heinlein nailed it. To be bad at math is to be a tolerable sub-human who has learned not to make messes about the house. No matter if you are smart, your mental horsepower will slip and find no traction. You will be frustrated knowing that you can, but you don’t know how, and nor can you tell if you’re (by luck) doing it right, nor when you start to stray.

When post-humans look back on humanity, they will recognize our math and our thinking built upon math as our only “people” achievements. It’s our monkey nature to build cities and write books. Only in math do we exceed the monkey. The post-humans themselves will think in math exclusively.

This is obvious to me, but I can’t prove it, because I’m bad at math. Now can you see why I’m so desperate to learn?

…or, why I slightly disagree with Eliezer Yudkowsky.

E.Y. says that “free will” has no sensible referent in the range between determinism and dice-rolling randomness. I’d add that appealing to “souls” only bumps the question up one recursion. Fair enough. However, in pondering the question, I think I have found a useful referent for “free will”.

An example of a tool which achieves solutions in a search space for which all the rules are known is an arithmetic engine. There are relatively few ways of doing 1+1=2. With those few choices hard-coded (to best suit hardware, etc) the solver becomes fully deterministic. To any one search target it will have only one path.

An example of a tool which achieves solutions in a search space with unknown facts and rules is an intelligent mind. There are many ways of looking for solutions to the problems humans run across. Often there are red herrings and traps. The right path is uncertain and must be adaptively sought. We gather evidence and update our beliefs, but in the end we are forced to make stochastic choices. Those choices be predicted from any simpler theory than simulation. They are “incompressible decisions”. This is free will.

Free will means the uncertainty in our knowledge of the search space. You could consider it as a sum over the possible paths in search space with non-negligible expected utility. (For future study: are paths in search space discrete from one another or continuous?)

Another way of putting it: free will is ignorance. When we know, we no longer have a choice of the right way to solve it. Extrapolating: total free will is total ignorance and undirected thrashing-around. Total knowledge means zero free will - we are back to our arithmetic solver. (This is not quite true. Total knowledge can still leave decision paths completely equivalent, never reducing down to one deterministic path.)

Corollary: as we gain knowledge (and later, as we gain intellect), our understanding will grow and our choices will narrow. We will have less free will — unless our targets grow and our search space expands, in which case we will be back to being ignorant about larger and more complex things. But the trend is away from freedom and towards determinism.

A person who understands that mind is an optimization process will recognize that more of this sort of determinism is good. It’s called “converging on a solution”.

It has been a few months since I posted here, and surprisingly much water has gone under the bridge. For most of my enlightenment, I owe Eliezer Yudkowsky, who is much smarter than me and something of a hero. (He’s also an autodidact. I will catch up to you!)

The two largest changes: I have come to understand a bit more of the Bayesian ideal of reasoning from E.Y.’s posts on Overcoming Bias blog, and I have reached a much better idea of artificial and natural intelligence (much credit to this paper), which has allowed me to unify my treatment of the two. Intelligence is that which hits a small goal in a large search space. General intelligence does this in the general case.

This has allowed me to understand that humans are not (as we arrogantly presume) great shakes in the generality department. You can see human generality in the understanding of maths, which is an easy problem from any objective standpoint. It’s rare and mostly weak. We are almost specific intelligences, as tailored for being a monkey as a chess program is for its niche.

This kind of thing has led me to four, if not new then at least newly emphasized positions:

1. I am now a transhumanist. I want to do better than this meat mind. It is suited to apes, not to people. As E.Y. puts it: how strange we are, a mind with the form of evolution. The bridge between the era of evolution and the era of mind. “Artificial Intelligence” should be read as meaning “mind born of mind”. The artificial is a badge of honor!
2. I am now a singularitarian. I understand the power of self-improving AI. Only two events are comparable: the big bang and the first ancestor of life. If we design an AI that is unsafe, we will be as chaff before the wind. Most kinds of unsafe AI are also boring (a variant of sorcerer’s apprentice mode). Making interesting AI will be hard. Making safe AI will be very hard. Technical trends that make it easier to build unsafe AI are a danger, not a blessing.
3. I am now an atheist fully. I understand mind, and I can’t harbor any romantic illusions about its nature. Mind is mathematical and computational. These are scientifically tractable concepts. Being info-computation is actually more interesting than being some undefined “soul”. Information has properties, and they are interesting ones.
4. Not coincidentally, I really like maths!

Addendum to the below: E.Y. has convinced me that we can’t dare to build a neural AI, because it’s mathematically intractable and can’t be proven safe. The same has to go for an instinct-simulating AI, except to the extent instinct approximates Bayesian reasoning. Although I can’t yet follow the maths, I am convinced on good authority that any system of understanding either maps to Bayesian probability, or is inconsistent — and that it only has value at all to the extent it approximates Bayes’ law. Ergo, any good AI will either be Bayesian, or a hack. (Humans are a hack.)

And now for something completely different. Some random thoughts that interested me:

• Does conventional AI go about modeling natural language in the wrong way? Suppose: prose is-a poem. Poetry is the superset!
• E.Y. explains that evolution has a mathematical upper bound on the information it can sustain against mutation. Can we apply this to the quasi-evolution of institutional memory in the economy? (The kind of information I’m talking about here is embodied in structure, not personal knowledge inside people’s heads. That’s why I think it will behave differently to intelligence.) Is there an upper bound on the information that corporate capitalism can sustain against economic churn? Suppose there is a limit which is predicated on dumb humans. Can we act as smart humans by designing an information pump that re-inserts this lost information? Can we design company structures that would be amenable to this sort of on-the-fly rebuilding?

General artificial intelligence has consistently failed. Although the number of problems thought to be general has been chipped away with narrow AI, the problem of generality shows no sign of being reduced.

At the same time, a theory of the organization of mind has been constructed from empirical data about the nature of learning. This has two systems.

System one is very slow learning, very fast responding, and can solve some incredibly difficult problems so quickly and definitively they never even appear as problems (example: vision). It’s equivalent to the unconscious mind.

System two is quick learning, and can follow rules, but is laboriously slow at processing. You use system two if you’re “thinking”. It’s equivalent to the conscious mind.

(credit: T. Gilovich, lecture on YouTube, whose content is paralleled here)

Hypothesis: AI has failed because every general AI first tries to imitate system two. They try to achieve something that “reasons”. However in the real brain system two is a backwater, a monitoring unit that does not really participate in the main line of computation. A complete emulation of system two would be a very incomplete mind. Non-progress is an illusion. They are successfully achieving the wrong goal.

Evidence: the problems of Symbolic AI are the same problems as those encountered by a human trying to do general thought with system two. If anything, symbolic AIs are better than humans at doing the wrong job. Both of them bog down on anything complex or where broad swathes of domain knowledge must be taken under consideration. Both are prone to “worrying” (repeated symbolic manipulations that revisit the same few states without developing the problem).

I propose: general AI should first seek to emulate system one.

I’m reading Hayek’s “The Road to Serfdom”. I haven’t got very far in yet, but I’m at the part where he describes the early shift from economic liberalism (at its most successful peak) to socialist hubris. He explains that 19th century economic liberal “rules of thumb” were crude and overbroad, and their manifest problems caused their claims of perfection to be punctured, resulting in abandonment. He suggests that instead, they ought to have been improved.

However, people thought planning was an improvement. Nowadays, they think regulation is an improvement. To anyone who wants liberty and progress, they plainly aren’t. (Their modern defenders are forced to fall back on the moral nobility of poverty as an excuse.)

How can a rule for making rules be structured to avoid the magnetic attraction of coercive means? Non-coercion is a necessity but not a heuristic. So how about this:

Every solution must be structured as a set of rules that are only meant to be applied by the individual to himself.

Interestingly, the first rule of libertarianism, non-aggression, can be structured in this form, thus:

I choose never to initiate force, although I reserve the right to return it.

Over the past few decades, growing evidence from cognitive science has revealed significant limits on the ability of individuals to criticize their own viewpoints. Even the most analytically gifted and experienced among us are susceptible to bias and self-deception to an extent that we (ironically enough) generally fail to appreciate.

[...] Science eventually yields impressive answers because it compels smart people to incessantly try to disprove the ideas generated by other smart people.

[...] In the present cultural climate, altering one’s beliefs in response to anything (facts included) is considered a sign of weakness. Students must be convinced that changing one’s mind in light of the evidence is not weakness: Changing one’s mind is the essence of intellectual growth.

( Thomas W. Martin, Scientific Literacy and the Habit of Discourse, Seed magazine online, 2007-09-21)

Science is certainly powerful as an algorithm, but it seems to me to be making an unjustified assumption that human nature is a given, and must be worked around.

I think it would be an interesting research project: can humans learn to be less biased? I mean a systematic Dune Mentat style “naive” stance that doesn’t so much quash bad ideas (all good scientists already try to do that), but aims to generate fresh ones without falling into the traps set by the unconscious mind.

Nick Szabo over at Unenumerated proposes a means to differentiate science fiction from imminent technology. In essence: require that if the exact form of the technology can’t be defined, then at least the experiments needed to resolve its unknowns can be defined. (For example, SENS meets this criterion.)

Personally I’d say this is a start, but a little narrow in scope. So, I’d suggest the following two generalizations:

1. Allow that technology may be imminent at the n‘th remove. First remove being the definition of imminent technology linked above, second remove being dependent upon the existence of a first remove technology, and so on.
2. Define the idea of improvement velocity (IV). If this is high, an imminent technology can be expected to very quickly become a proven fact. For example, the IV of computer technology in the 1950s through 1970s was immense. It would have been reasonable to treat the highly speculative as expected. If anything, their SF of the time underestimated the improvements that happened.

Combine the two, and you get something like a meaningful definition. Improvement Velocity means: for a velocity of n, it’s reasonable to treat technologies at n‘th remove as imminent. So for example, todays computer tech IV has fallen to something like 2 (it’s reasonable to consider the consequences of 16+ multi-core to be imminent).

Suppose I want to raise money for anti-aging research. A good tool would a prediction market for longevity. But how to achieve it?

I have an idea: the one-living-person-only repayment contract.

How it works: you sell a promise to pay one named living person ONLY, a fixed sum at a fixed gap into the future (eg: £10 in 10 years). You don’t have to pay it to their estate if they die. The obligation to pay can’t be transferred to any creditor or other long-lived corporate entity. It can only be paid to that one person. Usually, the buyer would be the person named as the recipient.

In effect, the person selling the contract is betting against longevity, and the person buying the contract is betting for it. The interplay sets the market price of the contract at certain time scales. For example, 100-year contracts are likely to be quite cheap right now.

I hadn’t updated my old blog in nearly a year. It was supposed to be a wonders-of-science blog, but science Reddit and other insiders did the same job better - I never felt a post by me would add anything. So then, a spring clean (in autumn) and I’ll repurpose the blog into a much more general field.

This blog is now a general scratchpad for my ideas on arbitrary topics, but over the long term it should be drawn together into a narrative by a focus on the nature of thinking and the mind, from both a personal subjective and a scientific evidence-led perspective, and trying to bridge the two.