The Rational Future Institute

The Principles of Rational Inference

The principles of rational thinking are the principles of inference. These principles tell us the only proper ways to infer new knowledge from things we already know.

Logical Deduction
Deduction is the process of reasoning from general rules to more specific facts. The proper way to do deduction is to avoid contradictions, that is, to reason logically.

Bayesian Induction
Induction is the process of reasoning from particulars to generalities, and from data to theories. The term "Bayesian" (pronounced bays-ian) refers to a formal rule that describes the proper way to reason inductively. In particular, the Bayesian rule tells us how to account for false-positives.

There's much more to say about these rules, and we shall begin with some simple explanations of these definitions. For the curious and intrepid, we will follow up with arguments for why these principles are special and complete, and for the philosophy of rationality.

A Simple Explanation of Deduction

Deductive logic is the kind of reasoning that most people are familiar with.

In deduction, we start from some general facts that we know, and we reason to specific facts. For example, if we know that all trains are running 10 minutes late today, then we can reason to the conclusion that a train that normally arrives at 3:00pm today will actually arrive at 3:10pm. We can write this reasoning down as follows:

I KNOW THAT: All trains are running 10 minutes late.
I KNOW THAT: A train normally arrives at 3:00pm.
I KNOW THAT: the train that normally arrives at 3:00pm WILL arrive at 3:10pm.

More generally, deduction works not just from facts that we know, but from facts that we assume to be true.  For example, I know that, if all trains were running late by ten minutes, then the 3:00pm train would arrive at 3:10pm.

IF : All trains are running 10 minutes late.
AND IF: A train normally arrives at 3:00pm.
THEN: a train that normally arrives at 3:00pm WOULD arrive at 3:10pm.

The assumptions we start from are called logical premises.

This kind of reasoning can be seen as an example of the principle of non-contradiction. Deductive reasoning removes potential conclusions that would lead to a contradiction. One potential conclusion that is excluded is the possibility that the 3:00pm train is on time. If the 3:00pm train were on time, that would contradict the knowledge or assumption that ALL trains were running 10 minutes late. The same exclusion applies to the possibility that the 3:00pm train will arrive at 3:11pm, because then the 3pm train would be running 11 minutes late.

When we go through the process of logical deduction, we are inferring which potential facts we can accept without creating a contradiction with our starting premises.

Historically, deduction has been regarded as the kind of inference that gives us certainty. If we know all trains are running 10 minutes late today, and the Orient Express is a train, then we know with certainty that the Orient Express is running 10 minutes late today.

A Simple Explanation of Induction

Induction uses the past as a guide to the future. By looking at our experiences, we infer what the future will probably be like. Induction cannot give us certainty, but it can make us rationally confident in our theories.

For example, we are all very confident that heavy objects that are released near the Earth's surface will fall to the ground. Where did this confidence come from? It came from making millions of observations of heavy things falling to the ground when dropped. The more regular our experiences with gravity, the more confident we become in the law of gravity.

Inductive inferences are not infallible. Induction is about probability, and we must possess a basic understanding of probability before we can be good at reasoning.

Probability is a way of dealing with incomplete or non-existent information. For example, it may rain tomorrow or it may not.  Probability gives us a way to quantify our confidence in each possibility. If we're in the tropics, and it's the rainy season, we might conclude from experience that the probability of rain tomorrow is much greater than the probability of no rain, say, a 99% probability of rain tomorrow, and a 1% chance of no rain at all.

How do we quantify our probabilities?

Initially, before we have any evidence to favor any one outcome, we have to distribute the probability equally among each outcome. This is because we would need evidence in order to give a reason why any one outcome was more probable than the others. Also, since we know one of the possible outcomes will occur, we know that the sum of the probabilities of all the possible outcomes must be 100%.

As we collect evidence, we incrementally update our estimate of the probability of each outcome. Each new experience moves the probability estimate up or down. With each new piece of evidence we factor in our prior estimate of probability, and take into account the specificity of the predictions of our theory. We also try to account for the possibility that the evidence might have been received by chance.

There's a formula called Bayes' Theorem which tells us precisely how to update our probabilities in light of new evidence. The formula looks like this:

P(T|E) =
P(T) P(E|T)
P(T) P(E|T) + P(~T) P(E|~T)

Don't be intimidated by the complexity of this equation; it is mostly intuitive. 

The following are the meanings of the terms in this formula:

P(T|E) the conditional probability that theory T is correct given the new evidence E.

P(T) the prior probability that theory T was correct before receiving the new evidence.

P(E|T) the probability that we might observe evidence E if T were true.

P(~T) the prior probability that theory T is incorrect before receiving new evidence E.

P(E|~T) the probability that we might observe evidence E if T were incorrect.

P(T) represents our prior confidence in the theory. If we came to the new evidence believing that the theory was wildly improbable, then the theory would have to make highly specific predictions (P(E|T)) about  the evidence in order for the evidence to turn our opinion around in one stroke. However, evidence can accumulate in favor of a theory. If the theory makes weak but successful predictions over a long period of time, the accumulation of evidence can change our minds and give us confidence in the theory through repeated applications of Bayes' Theorem.

The formula also warns us to take account of false positive evidence. What are the odds that we would have found the evidence even if the theory was false?

When a pharmaceutical company is testing a drug, they give half of the test subjects the drug, and half the subjects a placebo. By doing this, they get an idea of how many patients would recover without the drug. That is, they get an idea of P(E|~T). They know the approximate odds of seeing a recovery if the drug is not present or ineffective. For example, if I have a theory that marshmallows cure the common cold, and I find that after 7 days, most of the cold sufferers were cured, no one would be impressed. Why? Because, after 7 days, most cold sufferers would have cured themselves through their own immune response. If the sweet treats cured colds in just one day instead of 7, my marshmallow theory would be rationally convincing.

Thus, the last term in the denominator of Bayes' Theorem takes false positives into account.

For a more in-depth explanation of Bayes' Theorem, see, for example, An Intuitive Explanation of Bayes' Theorem by Eliezer Yudkowsky.

Philosophical Considerations

Are there any other ways to do inference? What are the limitations of Bayes' Theorem? What is knowledge?

These are philosophical questions which we answer on our philosophy page.

Copyright 2011 Rational Future Institute NFP