2. Probability

1. Sample Spaces and Events

An experiment is any action or process that generates observations.

The Sample Space of an Experiment

The sample space of an experiment, denoted by δ, is the set of all possible outcomes of that experiment.

Events

An event is any collection(subset) of outcomes contained in the sample space δ. 

An event is said to be simple if it consists of exactly one outcome and compound if it consists of more than one outcome.

Some Relations from Set Theory

The following concepts from set theory will be used to construct new events from given events:

1. The union of two events A and B, denoted by AUB and read "A or B", is the event consisting of all outcomes that are either in A or in B or in both events.
2. The intersection fo two events A and B, denoted by A∩B and read "A and B", is the event consisting of all outcomes that are both A and B.
3. The complement of an event A, denoted by A', is the set of all outcomes in δ that are not contained in A.
4. When event A and B have no outcomes in common, they are said to be mutually exclusive or disjoint event.

 

2. Axioms, Interpretations, and Properites of Probability

Given an experiment and a sample space δ, the objective of probability is to assign to each event A a number P(A), called the probability of the event A, which will give a precise measure of the chance that A will occur.

AXIOM1: For any event A, P(A)≥0;

AXIOM2: P(δ) =1 

AXIOM3:

1. If A₁,A₂,...A_k is finite collection of mutually exclusively events, then P(A₁UA₂U...A_k)=∑P(A_i)
2. If A₁,A₂,...A_k is an infinite collection of mutually exclusively events, then P(A₁UA₂U...A_k) = ∑P(A_i)

Interpreting Probability

Properties of Probability

PROPOSITION1:For any event A, P(A) = 1-P(A')

PROPOSITION2:If A and B are mutually exclusive, then P(A∩B)=0

PROPOSITION3:For any two events A and B, P(AUB)=P(A)+P(B)-P(A∩B)

Determining Probabilities Systematically

Equally Likely Outcomes

When the various outcomes of an experiment are equally likely, then the task of computing probabilities reduces to counting. In particular, if N is the number of outcomes in a sample space and N(A) is the number of outcomes contained in an event A, then P(A) = N(A)/N

3. Counting Techniques

There are, however, many experiment for which the effort involved in constructing such a list is probihitive because N is quite large. These rules are also useful in many problems involving outcomes that are not equally likely.

The Product Rule for Ordered Pairs

PROPOSITION:If the element or object of an ordered pair can be selected in _n1 ways, and for each of these _n1 ways the second element of the pair can be selected in _n2 ways, then the number of pairs is _n1n2.

Tree Disgrams

Tree structure, a way of representing the hierarchical nature of a structure in a graphical form

A More General Product Rule

If a six-side die is tossed five times in succession rather than just twice, the each possible outcome is an ordered collection of five number such as (1,2,3,4,1) or (5,6,2,3,4).

We will call an ordered collection of k objected a k-tuple(so a pair is 2-tuple and a triple is a 3-tuple)

Product Rule for k-Tuples

Suppose a set consists of ordered collections of k elements and that

there are n₁ possible choices for the first elements;

for each choice of the first element, there are n₂ possible choices of the second elements;...;for each possible choices of the first k-1 elements, there are n_k choices of the kth element.

Then there are n₁n₂...n_k possible for k-tuples.

Permutations

Any ordered sequence of k objects taken from a set of n distinct objects is called a permutation of size k of the objects. The number of permutations of size k that can be constructed from the n objects is denoted by P_k,n.

For any positive integer m, m! is read "m factorial" and is defined by m!=m(m-1)...(2)(1). Also 0!=1.

Using the factorial notation yields:

P_k,n = n!/(n-k)!

Combination

Given a set of n distinct objects, any unordered subset of size k of the objects is called a combination. The number of combinations of size k that  can be formed from n distinct objects will be denoted by(ⁿ_k). This notation is more common in probability in C_k,n, which would be analogous to notation for permutation.

C_k,n = n!/k!(n-k)!

 

4. Conditional Probability

We will use the notation P(A|B) to represent the conditional probability of A given that the event B has occured.

The Definition of Conditional Probability

For any two events A and B with P(B)>0, the conditional probability of A given that B has occured is definied by:

P(A|B) = P(A∩B)/P(B)

The Multiplication Rule for P(A∩B)

P(A∩B) = P(A|B)*P(B)

The multiplication rule is most useful when the experiment consists of several stages in succession.

Bayes' Theorem

The Law of Total Probability

Let A₁,...,A_k be mutually exclusive and exhaustive events. (The events are exhaustive if one A_i must occur, so that A₁U...UA_k=δ)Then for any other event B,

P(B) = P(B|A₁)P(A₁)+...+P(B|A_k)P(A_k) = ∑P(B|A_i)P(A_i)

Bayes' Theorem

Let A₁,...,A_k be mutually exclusive and exhaustive events with P(A_i)>0 for i=1,...k. Then for any other event B for which P(B)>0

P(A_j|B) = P(A_j∩B)/P(B) = P(B|A_j)P(A_j)/∑P(B|A_i)*P(A_i)

 

5. Independence

Two events A and B are independent if P(A|B) = P(A) and are dependent otherwise.

P(A∩B) When Events Are Independent

PROPOSITION: A and B are independent if and only if P(A∩B)=P(A)*P(B)

Independence of More Than Two Events

Events A₁,...,A_n are mutually independent if for every k (k=2,3,...,n) and every subset of indices i₁,i₂,...,i_k,

P(A_i1∩A_i2∩...∩A_ik) = P(A_i1)*P(A_i2)*...*P(A_ik)