Elements of Probability Theory

In any situation in which the outcome of an event is uncertain, we can talk about the probabilities of such outcomes, i.e. how likely they are. The most popular example is tossing of a fair coin: since a coin is fair and there are no other outcomes except the heads and tails, they are both equally probable and we say that their probabilities are eqaual to 1/2. As a branch of mathematics, the discipline of probability provides formal methods for quantifying chances of such uncertain outcomes. In this chapter, we first introduce the basic probability concepts, and show how to calculate the probabilities of some events using the rules from probability theory. Next, we present the concept of a random variable which enable us to move from the event outcomes themselves to a numerical function of that outcomes. For example, we can define a random variable X, being the number of heads we get from tossing two coins: X could then be 0, 1, or 2 heads. Except for such discrete random variable, the continuous one can take any value in some interval on the real number line. We characterize here the probability distributions (i.e. the probability with which they take on the various values in their range) of most representative random variables. Then, we also discuss probability models for the joint (simultaneous) behaviour of two or more random variables which is important for describing their dependency. At the end, after presenting the central limit theorem which states about limit distribution of sample mean, the most important theorem for statistical analysis, we show how to apply computer simulation methods to probability problems and model random events.