This condensed study guide connects definitions to the mechanisms that use them. Read the short examples to understand why a rule works, then use the reference tables to recall it. The topic list follows the catalog description below; it is not a verified checklist of the current objective assessment. Your current course materials take precedence.

WGU’s June 2026 catalog defines C959 around logic and proofs, Boolean algebra and functions, set theory, finite and infinite sequences and series, relations, graphs, and trees. C960 covers searching and sorting, Big-O, number theory and cryptography, recursion and induction, counting, probability, and models of computation. (wgu.edu)

For each topic, ask: what object am I working with, what does its definition allow, and which operation answers the question? A set of choices, an ordered list of choices, and a probability distribution may describe the same situation, but call for different calculations.

Part IIC960 — Discrete Mathematics II#

Reading route. C960 reuses C959's logic, sets, functions, and finite sums. Follow these chains rather than treating every formula as an isolated fact:

1. Trace an algorithm → count its work → compare searches and sorts. Recurrences count recursive work; induction proves claims about it.
2. Divisibility and remainders → gcd → modular inverse → RSA. Binary place value makes repeated squaring efficient.
3. Count outcomes → assign probabilities → measure outcomes with a random variable → choose a distribution.
4. Strings and languages → state machines → regular expressions and limits of finite memory.

11Algorithms and pseudocode#

The mathematical objects are now inputs to procedures. First follow the changing state to find what a procedure does. Only then ask whether it is correct and how its work grows; output, correctness, and cost are different questions.

An algorithm is a finite, unambiguous, effective procedure that transforms inputs into outputs.

A proper algorithm has:

• specified inputs;
• specified outputs;
• precise steps;
• termination after finitely many steps;
• executable operations;
• correctness for all allowed inputs.

11.1Common algorithmic structures#

Sequence#

Execute statements in order.

Selection#

Choose a branch:

if condition
    action A
else
    action B

Iteration#

Repeat using a loop.

Recursion#

A procedure calls itself on a smaller or simpler input.

11.2Reading pseudocode#

Key distinctions:

• Assignment changes a variable:
  $$x\leftarrow x+1$$
• Equality compares values:
  $$x=x+1$$
  would be false as an ordinary mathematical equation.
• return usually ends the current procedure immediately.
• A loop endpoint may be inclusive or exclusive.
• Array indexing may begin at $0$ or $1$.
• Updating a variable inside a loop affects subsequent iterations.

When tracing, maintain a small table containing:

• current iteration;
• relevant variable values;
• output produced so far.

Ignore variables that never change.

11.3Correctness#

An algorithm is correct if it produces the specified output for every permitted input.

A loop invariant is a statement that remains true:

1. before the first iteration;
2. after each iteration;
3. when the loop ends.

A correctness proof typically shows:

• initialization: invariant starts true;
• maintenance: one iteration preserves it;
• termination: invariant plus stopping condition implies the result.

This is induction expressed in algorithmic form.

12Complexity and Big-O#

Use the trace from the previous chapter, but replace a particular input size with n. Count how often the main operation runs. A bound summarizes that count; it does not change what the algorithm computes.

Complexity measures resource growth as input size $n$ increases.

Common resources are:

• time;
• memory;
• comparisons;
• arithmetic operations;
• recursive calls.

The focus is asymptotic growth, not exact wall-clock time.

12.1Best, worst, and average case#

• Best case: minimum operations among size-$n$ inputs.
• Worst case: maximum operations.
• Average case: expected operations under an assumed input distribution.

Unless specified, complexity questions often ask for worst-case behavior.

Case and bound are separate choices. First choose which inputs to describe, then bound their operation count. Linear search has constant best-case cost and linear worst-case cost. O does not mean “worst,” and Ω does not mean “best”; either notation can bound a best-case, worst-case, or average-case function.

12.2Big-O, Omega, and Theta#

Big-O#

$$f(n)=O(g(n))$$

means $g$ is an asymptotic upper bound. Formally, there are constants $c>0$ and $n_0$ such that:

$$0\le f(n)\le cg(n)$$

for all $n\ge n_0$.

Big-Omega#

$$f(n)=\Omega(g(n))$$

means $g$ is an asymptotic lower bound.

Big-Theta#

$$f(n)=\Theta(g(n))$$

means $g$ is both an upper and lower bound—a tight growth classification.

Big-O is not necessarily exact. A linear function is also technically $O(n^2)$, though $\Theta(n)$ is the informative tight classification.

12.3Simplifying complexity expressions#

Discard:

• constant factors;
• lower-order terms.

$$7n^3+20n^2+900 = \Theta(n^3)$$

$$500n\log n+3n = \Theta(n\log n)$$

As $n$ grows, the dominant term determines behavior.

12.4Growth hierarchy#

From slower to faster:

$$1 < \log n < \sqrt n < n < n\log n < n^2 < n^3 < c^n < n! < n^n$$

for fixed $c>1$.

Logarithm bases differ only by a constant factor:

$$\log_a n=\frac{\log_b n}{\log_b a}$$

so all fixed bases represent:

$$\Theta(\log n)$$

Polynomial growth is generally considered scalable relative to exponential or factorial growth.

12.5Recognizing loop complexity#

Fixed work#

x = a + b

$$\Theta(1)$$

Linear loop#

for i = 1 to n

$$\Theta(n)$$

Step size other than one#

for i = 1 to n step 10

Still:

$$\Theta(n)$$

Dividing the iteration count by a fixed constant does not change asymptotic class.

Doubling or halving#

i = 1
while i < n
    i = 2i

After $k$ iterations:

$$i=2^k$$

Stop near $2^k=n$, so:

$$k=\log_2n$$

Complexity:

$$\Theta(\log n)$$

Independent nested loops#

for i = 1 to n
    for j = 1 to n

$$n\cdot n=n^2$$

so:

$$\Theta(n^2)$$

Triangular loop#

for i = 1 to n
    for j = 1 to i

Total iterations:

$$1+2+\cdots+n=\frac{n(n+1)}2$$

therefore:

$$\Theta(n^2)$$

Linear outer, logarithmic inner#

for i = 1 to n
    j = 1
    while j < n
        j = 2j

$$\Theta(n\log n)$$

Sequential blocks#

For one block followed by another, add costs and retain the dominant term:

$$\Theta(n)+\Theta(n^2)=\Theta(n^2)$$

Nested blocks#

For genuinely nested independent repetitions, multiply counts.

12.6Common operation sums#

$$1+1+\cdots+1=n$$

$$1+2+\cdots+n=\Theta(n^2)$$

$$1^2+2^2+\cdots+n^2=\Theta(n^3)$$

$$1+2+4+\cdots+2^k=\Theta(2^k)$$

If $2^k\approx n$, the final sum is $\Theta(n)$, even though there are only $\Theta(\log n)$ levels.

13Searching and sorting#

Searching asks where an item is; sorting arranges items so later operations can exploit order. Linear search works without order. Binary search earns its speed by using sorted order to rule out half the candidates at each step.

13.1Linear search#

Inspect elements one by one until:

• the target is found, or
• the collection ends.

No sorting is required.

• best case: $\Theta(1)$;
• worst case: $\Theta(n)$;
• average case: $\Theta(n)$.

13.2Binary search#

Binary search requires sorted data.

1. inspect the middle element;
2. if equal, stop;
3. otherwise discard the half that cannot contain the target;
4. repeat.

After $k$ divisions, the remaining size is roughly:

$$\frac n{2^k}$$

Set this near $1$:

$$2^k\approx n$$

so:

$$k\approx\log_2n$$

Worst-case complexity:

$$\Theta(\log n)$$

Binary search on a linked list may not provide the same practical benefit because accessing the middle is not constant-time.

Two sorting terms recur below. Stable means equal-key records keep their original relative order. In-place means constant auxiliary storage for rearranging the items. Quicksort is often called in-place while its recursion stack is counted separately; check the space convention in the question.

13.3Bubble sort#

Repeatedly compare adjacent elements and swap inverted pairs.

After a full pass, a largest remaining element “bubbles” toward the end.

• worst case: $\Theta(n^2)$;
• average: $\Theta(n^2)$;
• optimized best case: $\Theta(n)$;
• stable;
• in-place.

13.4Selection sort#

Repeatedly find the smallest remaining item and place it next.

It performs roughly:

$$(n-1)+(n-2)+\cdots+1$$

comparisons:

$$\Theta(n^2)$$

It performs relatively few swaps but remains quadratic.

Usually:

• not stable in its basic form;
• in-place.

13.5Insertion sort#

Maintain a sorted prefix and insert each next item into its proper position.

• best case on already sorted data: $\Theta(n)$;
• average and worst: $\Theta(n^2)$;
• stable;
• in-place;
• efficient for small or nearly sorted inputs.

13.6Merge sort#

1. divide the list into halves;
2. recursively sort each half;
3. merge the sorted halves.

There are approximately:

$$\log_2n$$

levels, and each level performs $\Theta(n)$ merge work:

$$\Theta(n\log n)$$

Merge sort is:

• stable;
• reliably $\Theta(n\log n)$;
• usually requires $\Theta(n)$ auxiliary memory for arrays.

13.7Quicksort#

1. choose a pivot;
2. partition elements around it;
3. recursively sort partitions.

If partitions are reasonably balanced:

$$\Theta(n\log n)$$

If pivots repeatedly create highly unbalanced partitions:

$$\Theta(n^2)$$

Typical properties:

• average $\Theta(n\log n)$;
• worst $\Theta(n^2)$;
• often in-place;
• typically not stable.

13.8Algorithm categories#

Brute force#

Enumerate many or all possibilities.

Simple and general, but often exponential or factorial.

Greedy#

Choose the locally best option at every step.

A greedy method is correct only when the problem has a structure proving local choices lead to a global optimum.

Kruskal and Prim are greedy.

Divide and conquer#

Divide into subproblems, solve them, and combine.

Merge sort and binary search use this structure.

Dynamic programming#

Solve overlapping subproblems once and store their answers.

It converts many exponential recursive procedures into polynomial algorithms.

Naive Fibonacci recomputes the same values repeatedly; memoization removes that duplication.

14Number theory#

14.1Divisibility#

14.2Division algorithm#

For integer $a$ and positive integer $n$, there exist unique integers $q,r$ such that:

$$a=qn+r$$

with:

$$0\le r<n$$

Then:

$$q=a\operatorname{div}n$$

and:

$$r=a\bmod n$$

The nonnegative remainder requirement matters with negative dividends.

For example:

$$-21=(-1)(33)+12$$

so:

$$-21\bmod33=12$$

not $-21$.

14.3Modular congruence#

$$a\equiv b\pmod n$$

means:

$$n\mid(a-b)$$

Equivalent interpretations:

• $a,b$ have the same remainder modulo $n$;
• they represent the same residue class.

Congruence behaves well under addition and multiplication:

If:

$$a\equiv b\pmod n$$

and:

$$c\equiv d\pmod n$$

then:

$$a+c\equiv b+d\pmod n$$

and:

$$ac\equiv bd\pmod n$$

You may reduce values modulo $n$ after every step.

Ordinary division is not generally valid modulo $n$. It requires an invertible divisor.

14.4Prime numbers and factorization#

A prime is an integer greater than $1$ with exactly two positive divisors:

$$1\text{ and itself}$$

A composite number has a nontrivial factorization.

To test whether $N$ is prime, it is sufficient to test prime divisors no larger than:

$$\sqrt N$$

If $N=ab$ is composite, at least one of $a,b$ must be at most $\sqrt N$.

The fundamental theorem of arithmetic says every integer greater than $1$ has a unique prime factorization apart from order.

14.5Greatest common divisor and least common multiple#

$$\gcd(a,b)$$

is the greatest positive integer dividing both.

Two integers are coprime when:

$$\gcd(a,b)=1$$

The least common multiple satisfies:

$$\gcd(a,b)\operatorname{lcm}(a,b)=|ab|$$

14.6Euclidean algorithm#

Use repeated division:

$$a=bq+r$$

and the identity:

$$\gcd(a,b)=\gcd(b,r)$$

Continue until the remainder becomes zero. The last nonzero remainder is the gcd.

For example:

$$252=2(105)+42$$

$$105=2(42)+21$$

$$42=2(21)+0$$

Therefore:

$$\gcd(252,105)=21$$

The algorithm is efficient because remainders decrease rapidly.

14.7Bézout’s identity and extended Euclid#

Bézout’s identity states that there exist integers $s,t$ such that:

$$\gcd(a,b)=sa+tb$$

The extended Euclidean algorithm finds these coefficients by back-substituting through Euclid’s equations.

For the preceding gcd:

$$21=105-2(42)$$

and:

$$42=252-2(105)$$

so:

$$21 = 105-2(252-2(105)) = 5(105)-2(252)$$

Thus:

$$21=-2(252)+5(105)$$

14.8Modular inverses#

An inverse of $a$ modulo $n$ is a number $x$ satisfying:

$$ax\equiv1\pmod n$$

It exists exactly when:

$$\gcd(a,n)=1$$

If extended Euclid produces:

$$1=sa+tn$$

then reducing modulo $n$ gives:

$$sa\equiv1\pmod n$$

so:

$$a^{-1}\equiv s\pmod n$$

Reduce a negative coefficient to the standard range by adding multiples of $n$.

Undoing multiplication. Modulo 7, the inverse of 3 is 5 because 3×5 = 15 ≡ 1. Thus 3x ≡ 6 gives x ≡ 5×6 ≡ 2. Modulo 8, however, 2 has no inverse: 2 times any integer stays even and cannot be congruent to 1. The gcd condition tells you when this method is legal.

14.9Linear congruences#

Consider:

$$ax\equiv b\pmod n$$

Let:

$$d=\gcd(a,n)$$

A solution exists exactly when:

$$d\mid b$$

If $d=1$, multiply by the inverse of $a$:

$$x\equiv a^{-1}b\pmod n$$

If $d>1$ divides $b$, the congruence has $d$ incongruent solutions modulo $n$.

14.10Chinese remainder theorem#

For pairwise coprime moduli:

$$n_1,n_2,\ldots,n_k$$

a system:

$$x\equiv a_i\pmod{n_i}$$

has a unique solution modulo:

$$N=n_1n_2\cdots n_k$$

A constructive method is:

1. compute $N$;
2. set:
   $$N_i=N/n_i$$
3. find $y_i$ satisfying:
   $$N_iy_i\equiv1\pmod{n_i}$$
4. calculate:
   $$x\equiv\sum_i a_iN_iy_i\pmod N$$

For small systems, substitution is often faster.

15Number bases#

The division algorithm already splits a number into quotient and remainder. Base conversion repeats that operation: the remainder is the last digit, and the quotient contains the digits still to recover.

A base-$b$ numeral:

$$(d_kd_{k-1}\cdots d_0)_b$$

represents:

$$\sum_{i=0}^{k}d_ib^i$$

Allowed digits are:

$$0,1,\ldots,b-1$$

15.1Converting to decimal#

Expand by place value:

$$(2301)_4 = 2\cdot4^3+3\cdot4^2+0\cdot4+1$$

15.2Converting from decimal#

Repeatedly divide by the target base:

1. divide by $b$;
2. record the remainder;
3. divide the quotient again;
4. read remainders from last to first.

The first remainder produced is the least significant digit.

15.3Binary, octal, and hexadecimal#

Binary uses base $2$.

Octal uses base $8$, so each octal digit corresponds to 3 bits.

Hexadecimal uses base $16$, so each hex digit corresponds to 4 bits.

The number of bits needed to represent positive $N$ is:

$$\lfloor\log_2N\rfloor+1$$

The number of bits needed to represent all integers from $0$ through $N$ is:

$$\lceil\log_2(N+1)\rceil$$

16Modular exponentiation and cryptography#

You can now reduce remainders, find gcds, and undo multiplication with a modular inverse. Cryptography combines those operations. Binary exponents keep the arithmetic small; Euler’s theorem explains when powers repeat.

16.1Fast modular exponentiation#

To calculate:

$$a^k\bmod n$$

do not construct $a^k$ directly.

Write $k$ in binary and repeatedly square:

$$a,\quad a^2,\quad a^4,\quad a^8,\ldots$$

Reduce modulo $n$ after every multiplication.

If:

$$k=13=8+4+1$$

then:

$$a^{13}=a^8a^4a$$

Use only powers corresponding to 1-bits in the exponent.

This requires only:

$$O(\log k)$$

squaring/multiplication steps.

16.2Euler’s totient function#

$$\varphi(n)$$

counts positive integers no greater than $n$ that are coprime to $n$.

For prime $p$:

$$\varphi(p)=p-1$$

For distinct primes $p,q$:

$$\varphi(pq)=(p-1)(q-1)$$

More generally, if:

$$n=\prod_i p_i^{e_i}$$

then:

$$\varphi(n) = n\prod_{p_i\mid n}\left(1-\frac1{p_i}\right)$$

16.3Fermat’s little theorem#

If $p$ is prime and $p\nmid a$, then:

$$a^{p-1}\equiv1\pmod p$$

Equivalent general form:

$$a^p\equiv a\pmod p$$

This can reduce large exponents modulo $p-1$ when $a$ is not divisible by $p$.

16.4Euler’s theorem#

If:

$$\gcd(a,n)=1$$

then:

$$a^{\varphi(n)}\equiv1\pmod n$$

Fermat’s theorem is the prime-modulus special case.

16.5Cryptographic terminology#

• Plaintext: original information.
• Ciphertext: encrypted information.
• Encryption: transforms plaintext into ciphertext using a key.
• Decryption: reverses the transformation with the appropriate key.
• Symmetric encryption: the same shared secret, or directly related secrets, perform encryption and decryption.
• Public-key encryption: a public key encrypts or verifies; a private key decrypts or signs.
• Encoding: changes representation but does not provide secrecy.
• Hashing: maps input to a fixed-size digest and is intended to be one-way.

16.6Shift cipher#

Represent letters as residues, often:

$$A=0,\ B=1,\ldots,Z=25$$

Encryption:

$$E(x)=x+k\pmod{26}$$

Decryption:

$$D(y)=y-k\pmod{26}$$

A Caesar cipher is insecure because there are only 26 possible shifts.

16.7Affine cipher#

Encryption:

$$E(x)=ax+b\pmod m$$

The multiplier $a$ must be invertible modulo $m$:

$$\gcd(a,m)=1$$

Decryption:

$$D(y)=a^{-1}(y-b)\pmod m$$

The modular inverse is what makes the transformation reversible.

16.8RSA#

RSA connects nearly every number-theory mechanism in the course.

Key generation#

Choose distinct primes:

$$p,\ q$$

Compute:

$$n=pq$$

Compute:

$$\varphi(n)=(p-1)(q-1)$$

Choose a public exponent $e$ satisfying:

$$1<e<\varphi(n)$$

and:

$$\gcd(e,\varphi(n))=1$$

Find the private exponent:

$$d\equiv e^{-1}\pmod{\varphi(n)}$$

The public key is:

$$(n,e)$$

The private information includes $d$, and in real implementations usually $p,q$ as well.

Encryption#

For message representative $m$ with:

$$0\le m<n$$

calculate:

$$c\equiv m^e\pmod n$$

Decryption#

$$m\equiv c^d\pmod n$$

Use fast modular exponentiation for both.

Why it works#

Because:

$$ed\equiv1\pmod{\varphi(n)}$$

there is an integer $k$ such that:

$$ed=1+k\varphi(n)$$

When gcd(m, n) = 1, Euler’s theorem gives:

$$m^{ed} = m^{1+k\varphi(n)} \equiv m(m^{\varphi(n)})^k \equiv m \pmod n$$

The Euler argument above requires the message to be coprime to n. RSA also recovers other message representatives: work modulo each prime separately. If p divides the message, both the original and recovered value are 0 modulo p; otherwise Fermat’s theorem applies. Do the same modulo q, then use the Chinese remainder theorem to obtain the same message modulo pq. Textbook RSA here illustrates the arithmetic; real encryption also needs a secure padding scheme.

RSA traps#

• Find $d$ modulo $\varphi(n)$, not modulo $n$.
• $e$ must be coprime to $\varphi(n)$.
• The message block must be smaller than $n$.
• Reduce after every multiplication.
• In toy problems, factorization of $n$ is intentionally feasible; real RSA uses enormous primes.

17Recursion and recurrence relations#

Recall recursive sequences from C959: an initial value plus a rule determines later values. A recursive program uses the same idea to solve smaller inputs. Distinguish the value it returns from the number of operations it performs; each may obey a different recurrence.

17.1Recursive definitions#

A recursive definition contains:

1. one or more base cases;
2. a rule defining larger objects from smaller ones.

For factorial:

$$0!=1$$

$$n!=n(n-1)! \qquad(n\ge1)$$

A recursive algorithm must move toward a base case. Otherwise it fails to terminate.

17.2Call stack and unwinding#

When a recursive function calls itself, the current operation waits on the call stack.

For:

$$f(n)=n+f(n-1),\qquad f(0)=0$$

the calls expand:

$$f(3)=3+f(2)$$

$$=3+2+f(1)$$

$$=3+2+1+f(0)$$

Then return values unwind:

$$f(0)=0,\quad f(1)=1,\quad f(2)=3,\quad f(3)=6$$

For subtraction, division, or multiple recursive calls, the return order matters.

17.3Recurrence relations#

A recurrence defines sequence terms from preceding terms.

The order is the maximum number of positions referenced backward.

$$a_n=3a_{n-1}+2$$

is first-order.

$$a_n=a_{n-1}+a_{n-2}$$

is second-order.

Initial conditions are required to select one particular solution.

17.4Arithmetic and geometric recurrences#

Arithmetic:

$$a_n=a_{n-1}+d$$

Solution:

$$a_n=a_0+nd$$

Geometric:

$$a_n=ra_{n-1}$$

Solution:

$$a_n=a_0r^n$$

17.5First-order linear recurrence#

For:

$$a_n=ra_{n-1}+c$$

repeated substitution gives:

$$a_n=r^na_0+c(1+r+\cdots+r^{n-1})$$

When $r\ne1$:

$$a_n = r^na_0+c\frac{r^n-1}{r-1}$$

When $r=1$:

$$a_n=a_0+nc$$

This form models repeated growth plus a fixed deposit or input.

The timing matters. “Add the deposit, then apply interest” and “apply interest, then add the deposit” produce different recurrences.

17.6Solving by telescoping#

If:

$$a_n-a_{n-1}=f(n)$$

sum both sides:

$$a_n-a_0 = \sum_{k=1}^{n}f(k)$$

so:

$$a_n=a_0+\sum_{k=1}^{n}f(k)$$

This converts the recurrence into a known finite sum.

17.7Second-order homogeneous recurrences#

Suppose:

$$a_n=c_1a_{n-1}+c_2a_{n-2}$$

Try a solution:

$$a_n=r^n$$

Substitution yields:

$$r^2-c_1r-c_2=0$$

This is the characteristic equation.

Distinct roots#

If roots are $r_1,r_2$:

$$a_n=Ar_1^n+Br_2^n$$

Use two initial conditions to determine $A,B$.

Repeated root#

If one root $r$ has multiplicity two:

$$a_n=(A+Bn)r^n$$

The factor $n$ supplies the second independent solution.

17.8Fibonacci sequence#

$$F_0=0,\qquad F_1=1$$

$$F_n=F_{n-1}+F_{n-2}$$

Its characteristic equation is:

$$r^2-r-1=0$$

with roots:

$$\varphi=\frac{1+\sqrt5}{2}$$

and:

$$\psi=\frac{1-\sqrt5}{2}$$

leading to:

$$F_n=\frac{\varphi^n-\psi^n}{\sqrt5}$$

For exam purposes, the recurrence mechanism and growth behavior are more important than reproducing this closed form.

17.9Runtime recurrences#

A runtime recurrence counts calls and local work, not the returned value. In T(n) = 2T(n/2) + Θ(n), the 2 counts recursive calls, n/2 is their input size, and Θ(n) is the work outside them. For merge sort, each level processes n items in total, not n per call; about log₂n levels give Θ(n log n).

Linear recursion#

$$T(n)=T(n-1)+\Theta(1)$$

Unrolling produces $n$ constant contributions:

$$T(n)=\Theta(n)$$

Linear work at every decrement#

$$T(n)=T(n-1)+\Theta(n)$$

This sums:

$$n+(n-1)+\cdots+1$$

so:

$$T(n)=\Theta(n^2)$$

Repeated halving#

$$T(n)=T(n/2)+\Theta(1)$$

There are $\log n$ levels:

$$T(n)=\Theta(\log n)$$

Merge-sort form#

$$T(n)=2T(n/2)+\Theta(n)$$

There are $\log n$ levels and $\Theta(n)$ work per level:

$$T(n)=\Theta(n\log n)$$

Naive Fibonacci#

$$T(n)=T(n-1)+T(n-2)+\Theta(1)$$

The recursion tree branches and repeats subproblems, giving exponential growth.

17.10Simplified Master theorem#

This form assumes a fixed number a ≥ 1 of subproblems, a fixed shrink factor b > 1, and constant-size base cases. It does not apply to recurrences such as T(n−1), where the input shrinks by subtraction rather than division.

For:

$$T(n)=aT(n/b)+f(n)$$

compare $f(n)$ with:

$$n^{\log_ba}$$

• If $f(n)$ is polynomially smaller, recursive leaves dominate:
  $$T(n)=\Theta(n^{\log_ba})$$
• If they are the same order:
  $$T(n)=\Theta(n^{\log_ba}\log n)$$
• If $f(n)$ is polynomially larger and regularity conditions hold:
  $$T(n)=\Theta(f(n))$$

This is a structured way to read divide-and-conquer recurrences.

18Mathematical induction#

Induction proves statements indexed by integers.

To prove $P(n)$ for every $n\ge n_0$:

Base case#

Prove:

$$P(n_0)$$

Inductive hypothesis#

Assume:

$$P(k)$$

for an arbitrary $k\ge n_0$.

Inductive step#

Use the hypothesis to prove:

$$P(k+1)$$

This establishes the chain:

$$P(n_0)\to P(n_0+1)\to P(n_0+2)\to\cdots$$

The inductive hypothesis is a temporary assumption about $k$, not permission to assume the $k+1$ case.

18.1Induction for sums#

To prove a formula for:

$$1+2+\cdots+n$$

the $k+1$ case begins by separating the new term:

$$1+2+\cdots+k+(k+1)$$

Then replace the first portion using the inductive hypothesis.

That is the standard mechanism for identities involving cumulative sums.

For the sum identity 1 + … + n = n(n+1)/2, the base n = 1 gives 1 = 1. Assume the identity for k. Then the next sum is k(k+1)/2 + (k+1) = (k+1)(k+2)/2, exactly the required formula for k+1. The proof works because the new case contains the old case plus one new term.

18.2Induction for divisibility#

To prove:

$$d\mid f(n)$$

show:

1. $d\mid f(n_0)$;
2. assuming:
   $$f(k)=dm$$
   for some integer $m$, rewrite $f(k+1)$ as:
   $$dm+d(\text{integer})$$

and factor out $d$.

18.3Induction for inequalities#

The inductive hypothesis gives a bound on the $k$-case. Manipulate it in a direction that produces the required $k+1$ bound.

Be careful:

• multiplying by a negative reverses inequalities;
• taking reciprocals of positive values reverses inequalities;
• a true base case does not repair an invalid inductive step.

18.4Strong induction#

Strong induction assumes:

$$P(n_0),P(n_0+1),\ldots,P(k)$$

and proves $P(k+1)$.

Use it when the next case depends on several earlier cases.

Typical uses:

• prime factorization;
• recursively reducing by varying amounts;
• Fibonacci-like recurrences;
• tiling or construction problems.

Strong induction is not logically stronger than ordinary induction; it gives a more convenient hypothesis.

18.5Structural induction#

Structural induction applies to recursively defined objects.

For a recursively defined collection:

1. prove the property for every base object;
2. assume it holds for component objects;
3. prove each construction rule preserves it.

Applications include:

• strings;
• logical formulas;
• rooted trees;
• recursively generated expressions.

It is induction over construction depth rather than an explicit integer variable.

19Counting#

Counting problems are largely classification problems.

Before choosing a formula, determine:

1. Is this a sequence of decisions or a choice among cases?
2. Does order matter?
3. Is repetition permitted?
4. Are objects distinguishable?
5. Are there restrictions?
6. Can the complement be counted more easily?

19.1Sum rule#

If one outcome can occur through disjoint cases containing $m$ and $n$ possibilities:

$$m+n$$

The cases must not overlap. If they do, use inclusion-exclusion.

Recognition cue: OR between mutually exclusive alternatives.

19.2Product rule#

If a process has successive stages with:

$$n_1,n_2,\ldots,n_k$$

choices, then the total is:

$$n_1n_2\cdots n_k$$

Multiply when each partial choice has the stated number of extensions. The available objects may change: choosing two distinct people for ordered roles gives n(n−1). If branches have different numbers of extensions, count each branch and add. The word “and” alone does not justify multiplication.

19.3Bijection principle#

If a difficult collection can be paired one-to-one with an easier collection, they have the same size.

Much of combinatorics works by representing an object as:

• a bit string;
• a subset;
• a permutation;
• a tuple;
• a path.

19.4Division rule#

If a counting procedure counts every actual object exactly $d$ times, divide by $d$.

This is why combinations divide permutations by $r!$: every unordered $r$-element choice appears in $r!$ orders.

19.5Permutations#

When selecting and ordering $r$ distinct objects from $n$:

$$P(n,r) = n(n-1)\cdots(n-r+1) = \frac{n!}{(n-r)!}$$

For all $n$ objects:

$$n!$$

Order matters.

Examples of ordered roles:

• president, vice president, secretary;
• first, second, third place;
• password positions.

19.6Combinations#

When selecting $r$ objects from $n$ and order does not matter:

$$\binom nr = \frac{n!}{r!(n-r)!}$$

Symmetry:

$$\binom nr=\binom n{n-r}$$

Choosing which $r$ items are included is equivalent to choosing which $n-r$ are excluded.

19.7The four basic selection models#

This table should be automatic.

Count the same people in two ways. From five people, choose a chair and a secretary: 5×4 = 20 assignments because swapping the roles changes the result. Choose a two-person committee instead: divide by 2! to get 10, since each pair was counted in both orders. The formula follows from what counts as a different outcome.

19.8Repeated identical objects#

For $n$ total objects where identical categories occur $n_1,\ldots,n_k$ times:

$$\frac{n!}{n_1!n_2!\cdots n_k!}$$

The denominator removes rearrangements among indistinguishable copies.

This is a multinomial coefficient.

More generally:

$$\binom{n}{n_1,n_2,\ldots,n_k} = \frac{n!}{n_1!\cdots n_k!}$$

where:

$$n_1+\cdots+n_k=n$$

19.9Circular permutations#

When only rotation is considered equivalent, arrange $n$ distinct objects around a circle:

$$(n-1)!$$

Fix one object to remove rotational duplication.

If reflections are also considered identical, division by $2$ may be appropriate, provided the arrangements do not have special symmetry.

19.10Complementary counting#

Use:

$$|\text{desired}| = |\text{total}|-|\text{undesired}|$$

This is particularly effective for phrases such as:

• at least one;
• none;
• not all;
• contains a repeated item;
• violates no restriction.

“At least one success” is usually easier as:

$$1-P(\text{zero successes})$$

The same structural idea works in counting and probability.

19.11Stars and bars#

Count nonnegative integer solutions to:

$$x_1+x_2+\cdots+x_k=n$$

Imagine $n$ identical stars separated into $k$ categories by $k-1$ bars.

There are:

$$n+k-1$$

positions containing $n$ stars and $k-1$ bars, so:

$$\binom{n+k-1}{k-1}$$

solutions.

Equivalent form:

$$\binom{n+k-1}{n}$$

For positive solutions:

$$x_i\ge1$$

give one item to every category first. The remaining total is $n-k$, giving:

$$\binom{n-1}{k-1}$$

For lower bounds:

$$x_i\ge L_i$$

substitute:

$$y_i=x_i-L_i$$

so $y_i\ge0$.

Upper bounds generally require inclusion-exclusion.

19.12Multisets#

Choosing $r$ items from $n$ categories with repetition allowed and order irrelevant is equivalent to:

$$x_1+\cdots+x_n=r$$

with $x_i\ge0$.

Thus:

$$\binom{n+r-1}{r}$$

This is combinations with repetition.

19.13Pigeonhole principle#

If $N$ objects are placed into $k$ boxes, at least one box contains at least:

$$\left\lceil\frac Nk\right\rceil$$

objects.

To guarantee at least $r$ objects in one box, the maximum avoiding that outcome is:

$$k(r-1)$$

so the guarantee threshold is:

$$k(r-1)+1$$

The principle proves existence without identifying which box has the concentration.

19.14Inclusion-exclusion#

For two properties:

$$|A\cup B|=|A|+|B|-|A\cap B|$$

For three:

$$\begin{aligned} |A\cup B\cup C| ={}&|A|+|B|+|C|\\ &-|A\cap B|-|A\cap C|-|B\cap C|\\ &+|A\cap B\cap C| \end{aligned}$$

To count objects having none of several properties:

$$|\text{none}| = |\text{universe}|-|\text{at least one}|$$

and evaluate “at least one” using inclusion-exclusion.

19.15Binomial theorem#

$$(x+y)^n = \sum_{k=0}^{n} \binom nkx^{n-k}y^k$$

The coefficient of:

$$x^{n-k}y^k$$

is:

$$\binom nk$$

Pascal’s identity:

$$\binom nk = \binom{n-1}{k-1} + \binom{n-1}{k}$$

Interpretation: count $k$-element subsets according to whether they contain one designated element.

19.16Bit strings#

Number of bit strings of length $n$:

$$2^n$$

Number containing exactly $k$ ones:

$$\binom nk$$

Number containing at most $k$ ones:

$$\sum_{i=0}^{k}\binom ni$$

Number containing at least one $1$:

$$2^n-1$$

Number with no consecutive ones is often modeled recursively: strings ending in $0$ may follow any valid shorter string, while strings ending in $1$ must have a preceding $0$.

19.17Block method#

When designated objects must remain together, treat them as one block.

If $A,B,C$ must appear consecutively among other distinct objects:

1. treat $(ABC)$ as one object;
2. arrange that block with the others;
3. multiply by the internal arrangements of $A,B,C$.

If the internal order is fixed, omit the internal factorial.

19.18Gap method#

To prevent certain objects from being adjacent:

1. arrange the unrestricted objects;
2. identify gaps around and between them;
3. place restricted objects into distinct gaps.

This is useful for “no two identical/category-$X$ objects adjacent.”

19.19Lexicographic order#

Lexicographic order compares sequences at their first differing position.

Next permutation#

Given an ordering:

1. find the largest $i$ such that:
   $$a_i<a_{i+1}$$
2. find the largest $j>i$ such that:
   $$a_i<a_j$$
3. swap $a_i,a_j$;
4. reverse the suffix after position $i$.

If no such $i$ exists, the permutation is the last one.

Next $r$-combination#

For:

$$a_1<a_2<\cdots<a_r$$

from $\{1,\ldots,n\}$:

1. find the rightmost $a_i$ with:
   $$a_i<n-r+i$$
2. increment $a_i$;
3. reset every later entry to the smallest consecutive values.

20Discrete probability#

Probability assigns numerical weights to outcomes.

• Experiment: process producing an outcome.
• Outcome: one possible result.
• Sample space $S$: set of all outcomes.
• Event: subset of $S$.

For equally likely finite outcomes:

$$P(E)=\frac{|E|}{|S|}$$

This ratio is invalid when outcomes are not equally likely.

Counting from the previous chapter supplies the numerator and denominator only when the individual outcomes are equally likely. For two fair dice, the 36 ordered pairs are equally likely; the possible sums 2 through 12 are not. Always identify the elementary outcomes before dividing counts.

20.1Probability axioms#

$$P(E)\ge0$$

$$P(S)=1$$

For pairwise disjoint events:

$$P\left(\bigcup_iE_i\right)=\sum_iP(E_i)$$

Consequences:

$$P(\varnothing)=0$$

$$0\le P(E)\le1$$

20.2Complement and unions#

Complement:

$$P(E^c)=1-P(E)$$

General addition rule:

$$P(A\cup B) = P(A)+P(B)-P(A\cap B)$$

If $A,B$ are mutually exclusive:

$$P(A\cap B)=0$$

and:

$$P(A\cup B)=P(A)+P(B)$$

20.3Conditional probability#

Suppose 30 of 100 devices are laptops, and 12 of those laptops are patched. Given that a chosen device is a laptop, only those 30 devices remain possible: the patched fraction is 12/30. The original population size 100 is no longer the denominator.

$$P(A\mid B) = \frac{P(A\cap B)}{P(B)}$$

provided $P(B)>0$.

The condition $B$ becomes the new restricted sample space.

Rearranging gives the multiplication rule:

$$P(A\cap B)=P(A\mid B)P(B)$$

Also:

$$P(A\cap B)=P(B\mid A)P(A)$$

For a sequence of events:

$$P(A_1\cap\cdots\cap A_n) = P(A_1) P(A_2\mid A_1) \cdots P(A_n\mid A_1\cap\cdots\cap A_{n-1})$$

On a probability tree:

• multiply along a path;
• add probabilities of disjoint successful paths.

20.4Independence#

Events $A,B$ are independent when:

$$P(A\cap B)=P(A)P(B)$$

Equivalent, when denominators are nonzero:

$$P(A\mid B)=P(A)$$

and:

$$P(B\mid A)=P(B)$$

Learning one event does not change the probability of the other.

Independence versus mutual exclusivity#

Mutually exclusive means:

$$A\cap B=\varnothing$$

If both events have positive probability, mutually exclusive events cannot be independent:

$$P(A\cap B)=0$$

but:

$$P(A)P(B)>0$$

Sampling with replacement is often independent. Sampling without replacement is usually dependent because probabilities change.

20.5Law of total probability#

If:

$$A_1,\ldots,A_k$$

partition the sample space, then:

$$P(B) = \sum_{i=1}^{k}P(B\mid A_i)P(A_i)$$

This adds every mutually exclusive route by which $B$ can occur.

20.6Bayes’ theorem#

$$P(A_i\mid B) = \frac{P(B\mid A_i)P(A_i)} {\sum_jP(B\mid A_j)P(A_j)}$$

Conceptually:

$$\text{posterior} = \frac{\text{likelihood}\times\text{prior}} {\text{total probability of evidence}}$$

Bayes reverses a conditional.

Given:

• base rates $P(A_i)$;
• likelihoods $P(B\mid A_i)$;
• observed evidence $B$;

it finds:

$$P(A_i\mid B)$$

The denominator is calculated using total probability.

Base rates matter. A highly accurate test can still produce many false positives when the condition tested is rare.

Bayes as a count. Suppose 10 of 1,000 devices are infected. A scanner flags 9 of those 10 and also flags 99 of the 990 clean devices. Among the 108 flagged devices, only 9 are infected, so P(infected | flagged) = 9/108 ≈ 8.3%. This is not the 90% detection rate: that rate answers P(flagged | infected), the reverse question. The total probability denominator counts all ways to get flagged.

21Random variables#

Toss two fair coins. The outcomes HH, HT, TH, TT each have probability 1/4. Let X count heads: its values are 2, 1, 1, 0. Thus P(X=1) = 1/2 because two different outcomes give that value. A random variable converts outcomes to numbers; its distribution groups and adds their probabilities.

A random variable is a function:

$$X:S\to\mathbb R$$

assigning a number to every outcome.

For a discrete random variable, the probability mass function is:

$$p_X(x)=P(X=x)$$

Requirements:

$$p_X(x)\ge0$$

and:

$$\sum_xp_X(x)=1$$

21.1Expected value#

$$E[X] = \sum_xxP(X=x)$$

Expected value is a long-run probability-weighted average. It need not be a possible individual outcome.

For transformed values:

$$E[g(X)] = \sum_xg(x)P(X=x)$$

Linearity:

$$E[aX+b]=aE[X]+b$$

$$E[X+Y]=E[X]+E[Y]$$

The second identity does not require independence.

For games, use net payoff:

$$\text{prize}-\text{cost}$$

and represent losses as negative.

A fair game has expected net value $0$.

21.2Variance and standard deviation#

Variance measures squared deviation from the mean:

$$\operatorname{Var}(X) = E[(X-E[X])^2]$$

Computational form:

$$\operatorname{Var}(X) = E[X^2]-(E[X])^2$$

Standard deviation:

$$\sigma_X=\sqrt{\operatorname{Var}(X)}$$

Scaling properties:

$$\operatorname{Var}(aX+b)=a^2\operatorname{Var}(X)$$

Adding a constant changes the mean but not variance.

For independent $X,Y$:

$$\operatorname{Var}(X+Y) = \operatorname{Var}(X)+\operatorname{Var}(Y)$$

22Common discrete distributions#

A distribution is a reusable model for a random variable. Choose it from the experiment, not from the formula’s appearance: one success/failure trial, a fixed number of trials, trials until the first success, or a sample without replacement.

22.1Bernoulli#

A Bernoulli random variable represents one trial:

$$X= \begin{cases} 1,&\text{success}\\ 0,&\text{failure} \end{cases}$$

with:

$$P(X=1)=p$$

Mean:

$$E[X]=p$$

Variance:

$$p(1-p)$$

22.2Binomial#

A binomial model requires:

1. a fixed number $n$ of trials;
2. two outcomes per trial;
3. independent trials;
4. constant success probability $p$.

If $X$ counts successes:

$$P(X=k) = \binom nkp^k(1-p)^{n-k}$$

Mean:

$$E[X]=np$$

Variance:

$$\operatorname{Var}(X)=np(1-p)$$

“At least one” is:

$$P(X\ge1) = 1-P(X=0) = 1-(1-p)^n$$

22.3Geometric#

If $X$ counts the number of trials until the first success:

$$P(X=k) = (1-p)^{k-1}p$$

for $k=1,2,\ldots$.

Mean:

$$E[X]=\frac1p$$

The geometric distribution is memoryless:

$$P(X>s+t\mid X>s)=P(X>t)$$

Previous failures do not alter the success probability of future independent trials.

22.4Hypergeometric#

Use the hypergeometric distribution for sampling without replacement.

Suppose:

• population size $N$;
• $K$ successful objects;
• sample size $n$;
• $X$ counts sampled successes.

Then:

$$P(X=k) = \frac{\binom Kk\binom{N-K}{n-k}} {\binom Nn}$$

This differs from binomial because the trials are dependent and the success probability changes.

23Models of computation#

23.1Alphabets, strings, and languages#

24Deterministic finite automata#

A DFA answers one question: is this string in the language? Imagine a light that remembers whether the number of 1s read so far is even or odd. Two states suffice because the exact count is irrelevant; only its parity affects the answer.

A DFA is a five-tuple:

$$(Q,\Sigma,\delta,q_0,F)$$

where:

• $Q$: finite state set;
• $\Sigma$: input alphabet;
• $\delta:Q\times\Sigma\to Q$: transition function;
• $q_0$: start state;
• $F\subseteq Q$: accepting states.

For every state and every input symbol, a DFA has exactly one next state.

24.1State diagrams#

• circles represent states;
• a free incoming arrow marks the start state;
• double circles mark accepting states;
• labeled arrows represent transitions.

The state is the machine’s entire memory.

24.2Tracing a DFA#

1. begin at $q_0$;
2. read the input from left to right;
3. follow one transition per symbol;
4. after consuming the entire string, inspect the final state;
5. accept exactly when that state lies in $F$.

Entering an accepting state before the input ends does not guarantee acceptance. Remaining symbols may lead elsewhere.

The empty string is accepted precisely when:

$$q_0\in F$$

because no transitions are taken.

For the even-number-of-1s machine, start in the accepting even state. Reading 101 gives even → odd → odd → even, so it accepts. Reading 10 ends odd and rejects. The labels describe what the machine remembers, not which character it just read.

24.3Transition tables#

A table gives the same function as a state diagram:

Every row-symbol combination must contain exactly one destination in a DFA.

24.4State design patterns#

A state should represent exactly the finite information needed about the input read so far.

Even or odd parity#

Use two states:

• even number of $1$s seen;
• odd number of $1$s seen.

Reading $1$ switches states. Reading $0$ leaves the state unchanged.

Count modulo $k$#

Use $k$ states representing residues:

$$0,1,\ldots,k-1$$

Each new symbol updates the residue.

Has a symbol appeared?#

Use:

• “not yet seen”;
• “already seen.”

Ends with a pattern#

States represent the longest suffix of the input seen so far that is also a prefix of the target pattern.

Trap state#

A trap or sink state is a nonaccepting state whose transitions all return to itself.

Once entered, acceptance is impossible.

24.5Binary numbers modulo $k$#

If a state represents the remainder $r$ of a binary prefix modulo $k$, appending bit $b$ produces:

$$r_{\text{new}} \equiv 2r+b \pmod k$$

This is a common way to construct automata recognizing binary numbers divisible by $k$.

25Nondeterministic finite automata#

Keep the same string and acceptance question, but allow several possible next states. Track all possibilities as a set, not as a guess about which branch is right. Subset construction turns that set into one ordinary DFA state.

An NFA may have:

• no transition for a state-symbol pair;
• one transition;
• several transitions;
• $\varepsilon$-transitions that consume no input.

Its transition function returns a set of states:

$$\delta:Q\times\Sigma\to\mathcal P(Q)$$

or, with epsilon transitions:

$$\delta:Q\times(\Sigma\cup\{\varepsilon\})\to\mathcal P(Q)$$

An NFA accepts a string when at least one possible path:

1. consumes the full input;
2. ends in an accepting state.

One failed branch does not cause rejection if another branch succeeds.

25.1Epsilon closure#

The $\varepsilon$-closure of a state set is every state reachable using zero or more $\varepsilon$-transitions.

When tracing an NFA with epsilon transitions:

1. take the epsilon closure;
2. consume the next symbol from every current state;
3. take epsilon closure again;
4. repeat.