Foundations Of Statistics Part 0 — Set Theory

6 min readApr 28, 2023

Introduction

This is the first of a series of articles covering foundational mathematics I think you should know about statistics to understand the mathematics behind data science and machine learning.

This will be a useful summary and reference guide for data scientists, and those who need a refresher on things.

At the bare minimum you should be comfortable with the concepts taught in a two-part precalculus course. These ideas are often taught in discrete mathematics, but anyone who has taken introductory differential and integral calculus should be completely comfortable with the concepts and notation used.

This article in particular will be fairly simple, but later articles will dive into concepts that require calculus.

Definition

A set is a collection of distinct objects.

The objects within a set are the elements or members of the set.

Every element within a set is unique, and sets are used to group these unique objects together.

If a set has a finite number of elements then it is finite. Otherwise it is an infinite set.

Including, Excluding

There should be no ambiguity as to whether an element belongs to a set or not. It is binary.

If x is an element of set A, then we say “x belongs to A” and write:

If x does not belong to A we write:

Cardinality

The cardinality of a set is the number of elements that belong to that set.

A set with cardinality of 1 is called a singleton set.

Set builder notation

If you have taken precalculus then this should be familiar. Set builder notation is used to define a set by stating clearly the requirements to be an element of the set.

Syntax:

{expression : rule}

Where : or | means “such that”.

Subsets

A set A is a subset set B, if every element of A is an element of B.

And we write:

Note that:

A subset of a subset is a subset.
And By the above definition, every set is a subset of itself. This means that A and B can both be subsets of each other if they both have the same elements.

That’s not terribly useful. How do we find sets that aren’t just copies of each other? We look to proper subsets.

Proper Subsets

Set A is a proper subset of B if

A is a subset of B
There is at least one element in B that is not in A

A simple example from calculus: The set of all integers Z is a proper subset of the set of all real numbers R.

Relative Complement

If A is a subset of B the Relative Complement of A in B is the set of elements in B that are not elements of A.

The relative complement of A in B is denoted by B — A and it can be described by its membership criterion as:

Disjoint & Equal Sets sets:

Two sets are Disjoint if they have no elements in common.

Two sets are equal if they have the same elements.

For equal sets we write:

Two sets are equal if and only if they are subsets of each other.

The Empty Set

A set is empty if it has no elements.

It has cardinality 0

There is only one empty set: the unique empty set, or null set.

The Universal Set

The universal set is a concept that is absolutely critical for the understanding of probability, which will show up in the next article.

For a sneak peek; the universal set is used to represent the population, and samples are subsets of that universal set.

If a set or a group of sets are all subsets of a set U, then U is called the Universal Set.

If a set contains all the elements of a group of sets, it is a universal set to those sets.

Absolute Complement

Once the universal set of U is fixed the relative complement of a subset A in U is called the absolute complement of A

denoted by:

The absolute complement of the absolute complement of any set A is the set A itself

Intersection

The set consisting of all elements common to both A and B is called the intersection of A and B

Denoted by:

The intersection of any set and the empty set is the empty set

The intersection of any set A and A (the intersection of a set with itself) is A.

The intersection of a set and its Absolute Complement is empty since no element can be simultaneously in A and not in A.

Union

The set consisting of all elements that belong to either A or to B or to both A and B is called the Union of A and B

denoted:

The union of a set A and the empty set is the set A

The union of A and A is also A (the union of a set with itself is itself).

Commutation And Associativity Of Set Intersection And Union

It follows from the definition that set intersection has the Commutative Property

the intersection A and B is equal to the intersection of B and A

Set union is also commutative

Distributive Laws & DeMorgan’s Laws:

Two theorems related to set operations involving intersections, unions, and taking absolute complements. These require a good bit of explaining to prove analytically, but are easily demonstrated with a Venn diagram.

Distributive Laws