Computer Engineering is coming soon
We're still writing and reviewing the notes for every subject, and we'll open it when every subject is ready. Meanwhile you can practise aptitude, ask Aria about any subject, or see the branches that are ready.
Computer Engineering
Machine Learning
Regression, classification, clustering, ensembles, reinforcement learning, neural networks.
Your progress in this subject0 of 25 · 0%
- 1Introduction to machine learningIntroduction to machine learning: formal problem statement, losses and risk, empirical risk minimisation, types of learning, basic assumptions and how an engineer chooses and evaluates models.
- 2Types of learningOverview of the main types of machine learning (supervised, unsupervised, semi‑supervised, reinforcement, self‑supervised, transfer, online/batch, active and federated), with formal definitions, core formulas (empirical/expected risk), exam-style derivations, diagrams to draw, worked model-answer ex
- 3Models of machine learningOverview of common families of machine-learning models, how they represent functions, their assumptions, and how to choose and evaluate them.
- 4Feature engineeringFeature engineering means converting raw data into informative features (numeric or categorical) that help machine-learning models learn better; it covers scaling, encoding, missing-value handling, transformations, interaction and polynomial features, binning, and leakage avoidance.
- 5PCA and LDAPrincipal Component Analysis (PCA) finds directions of maximal variance for dimensionality reduction; Linear Discriminant Analysis (LDA) finds directions that best separate labelled classes by maximising between-class to within-class scatter.
- 6Regression basics and the bias–variance trade-offBasics of regression (linear least squares) and the bias–variance trade-off: definitions, OLS derivation, assumptions, error decomposition, diagrams to draw, worked numerical examples, common mistakes and exam tips.
- 7Linear and polynomial regressionLinear regression fits a line (or a polynomial curve) to data by minimising squared errors; this note gives definitions, derivations (normal equations, Gauss–Markov), formulas, worked numerical examples and exam guidance.
- 8Tree-based and support vector regressionClear, exam-oriented notes on decision-tree (CART) regression and support vector regression (SVR): how they work, key formulas, derivations, worked examples, common mistakes and exam tips.
- 9Regularisation: ridge and lassoRidge and lasso are L2 and L1 regularisation methods for linear models that reduce overfitting by penalising large coefficients; ridge shrinks coefficients continuously, lasso can set some to zero and perform feature selection.
- 10Regression evaluation metricsClear, exam-focused notes on common regression evaluation metrics (residuals, MSE, RMSE, MAE, MAPE, R², adjusted R², explained variance), when to use each, derivations, diagrams to draw, and worked numerical examples with units.
- 11Classification basics and imbalanced dataBasics of binary classification, the Bayes optimal rule, confusion-matrix metrics, problems caused by imbalanced classes, and practical fixes (sampling, class weights, thresholding, cost-sensitive learning, SMOTE) with exam-style derivations and worked examples.
- 12Confusion matrix, precision, recall and F1Confusion matrix and the measures precision, recall (sensitivity) and F1 score: how to compute them, why they matter, worked numerical examples, common exam mistakes and practice questions.
- 13Multiclass classificationMulticlass classification: predicting one label out of K (>2) classes, key methods (one-vs-rest, one-vs-one, softmax/multinomial logistic), evaluation (multiclass confusion matrix, per-class precision/recall, macro/micro averaging), and gradient derivation for softmax cross-entropy.
- 14K-nearest neighboursK-nearest neighbours (KNN) is a simple, instance-based supervised learning algorithm used for classification and regression that predicts a query point's label from the labels of its nearest training examples.
- 15Support vector machines and kernelsConcise study notes on Support Vector Machines (SVM) and kernels: definitions, primal and dual formulations, margin geometry, kernel trick, Mercer's condition, worked derivations and exam-style examples.
- 16Clustering and hierarchical clusteringPractical, exam-ready notes on clustering with emphasis on hierarchical (agglomerative and divisive) methods, linkage rules, dendrograms, and worked examples.
- 17K-means and K-medoidsK-means partitions data by minimising within-cluster squared Euclidean distance using centroids; K-medoids is a robust variant that uses actual data points (medoids) and minimises sum of pairwise distances.
- 18DBSCAN and Gaussian mixture modelsStudy notes comparing DBSCAN (a density-based clustering algorithm) and Gaussian Mixture Models (GMMs) with the EM algorithm, covering definitions, derivations, formulas, examples, common mistakes and exam guidance.
- 19Ensemble learning and votingEnsemble learning combines many base learners by voting (majority, weighted or other rules) to reduce error and variance; key results include the binomial formula for majority error and the log-odds optimal weights under independent classifiers.
- 20Bagging and random forestsBagging (bootstrap aggregating) and Random Forests are ensemble methods that reduce model variance by training many decision trees on bootstrap samples and aggregating their predictions; random forests further decorrelate trees by selecting random subsets of features at each split.
- 21Boosting and stackingBoosting builds a strong predictor by adding many weak learners in a stagewise way (AdaBoost, gradient boosting); stacking combines different trained models by learning a meta-learner on their predictions.
- 22Reinforcement learning basicsBasic reinforcement learning: Markov decision processes, policies, returns, value and action-value functions, Bellman equations, solution methods (value/policy iteration, Q-learning), and exploration–exploitation.
- 23Markov decision processesMarkov decision processes (MDPs) formalise sequential decision making under uncertainty using states, actions, transition probabilities and rewards; they underpin dynamic programming and reinforcement learning methods such as value iteration and policy iteration.
- 24Value functions and the Bellman equationValue functions summarise expected returns in an MDP; the Bellman equations relate a state's value to immediate reward plus discounted value of successor states and are the basis for evaluation and control algorithms (policy evaluation, policy iteration, value iteration).
- 25Exploration, exploitation and Q-learningNotes on exploration vs exploitation in reinforcement learning, multi-armed bandits, common exploration strategies (ε-greedy, softmax, UCB), and the Q-learning algorithm with its update, assumptions and a one-step program example.
Coming next in Machine Learning
- Kernel PCA and local binary patterns· notes coming
- Feature selection: sequential forward and backward· notes coming
- Multidimensional scaling and matrix factorisation· notes coming
- Gradient descent· notes coming
- Cross-validation and micro and macro averaging· notes coming
- Spectral clustering· notes coming
- Outlier detection: isolation forest and LOF· notes coming
- Evaluating clusters: elbow, extrinsic and intrinsic measures· notes coming
- Perceptron and multilayer perceptron· notes coming
- Backpropagation learning· notes coming
- Functional link and radial basis function networks· notes coming
- Activation functions· notes coming
- Introduction to CNNs and RNNs· notes coming
- Logistic regression· notes coming
- Decision trees and CART· notes coming
- ROC curves and the kappa statistic· notes coming
- XGBoost· notes coming
- SVM as constrained optimisation· notes coming
- Clustering with minimal spanning trees· notes coming
- Expectation maximisation· notes coming
- Singular value decomposition· notes coming
- Hypothesis space and inductive bias· notes coming
- Collaborative filtering for recommendation· notes coming
- Bayesian learning and naive Bayes· notes coming
- Introduction to deep neural networks· notes coming
- PAC learning and sample complexity· notes coming
- VC dimension· notes coming
- Recent trends: self-supervised and federated learning, XAI and AutoML· notes coming
- Factor analysis, Isomap and locally linear embedding· notes coming
- Bayesian decision theory: losses, risks and discriminant functions· notes coming