MDA511 Mathematical and Statistical Methods - Matrix Calculus and Its Use in Optimisation

Assignment Description

1. Readings on Matrix Calculus and Assignment Report Requirements

In Assignment 1, you are asked to write a brief report on the relationship between Matrix Calculus (or Vector Calculus) and Optimization, and how it is applied to Machine Learning after you have read the following two references:

• Matrix Calculus: The Mathematics of ‘Learning’ https://impartiallyderivative.com/2018/02/09/matrix-calculus-the-mathematics-of- learning/
• Vector Calculus for Machine Learning https://medium.com/intuition/vector-calculus-for-machine-learning- 5d8842d8cd23

The two references are quite short. Together with the materials covered in the first 3 weeks of this unit, you should be able to understand most of the concepts explained in these references. (Note: you do not need to understand everything in the two references to appreciate what Matrix Calculus is about).

The reason for basing Assignment 1 on the topic of Matrix Calculus and Optimization is that this area of mathematicsis heavily used in Machine Learning and therefore in Data Science applications. The Gradient Descent algorithm, explained briefly in the second of the two references, is an important numerical tool used in optimizing a function or model based on Matrix Calculus.

Requirements of the Report:

• 750 words maximum, including any additional references besides the two
• (a) Discuss the main points in each
• (b) Explain how Matrix Calculus is used in
• (c) Explain the connection(s) between optimization and machine learning (including deep learning).

You can use Internet search to find resources to support your discussion and explanation. However,

you must cite any source according to the IEEE referencing style (https://library.mit.edu.au/itreferencing).