Calculus and Linear Algebra

Calculus and linear algebra lay the foundation for many advanced math topics used in quantitative finance. So be prepared to answer some calculus or linear algebra problems—many of them may be incorporated into more complex problems—in quantitative interviews. Since most of the tested calculus and linear algebra knowledge is easy to grasp, the marginal benefit far outweighs the time you spend brushing up your knowledge on key subjects. If your memory of calculus or linear algebra is a little rusty, spend some time reviewing your college textbooks!

Needless to say, it is extremely difficult to condense any calculus/linear algebra books into one chapter. Neither is it my intention to do so. This chapter focuses only on some of the core concepts of calculus/linear algebra that are frequently occurring in quantitative interviews. And unless necessary, it does so without covering the proof, details or even caveats of these concepts. If you are not familiar with any of the concepts, please refer to your favorite calculus/linear algebra books for details.

3.1 Limits and Derivatives

Basics of derivatives

Let’s begin with some basic definitions and equations used in limits and derivatives. Although the notations may be different, you can find these materials in any calculus textbook.

Derivative: Let $y=f(x)$, then $f'(x)=\dfrac{dy}{dx}=\lim_{\Delta x\to0}\dfrac{\Delta y}{\Delta x}=\lim_{\Delta x\to0}\dfrac{f(x+\Delta x)-f(x)}{\Delta x}$

The product rule: If $u=u(x)$ and $v=v(x)$ and their respective derivatives exist,

The quotient rule: $\dfrac{d}{dx}\left(\dfrac{u}{v}\right)=\left(v\dfrac{du}{dx}-u\dfrac{dv}{dx}\right)/v^2,\quad \left(\dfrac{u}{v}\right)'=\dfrac{u'v-uv'}{v^2}$

The chain rule: If $y=f(u(x))$ and $u=u(x)$, then $\dfrac{dy}{dx}=\dfrac{dy}{du}\dfrac{du}{dx}$

The generalized power rule: $\dfrac{dy^n}{dx}=ny^{n-1}\dfrac{dy}{dx}$ for $\forall n\ne0$

Some useful equations:

What is the derivative of $y=\ln x^{\ln x}$?¹

Solution: This is a good problem to test your knowledge of basic derivative formulas—specifically, the chain rule and the product rule.

Let $u=\ln y=\ln\left(\ln x^{\ln x}\right)=\ln x\times\ln(\ln x)$. Applying the chain rule and the product rule, we have

To derive $\dfrac{d(\ln(\ln x))}{dx}$, we again use the chain rule by setting $v=\ln x$:

Maximum and minimum

Derivative $f'(x)$ is essentially the slope of the tangent line to the curve $y=f(x)$ and the instantaneous rate of change (velocity) of $y$ with respect to $x$. At point $x=c$, if 35$f'(c)>0$, $f(x)$ is an increasing function at $c$; if $f'(c)<0$, $f(x)$ is a decreasing function at $c$.

Local maximum or minimum: suppose that $f(x)$ is differentiable at $c$ and is defined on an open interval containing $c$. If $f(c)$ is either a local maximum value or a local minimum value of $f(x)$, then $f'(c)=0$.

Second Derivative test: Suppose the secondary derivative of $f(x)$, $f''(x)$, is continuous near $c$. If $f'(c)=0$ and $f''(c)>0$, then $f(x)$ has a local minimum at $c$; if $f'(c)=0$ and $f''(c)<0$, then $f(x)$ has a local maximum at $c$.

Without calculating the numerical results, can you tell me which number is larger, $e^\pi$ or $\pi^e$?²

Solution: Let’s take natural logs of $e^\pi$ and $\pi^e$. On the left side we have $\pi\ln e$, on the right side we have $e\ln\pi$. If $e^\pi>\pi^e$, $e^\pi>\pi^e\iff\pi\times\ln e>e\times\ln\pi\iff\dfrac{\ln e}{e}>\dfrac{\ln\pi}{\pi}$.

Is it true? That depends on whether $f(x)=\dfrac{\ln x}{x}$ is an increasing or decreasing function from $e$ to $\pi$. Taking the derivative of $f(x)$, we have $f'(x)=\dfrac{1/x\times x-\ln x}{x^2}=\dfrac{1-\ln x}{x^2}$, which is less than 0 when $x>e$ ($\ln x>1$). In fact, $f(x)$ has global maximum when $x=e$ for all $x>0$. So $\dfrac{\ln e}{e}>\dfrac{\ln\pi}{\pi}$ and $e^\pi>\pi^e$.

Alternative approach: If you are familiar with the Taylor’s series, which we will discuss in Section 3.4, you can apply Taylor’s series to $e^x$: $e^x=\displaystyle\sum_{n=0}^{\infty}\dfrac{1}{n!}=1+\dfrac{x}{1!}+\dfrac{x^2}{2!}+\dfrac{x^3}{3!}+\cdots$. So $e^x>1+x,\ \forall x>0$. Let $x=\pi/e-1$, then $e^{\pi/e}/e>\pi/e\iff e^{\pi/e}>\pi\iff e^\pi>\pi^e$.

L’Hospital’s rule

Suppose that functions $f(x)$ and $g(x)$ are differentiable at $x\to a$ and that $\lim_{x\to a}g'(a)\ne0$. Further suppose that $\lim_{x\to a}f(a)=0$ and $\lim_{x\to a}g(a)=0$ or that $\lim_{x\to a}f(a)\to\pm\infty$ and 36$\lim_{x\to a}g(a)\to\pm\infty$, then $\lim_{x\to a}\dfrac{f(x)}{g(x)}=\lim_{x\to a}\dfrac{f'(x)}{g'(x)}$. L’Hospital’s rule converts the limit from an indeterminate form to a determinate form.

What is the limit of $e^x/x^2$ as $x\to\infty$, and what is the limit of $x^2\ln x$ as $x\to0^+$?

Solution: $\lim_{x\to\infty}\dfrac{e^x}{x^2}$ is a typical example of L’Hospital’s rule since $\lim_{x\to\infty}e^x=\infty$ and $\lim_{x\to\infty}x^2=\infty$. Applying L’Hospital’s rule, we have

The result still has the property that $\lim_{x\to\infty}f(x)=\lim_{x\to\infty}e^x=\infty$ and $\lim_{x\to\infty}g(x)=\lim_{x\to\infty}2x=\infty$, so we can apply the L’ Hospital’s rule again:

At first look, L’Hospital’s rule does not appear to be applicable to $\lim_{x\to0^+}x^2\ln x$ since it’s not in the format of $\lim_{x\to a}\dfrac{f(x)}{g(x)}$. However, we can rewrite the original limit as $\lim_{x\to0^+}\dfrac{\ln x}{x^{-2}}$ and it becomes obvious that $\lim_{x\to0^+}x^{-2}=\infty$ and $\lim_{x\to0^+}\ln x=-\infty$. So we can now apply L’Hospital’s rule: