What motivates that strange definition of the determinant?

So historically, Leibniz first came up with the following definition of determinant while trying to derive a solvability condition for system of linear equations.
Afterwards, Laplace came up with the Laplace expansion formula.
Here, I am going to derive the Laplace expansion formula by solving a system of 3x3 equations "inducting" on the determinant definition for a 2x2 system.

We all know that determinant for a 2x2 matrix \begin{bmatrix} a & b \\ c & d \end{bmatrix} the determinant is $ad-bc$. This definition naturally arises when you try to row-reduce the system.
The inverse matrix (and hence solution to the associated system) is given by: $$ \frac{1}{ad-bc} \begin{bmatrix} d & -c \\ -b & a \end{bmatrix} $$ Let's try to find the definition of determinant for 3x3 matrix by row-reduction and re-using this result here.
So we start with: $$ M = \begin{bmatrix} a & b & c\\ d & e & f\\ g & h & i\\ \end{bmatrix} $$ We can apply row reduction. At each step, i am going to give the resulting matrix and also the "elementary matrix" which encodes the row reduction operation. Namely, I give you a matrix \(E\) such that when you perform \(EM\) you get the result of the row-reduction.

Multiply row 2 with \(a\) and write to row 2. First I give you the resulting matrix followed by the elementary matrix: $$ \begin{bmatrix} a & b & c\\ ad & ae & af\\ g & h & i\\ \end{bmatrix} \qquad \begin{bmatrix} 1 & 0 & 0\\ 0 & a & 0\\ 0 & 0 & 1\\ \end{bmatrix} $$

new row 3 = old row 3 * a. Remember that I am giving you the "cumulative" matrix and the "individual" elementary matrices. $$ \begin{bmatrix} a & b & c\\ ad & ae & af\\ ag & ah & ai\\ \end{bmatrix} \qquad \begin{bmatrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 0 & a\\ \end{bmatrix} $$

new row 3 = old row 3 - ((row1) * g) $$ \begin{bmatrix} a & b & c\\ ad & ae & af\\ 0 & ah-bg & ai-cg\\ \end{bmatrix} \qquad \begin{bmatrix} 1 & 0 & 0\\ 0 & 1 & 0\\ -g & 0 & 1\\ \end{bmatrix} $$

new row 2 = old row 2 - ((row1) * d) $$ \begin{bmatrix} a & b & c\\ 0 & ae-bd & af-cd\\ 0 & ah-bg & ai-cg\\ \end{bmatrix} \qquad \begin{bmatrix} 1 & 0 & 0\\ -d & 1 & 0\\ 0 & 0 & 1\\ \end{bmatrix} $$ Now you are seeing some "determinant-like" expressions. These are exactly the determinant of adjugate matrices. For example, I can define \(M_{33}\) as the determinant of the matrix obtained by removing row 3 and column 3 from the original matrix \(M\): \(M_{33} = ae - bd\).

Using this new "determinant of adjugate sub-matrices" definition, we have row-reduced our original matrix \(M\) to \begin{bmatrix} a & b & c\\ 0 & M_{33} & M_{32}\\ 0 & M_{23} & M_{22}\\ \end{bmatrix}

Our cumulative elementary operation matrix is \begin{bmatrix} 1 & 0 & 0\\ -d & a & 0\\ -g & 0 & a\\ \end{bmatrix}

So now, we can think of the the right bottom sub-matrix (call it \(S\)) and use the 2x2 inverse formula to get the following elementary operation matrix (bad name since it is not really elementary): \begin{bmatrix} 1 & 0 & 0\\ 0 & \frac{1}{\det(S)}M_{22} & \frac{-1}{\det(S)}M_{32}\\ 0 & \frac{-1}{\det(S)}M_{23} & \frac{1}{\det(S)}M_{33}\\ \end{bmatrix}

This will take the original matrix further to a simple \begin{bmatrix} a & b & c\\ 0 & 1 & 0\\ 0 & 0 & 1\\ \end{bmatrix}

at this point, if we "left-apply" the elementary matrix \begin{bmatrix} 1/a & -b/a & -c/a\\ 0 & 1 & 0\\ 0 & 0 & 1\\ \end{bmatrix} you will row-reduce \(M\) to identity.

but now we multiply all the elementary matrix to get the whole complete inverse. $$ \begin{bmatrix} 1/a & -b/a & -c/a\\ 0 & 1 & 0\\ 0 & 0 & 1\\ \end{bmatrix} \begin{bmatrix} 1 & 0 & 0\\ 0 & \frac{1}{\det(S)}M_{22} & \frac{-1}{\det(S)}M_{32}\\ 0 & \frac{-1}{\det(S)}M_{23} & \frac{1}{\det(S)}M_{33}\\ \end{bmatrix} \begin{bmatrix} 1 & 0 & 0\\ -d & a & 0\\ -g & 0 & a\\ \end{bmatrix} $$

Lots of really nice cancellations happen, for example \(gM_{32}-dM_{22} = -aM{12}\). Very soon, you should see something like this $$ \frac{a}{\det(S)}\begin{bmatrix} M_{11} & -M_{21} & M_{31}\\ -M_{12} & M_{22} & -M_{32}\\ M_{13} & -M_{23} & M_{33}\\ \end{bmatrix} $$ Simplify the scalar coefficient you get the 3x3 inverse and the 3x3 determinant formula in Laplace expansion form. This obviously can be generalized inductively to n-by-n.

Exact and closed differential forms

This post explains exact and closed differential forms. The most general definitions work with general differential forms on m-dimensional manifolds. But let us begin by looking at 1-forms (line integrals). Let's begin with a definition.
Definition: a line integral \(\int\omega\) over a domain \(D\) is exact if \(\int\omega\) is dependent only on its initial and end point (independent of the path taken).
Definition: a line integral \(\int\omega\) over a domain \(D\) is closed if \(\int\omega\) it integrates to zero over every closed loop.
Theorem: Assume that the vector field (1-form) has continuous partials and the domain is simply connected (the condition can be relaxed for some directions, but we assume so for simplicity.) The following are equivalent:

1. A line integral is exact.
2. The line integral is closed.
3. If the line integral's vector field has continuous partial derivatives, the 1-form (vector field) \(\omega\) is a gradient (i.e. has an anti-dervative)

The two non-trivial directions:
(1)=>(3): This follows because we can construct the anti-derivative (with some arbitrary starting point \(x_0\)) $$ F(\vec{x}) = \int_{\vec{x_0}}^{\vec{x}} \omega $$ The proof follows similar to the traditional fundamental theorem of calculus.
(2)=>(1): This follows from Green's theorem.

More:
One dimension
In one dimension fundamental theorem of calculus tells us that every continuous function is an exact integral.
Two dimensions In two dimensions, if a vector field is exact (i.e. is a gradient), then its mixed derivatives must be equal. i.e. if $$ \vec{F}(x,y) = \nabla U(x,y) $$ then, denoting the \(x\) and \(y\) components of \(F\) by \(P\) and \(Q\): $$ \frac{\partial P}{\partial y} = \frac{\partial Q} {\partial x} $$ This condition here is called closed.
If Green's theorem applies, then closed would imply exactness. Green's theorem in two dimensions require simple connectedness and the vector field being C1. If the domain is made small enough, then the domain can always be made simply connected. This means that closed really means locally exact. The intuition behind the simply connected requirement is exact because simply connectedness allows us to break the domain down to very small boxes where the vector field is locally exact.
When integrating a closed vector field around a very small circle, Green's theorem applies and all line integral over closed path gives zero. We call this vector field irrotational because it measures the infinitisimal tendency for the fluid to rotate if the vector field describes the fluid's velocity.
Of course there are vector fields that are not defined on simply connected regions that are still exact. The standard \(F(\vec{r}) = \frac{1}{|r|}\hat{r}\) fits this. The potential in two dimensions is \(\ln(|r|)\).
There are also vector fields that meets the closed condition who are not exact. Consider $$ \vec{F}(x,y) = \left(\frac{-y}{x^2 + y^2}, \frac{x}{x^2 + y^2} \right) $$
Three dimensions
The analogue of Green's theorem becomes Stoke's theorem in 3D. Here, Stoke's theorem applies on a closed curve as long as there exists an enclosed surface with no discontinuities in the C1 vector field. So in 3D there is much more leeway to applying the "closed=>exact" logic.
Of course this all make sense because simply connectedness in 3D is an easier condition to meet in 3D than 2D. One would need a line of discontinuity, for example we could simply extend the above vector field to $$ \vec{F}(x,y,z) = \left(\frac{-y}{x^2 + y^2}, \frac{x}{x^2 + y^2}, 0 \right) $$
Complex Analysis and Cauchy's Theorem
Normally when applying the two dimensional Green theorem to functions it typically requires that the partial derivatives are continuous. Cauchy's integral theorem says that holomorphic functions are exact without a priori requiring continuous partials.

Mobius strip parametrization

Recently I thought about a way to parametrize the Mobius strip embedded in 3D.

The intuition was the following:

If someone were to look "down" along the circle of the strip, they would see a circle rotating by 180 degrees.  A circle of this kind in 2D is very simply parameterized by:

$$
x_c(r,\psi) = r \cos(\psi) \\
y_c(r,\psi) = r \sin(\psi)
$$

Now we need to wrap this around a circle.  This would mean that the plane the half-circle sits in should be normal to the tangent vector of the circle in 3D.

I choose to wrap my Mobius strip around the axis z with overall radius $R$.  So I have the parameterization

$$
x = R \cos (\phi) \\
y = R \sin (\phi) \\
z = 0
$$

It's not hard to see that at every point on this circle, two basis vector for the plane normal to the tangent vector is given by:

$$
\hat e_1 = (\cos \phi, \sin \phi, 0) \\
\hat e_2 = (0, 0, 1)
$$

With offset
$$
(R \cos \phi, R \sin \phi)
$$

Together, using the equations for \((x_c, y_c)\), I get the overall parametrization:

$$
x_c(r,\phi)\hat e _1 (2\phi) + y_c(r,\phi) \hat e_2 (2\phi) + (R \cos \phi, R \sin \phi, 0)
$$

Using matplotlib, I have something like:

import numpy as np
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
from matplotlib import interactive
import math
import matplotlib
matplotlib.use('Qt5Agg')

R = 4

def mobius(r, phi):
    x_ = r * math.cos(phi)
    e1 = np.array([math.cos(2*phi), math.sin(2*phi), 0])
    y_ = r * math.sin(phi)
    e2 = np.array([0, 0, 1])
    return x_*e1 + y_*e2 + np.array([R*math.cos(2*phi), R*math.sin(2*phi),0])


# Generate torus mesh
phi = np.linspace(-0.5*np.pi, 0.5*np.pi, 50)
r = np.linspace(-1, 1, 50)
r, phi = np.meshgrid(r, phi)
X = np.ndarray(r.shape)
Y = np.ndarray(r.shape)
Z = np.ndarray(r.shape)
for (x,y) , r_scalar in np.ndenumerate(r):
    phi_scalar = phi[x,y]
    result = mobius(r_scalar, phi_scalar)
    X[x,y] = result[0];
    Y[x,y] = result[1];
    Z[x,y] = result[2];

# Display the mesh
# plt.switch_backend('Qt5Agg')
# %matplotlib qt5
fig = plt.figure()
ax = fig.gca(projection = '3d')
ax.set_xlim3d(-5, 5)
ax.set_ylim3d(-5, 5)
ax.set_zlim3d(-1, 1)
ax.plot_surface(X, Y, Z, color = 'w', rstride = 1, cstride = 1)
plt.show()

Output looks like:

A intuitive explanation for the chain rule

The chain allows us to take derivative of composition of two functions.  It has the form
$$(f\circ g)'(x)=f'(g(x))g'(x)$$

Intuitively, why should it be a product of the two function's individual derivatives?

We can answer that by looking at Taylor expansions.
If we wiggle \(x\) in \(g(x)\) we get
$$
g(x+\delta)\approx g(x)+g'(x)\delta
$$

That means if we wiggle \(x\) by \(\delta\), f gets wiggled by \(g'(x)\delta\).

A function \f(y)\ would change in a very similar way if we wiggle y:
$$
f(y+\delta') \approx f(y) + f'(y)\delta'
$$

We replace \(\delta'\) by \(g'(x)\delta\) because that is how much the "y" is perturbed if we perturb \(x\) by \(\delta\)

Overall, we would get
$$
f(g(x+\delta)) \approx f(g(x))+f'(g(x))g'(x)\delta
$$

Permutations and n-dimensional rotations

The question that motivates today's post is, how do we define rotations in n-dimensions?

Let's the standard Euclidean bases as \((e_1, e_2\ ...)\).

In two dimensions, a sequence of counter-clockwise rotations takes the standard basis thru the following sequence:

$$(e_1, e_2), (e_2, -e_1), (-e_1, e_2), (-e_2, e_1)$$

So each rotation switches the basis and negates the y-axis.

Similar situation happens in 3-dimensional rotation along the three axes of rotations.

This means it make sense to think of rotations (counter-clockwise) as operations that preserve an alternating tensor product.  WLOG, we can take that alternating product to be the determinant of all the basis vectors.

But rotation through the origin also has the property that it preserves distances.  Namely an isometry.

$$\| Rx \| = \|x\|$$

We can quickly derive the following properties:
If \(x \cdot y = 0 \Rightarrow Rx \cdot Ry = 0 \).
\( \| Re_i\| = 1 \Rightarrow [R^tR]_{(i,i)} = \pm1 \)

This means that \(R^tR\) must be a diagonal matrix with \(\pm1\) on the diagonal.  Hence its determinant is also \(\pm1\).  Restricting it to +1 would make the isometry a proper rotation.  If we allow -1, it is not hard to see that it would introduce reflections into R.

This all make sense.  If we "locally rotate" two axes in n-dimensions, which means fix all other n-2 axes, and only rotate the remaining 2 in the ordinary sense, we are actually preserving the sign of the determinant.  This means that our intuitive sense of 2-dimensional rotation has been successfully extended to n-dimensions.

Two failed modifications of Paxos: towards a better understanding of the protocol

Two failed attempts at modifying Paxos.

The Paxos algorithm solves the following problem:
There are n processes.  Each process starts with a random value of 0 or 1.
Processed communicate via messages.  Messages might be lost or arbitrarily delayed without bound.
At some time in the future, all processes need to decide on one of the two value.  There must be a way to find out when the decision has been made and what value it is.  Furthermore, to prevent trivial solutions, the decision outcome must be a subset of the starting values.  (I.e., if all processes start with 0, they cannot ever decide on 1).  Once a decision is reached, it must never change.

The following algorithm solves it assuming that a majority of servers can communicate with each other.  If they cannot, it just means that a decision will not be reached (liveness), there won't be a case where a decision is reached but later changed (safety).

Each machine keeps a local counter counter and also is assigned a unique server_id offline before everything starts.
Each server tries to make proposals:
1. Send a "prepare-to-propose" message to all machines (or a majority) containing its ballot number which is the pair (counter, server_id).  Increase its counter.
2. When it receives reply from a majority, do the following:
Each reply should contain a (max_ballot_accepted, and accepted_value) but might be null.  If everyone in the majority is null, then send propose message containing the ballot (counter, server_id) with any value.  However, if not everyone in the majority replied null, then sort by max_ballot_accepted, and propose that particular accepted_value.

Each server should also listen for prepare-to-propose message and proposals.  It should also keep track of (max_ballot_accepted), max_ballot_promised, and accepted_value.
1. Upon receiving prepare-to-propose, if the ballot is smaller than max_ballot_promised, ignore, if it is greater then, update max_ballot_promised and reply with (max_ballot_accepted, and accepted_value).
2. Upon receiving a proposal, check to see if it is greater or equal to max_ballot_promised, if so, accept the proposal and update our state.

When is consensus reached.  Consensus is reached when a majority of acceptors accepted the same ballot.  We can find out when that happens if all the processes for example, send out a status update whenever a proposal is accepted.  Of course, the status update might be lost and there is no way of knowing whether the message is lost.  But we are guaranteed that the consensus value will never change once it is changed (safety).

Some intuition on why it works:
1. The prepare-to-propose serves as a barrier to ensure that any accepted ballots will be "known" by later ballots.
2. By only proposing the largest ballot from a majority, we ensure that newer proposed values will remain the consensus value if consensus have been reached.

Once some majority M has accepted a ballot, then any proposer proposing higher ballot numbers must not have sent the "prepare-to-propose" message to this majority until after the acceptance.  Otherwise the prepare-to-propose message servers as an obstacle to getting this particular ballot accepted in at least one common member.
Lower ballot numbers also suffer the same issue.
Now, after receiving reply to prepare-to-propose messsages from a majority M', M' has at least one member in common with M.  We can do a strong induction on the ballot number.  If the current ballot is n, we can assume all proposed ballots between the original accepted ballot and n-1 (inclusive) all propose the same consensus value.  The common member between M' and M would have replied a ballot number at least as great as the original accepted ballot number.  The proposer must propose the value corresponding to the largest ballot number, hence it must also propose the consensus value.

To help us understand better, let's ask whether we can modify the protocol in two ways.
1. Instead of sending out a "prepare-to-propose" message which extracts a promise not to accept lower numbers, can we simply query for proposed values instead of asking for a promise.  We will block out lower ballots at the acceptance step.
The issue is that ballots with the same number might override the consensus value.
We could reach a state each row represents one process, their accepted ballot and value)
1, v1
1, v1
1, v1
2, v2
null
Consider the following sequence of messages (p1 stands for process one).
p1.PrepareToPropose(ballot 1).SendTo(1, 2, 3)
p2.PrepareToPropose(ballot 2).SendTo(1, 2, 3)
p1.Received.PrepareResponse(null, null, null).Propose(1, v1).SendTo(1, 2, 3)
(p1, p2, p3).Accept(ballot 1, v1)
p2.Received.PrepareResponse(null, null, null).Propose(2, v2).SendTo(4) // messages to other members of the majority are lost
p4.Accept(ballot 2, v2)

This is happening because p2 does not know that ballot 1 is already in progress.

2. Is it possible to declare that consensus is reached as soon as a majority has picked the same value?  Right now, we require majority to pick the same ballot.
No.  After majority picked the same value, the consensus value might not remain the same.
We again describe accepted ballot, accepted number, put also show the internal "promise" counter.
To simplify we only show global state transitions.  The user can fill in the messages and lost messages.

1: v1 (1)
1: v1 (1)
2: v2 (2)
2: v2 (2)
null (2) (proposal is lost, but prepare was received)

1: v1 (3) // proposal three only reaches the fifth process
1: v1 (3)
2: v2 (2)
2: v2 (2)
3: v1 (3) // here, we are wrong to think that consensus has been reached

Proposer now prepares and gets response from p1, p2, p3.  It will now propose v2.

4: v2 (4)
4: v2 (4)
4: v2 (4)
2: v2 (2)
3: v1 (3) // consensus value changed

Reducible state space Markov chains

While reviewing a paper last night, I started thinking about reducible state space Markov chains.  Most undergraduate probability students are fed theorems about irreducible Markov chains.  To summarize these theorems.

• If two states communicate, then they are either both recurrent or both transient.
• If a MC is irreducible and all states are positive recurrent, then there exists a unique stationary distribution.
• If a MC is irreducible, aperiodic, and has a stationary distribution, then it converges to that.

We can ask a set of questions about reducible Markov chains to test our intuition for the subject.

• Let's setup a trivial 4 state Markov chain where the first two and last two states form separate communicating classes.
• How many stationary distributions exist?
• Do limiting distributions exist?
• Does the limit distribution depend on the initial distribution?  How many possibilities are there?

Let's try to answer them.

• There are an infinite number of stationary distributions.  If you take the stationary distribution of the first class and the stationary distribution of the second class, then any linear combination (equal to 1) of these two would be a stationary distribution would be a stat. distribution of the full 4-state Markov chain.
• Limiting distributions do exist.  Let's think about the intuition here.  If the total prob. mass on the first two states is m_0 and the rest is 1-m_0, then starting from this initial distribution, each iteration of the chain would "stir" this vector but only stir within their respective classes.  So the first two states of this vector would converge to its limit distribution and same with the last two.  This means the overall limit distribution would be a mixture of the individual limit distributions weighted by the (m_0, 1-m_0) vector.

Course Catalog: Classical Composition

Description: This is a course on classical music composition that will teach the principles of melody, harmony, and orchestration.  However, this course does not teach you to form you musical ideas.  That comes from your talent.  If you don't have it, you can become an orchestrator.  The distinguished alumni of this course usually breaks with the principle we teach them and write great music nevertheless.  (Shostakovich, Debussy)

Course Credit: None

Pre-Requisites: No technical pre-requisite.  However, one should try to live a life of calamity, solitude, and mental instability through the achievement of at least one of the following:

1. Unfulfilling / unrequited love for a string of women who are outside of your social class, for example members of the aristocracy or the royals.  See Beethoven.
2. If you are homosexual, the unfortunate need to oppress your orientation due to social pressures of your society.  See Tchaikovsky.
3. Have a high infant mortality rate.  See Bach.
4. Losing of either vision or hearing.  (See Beethoven or Rodrigo)
5. Live under a repressive Communist dictatorship (Shostakovich)
6. Lastly, having your wife engaging in an affair with one of your best friend.  (Schumann)

AI-assisted political science

In the wake of the Sandy Hook shooting, there has been much debate on the radio about the gun policy in the United States.  The gun advocates stress that banning ownership of guns will not take it out the hands of criminals and decrease violent crime.  Both sides of the debate totes different statistics in order to support their own side.  During the presidential debates, there were numerous occasions where numbers were presented and its truthfulness contested by Obama and Romney.  One instance that comes to mind was when the factual accuracy of Romney's tax policy for the middle class was in dispute.

I think this is one area where information retrieval / data science /weak AI (doesn't mean crappy AI, but just to distinguish it from the well-defined Strong AI term) can really contribute to the public sphere.  Imagine a Google or Wolfram Alpha type program that parses the content of a debate and automatically checks for factual accuracy: gun crime rate, economic data, demographics, tax rates, tax codes etc, perhaps via an overlay on the screen.  This really would sway public discussions from "what are the facts" vs "what is the right policy".  It also prevents cheating and mis-representation of facts.

To push this idea further, imagine an AI-assisted data retrieval / analytics program for the common masses.  To take an example, I wanted to draw a simple correlation graph of gun ownership rates by country vs homicide rate by country.  Just to answer this one simple question, I had to manually enter / merge two separate tables found on Wikipedia, and then filter the data for missing entries using MATLAB, and perform a plot.  This set of computer skill, in the short future, is beyond the common knowledge of the average citizen, yet they need these types of questions answered concretely in order to vote on the right candidate / policy for this country.  How about a more advanced search  / analytics that would, given a question in natural language format, e.g. "Has the crime rate in NYC decreased after 1990?", it would automatically retrieve & clean the relevant data and perform the relevant statistical analysis.  (Regression, classification, etc.)  Currently Google / Bing can answer factual questions, but they are still one step away from answering analytical questions.

Richard Dawkins on the evolutionary basis of morality

I was just watching a Richard Dawkins lecture at Randolph-Macon Women's College on YouTube.  He was asked a question about the basis of atheistic morality.  Dawkins proposed that our moral sense comes from the fact that our pre-historic ancestors lived mostly with next-of-kins hence 1) altruism will benefit the preservation of similar genes 2) there's a high probability of being in long-term contact with these individuals who can then reciprocate the good will.
I would propose that altruism serves to improve upon another evolutionary survival objective, the preservation of the species.  When pre-historic human beings lived in tribes, there was a high risk of destruction from other animals, diseases, and natural elements.  Hence at some point, competition to extinguish a rival member of the same species is out-weighed by the risk that the size of the species might fall below a critical threshold and threaten extinction.  For example, you don't want to kill Adam for stealing your girlfriend because he and you can work together when the tigers attack tomorrow night.
If we think about tribes and animal social organizations as entire units, then that unit would be evolving to maximize its survival as a whole.  If a society randomly generated a moral sense that does not include any altruism in an "attempt" to maximize individual benefits, that group as a whole (and consequently the individual) might have a lower probability of survival because the survival benefits that are derived from the group would be lost (i.e. strength in numbers).  In another words, when the survival of the individual is linked to a certain critical mass and critical health of his immediate social organization unit, this species would likely evolve some moral sense due to the pressure of natural selection even though this moral sense, on the surface, appears to be anti-beneficial to the individual upon a first-order examination.

Similarity between music performance and muay thai

I'm writing a somewhat satirical comparison between music performance (mainly violin) and muay thai.

1. In violin, you practice boring etudes and scales in order to improve your fundamentals so you can better play your beautiful pieces.  In Muay Thai, you practice boring conditioning, mitt, and heavy bag drills in order to improve your athleticism and technique so you can better hold out in your fights.
2. In violin, technique alone is not enough.  At a high enough level, there's a deep, almost unteachable, element of artistic creativity and musicianship that brings the techniques alive in a coherent work of musical art.  In Muay Thai, technique alone is not enough.  At the high level, you need a great sense of strategy, movement, and mental determination that is almost unteachable to bring all your techniques together in a work of martial art.
3. In violin performance, you "practice-perform" your pieces by getting nervous and playing in a Masters class.  In Muay Thai, you "practice-fight" by sparring with partners.
4. Before a violin performance, you become frigging nervous about messing up.  Before a Muay Thai bout, you become frigging nervous about losing and getting messed up.
5. In a Muay Thai fight, if you mess up you might get embarrassed and/or knocked out.  In a music performance, if you mess up you might get embarrassed and/or knocked out.
6. When you're too old to be a travelling musician, you become a teacher in a music school.  When you're too old to be fighting professionally, you become a coach in a gym.
7. If you cannot make a living performing music, you can always teach the instrument.  If you cannot make a living fighting Muay Thai, you can always become a coach or personal trainer.

Practical use for characteristic function & electrical engineering technique in evaluating sines

Yesterday I was thinking about the uniqueness of the heat equation in both the Cauchy setting and the Dirichlet boundary setting.  While thinking about this, the mathematics came down to computing the convolution of the heat kernel with the initial condition which is periodic with fundamental domain \([0,L]\).

After while, I needed to compute the convolution of the heat kernel with a sin function.  In probabilistic terms, I need to compute

$$ E[sin(X)] \; X \sim N(\mu, \sigma) $$

Brute forcing the expected value is very difficult. Then i realized that we can use the good old characteristic function if we represent \(sin(X)\) as \(\frac{e^{iX}-e^{-iX}}{2i}\)

If we convolve a sine signal with some kernel, how do we know the output is still a sine signal?  Well if the kernel is symmetric (even), that means the Fourier transform will be even.  Since sine has a spectrum consisting of two Dirac delta functions at \(+\omega\) and \(-\omega\), then the output of the convolution will be another sine function of a different magnitude.

Why a strong grasp of history is important for scientists

The purpose of the study of history is to learn from the past so we can make better decisions in the future. In a geeky machine learning analogy, it is kind of like processing more "training data" so that we can minimize our prediction error.  A scientist should have a pretty strong concept of the history of science including
1. The history of development in this sub-discipline.  In machine learning, that translates to early inception of the field in A.I., the problems with the A.I. (search space) technique, entrance of statistics (VC, generalization errors), parallel development in neural science, up till the present day.  With this, the researcher can better plan his strategy to maximize success in the long run.
2. Also the broader context of the development of scientific thoughts.  Scientific revolution, paradigm shifts in scientific progress.

Similarity between orchestra and basketball team

Last Tuesday I was playing basketball and then went quickly to my weekly university orchestra rehearsal.  As much as the two different crowds can be, there are some striking similarity between these two activities.

On an individual level, each player on a basketball team or in an orchestra must maintain individual technical proficiency.  The musician practices his/her instrument by spending hours on scales, etudes, and concert pieces to build better facility and musicianship.  The athlete spends hours on drills, to improve shooting,dribbling, defending, endurance.  Both require tremendous discipline and dedication.  On a side note, both also suffer from performance anxiety (like performing at Carnegie Hall or taking that fateful foul shot when you have 5 seconds left in the game)

There is also similarity in the team dynamics.  In an orchestra, each musician subjugates his/her individual interpretation of the piece and tries to blend with the whole group.  Each musician is also given different parts thus different roles.  It can be the solo melody, simmering background accompaniment, or a grand tutti.  The violinist must pass off the beautiful theme to the woodwinds and play pianissimo trills so that the entire orchestra can make music together.  The basketball player need to pass the ball selflessly and sacrifice your own personal glory for team victory.

On a last note, there is the coach/conductor.  Their job is to make sure mend personalities (musicians can have sometimes bigger egos than basketball players) and dictate the team strategy / musical interpretation.

How can PhD in engineering help in music performance?

In the fourth year of my PhD now and just writing a blog piece about some of my changing habits in violin playing/practicing after grueling through some serious academic research.
PhD students are trained to research. To find interesting solvable problems in existing knowledge, explore and make some advances. The whole process is very amorphous and self-directed. First, a large amount of time is spent finding a good problem. This is very fortuitous and often happens at a "Eureka" moment. Over time, the PhD student develops the ability to seek out interesting problems that are solvable. This is similar to a musician picking a piece that interests him/her and is within the reach of his/her technical ability.
After finding a problem, you are required to analyze it into smaller projects that can be tackled.  It also involves a certain about "diagnostics" in the implementation phase, to figure out what stage of your solution has broken down.  A violinist must also analyze his/her target piece.  Usually there would be two phases, a technical analysis and a musical analysis.  In the technical analysis, one must discover trouble spots and usually design efficient and clever "mini-etudes(studies)" to overcome such difficulty.  These difficulties can include

1. Left-hand: lack of finger pressure, side of fingers touching nearby strings, lack of finger dexterity (especially on the fourth finger), dreaded intonation (especially on double stops), getting a nice wide vibrato (difficult on double stops)
2. Right-hand: awkward string crossing, spiccato, upbow staccato, fast whole bow strokes.  Overly noticeable bow changes.

After technical analysis, the musician has to think about phrasing, composition background, balance, dynamics etc.

On the whole, my playing is more rigorous and I have gotten better at figuring out how to practice for a difficult piece.  (Waxmen Carmen fantasy and Devil's Trill right now)

沁园春 和 黄河钢琴协奏曲

昨天路上在我新买的iPad 读《沁园春》。  太久没读汉语的词了， 所以用iPad 得"Pleco reader"边查字典边读。
虽然沁园春开头是描述“北国风光”和“山舞银蛇“而《黄河》歌颂“黄河船夫” 和“黄河愤” ， 但是他们的爱故主义气氛是非常生气勃勃的。  （甚至“保卫黄河”还满杀气勃勃的）  我觉的这两部艺术作品可以互相搭配，妮布。    当然， 《沁园春》二阕离开了自然而走向了政治。

其实用大河或者其它地里世贸作为爱国主义歌曲主题还很长见。  比如斯美塔那的“伏尔塔瓦河”（从他著《我的祖国》），  科普兰的《阿巴拉契亚之春》。    所以我们不要忘记了， 每个国家都有狂热的爱国分子。  但是永远不能让自己国家的利益战胜正义。

Why I left a lucrative financial engineering career?

The film "Inside Jobs" by Charles Ferguson sums it up quite well.

Piano and Violin learning curve

I can't believe it's almost nine months since I started playing the piano (casually).  I must make some quick comments comparing the difficulty curve of the violin vs. the piano.

  In terms of tone production and intonation, the violin is magnitudes harder.  You need precise motor control for your left hand fingers, refined muscle control of the right arm for proper bow pressure, and also a trained ear for detecting minor deviations in pitch.  On the piano, all the notes are laid out and varying finger pressures do not affect the tone that much.  This is very good for beginners because a piano student can start playing nice melodies immediately as opposed to waiting for years before producing a nice tone.

  The piano, on the other hand (no pun intended), requires some tricky coordination between the left and right hands.  At the advanced level, this can become excruciatingly difficult (Hungarian Rhapsody No. 2).  But advanced level violin playing have a slew of dizzying techniques on both hands (left hand pizzacato, quick double stops, spiccato, up-bow staccato).  They have comparable difficulty on the upper hand.

Hence, it is reasonable to conclude that violin has a steeper learning curve.


Voila.

Process of innovation from theory to application

More often than not, science/engineering academic researchers are disinterested in developments and applications in the industry.  This definitely has benefits as it allows researchers the freedom to explore new frontiers without the hindrance of a demand for immediate application.  However, experiences and trends in the industry should influence academics whose research are more applied in nature.
I see the process of human technological innovation in four stages.

1.  Very basic research that yields important paradigm shift in the way that humans think (Switch from Aristotlean to Copernican view of the universe, switch from the occult world to reason and rationalism, acceptance of evolution, Godel's incompleteness theorem).  I would put breakthrough advances in disciplines like philosophy and mathematics in this category.
2. The paradigm shift lead to the creation of new fundamental knowledge that are not directly applicable.  (Physics, chemistry, astronomy, physiology)
3. These discipline in the arts & science lead to more "applied" vocational fields.  These vocational fields create basic technologies (for example, electrical engineering created digital circuits and computers, chemistry created biochemistry & pharmacology)
4. Businesses then generate ideas on how these technologies can be used in clever ways.  (For example, using the internet to sell books, using new flat-panel display technologies to read books, using new biometrics for homeland security).

In another word, at the one end of scale is a philosophy/method of thinking that explains theories which explain specific phenomenons that can solve our specific problems.  The other end are immediately useful things which are only useful immediately.  (Credit this sentence to Olivier Bosquet's blog)

I think it is important for a researcher to know some of all four stages albeit with different weights.  They all feedback on each other and would benefit innovators at each of those four stages.  I would place myself at the third stage leaning towards the fourth stage.  However, I have to do innovation at stage 2 sometime in order to solve a problem that I'm trying to tackle at stage three.  Other times, the problem I'm working on in stage 3 is inspired by an application idea coming from stage 4.

Future of the publishing industry

As I was forced to clean my room today I inevitably thought about the waste & hassle printed books caused.  Then I asked the question of what will be the future of printing?
Here is a very good article on the topci
http://publishing2.com/2007/12/06/the-future-of-print-publishing-and-paid-content/