Closing of the American mind - Questioning the mixed philosophy which might be contradictory sometimes

 So I have been reading (or hearing, to be precise) "The closing of the American mind" for some months. I hear it often when I am working out, so around 1 hr per day. The audiobook on audible is around 17 hours long, and given my attention span of a small bird, I don't really catch everything Alan Bloom says in the book. 

With the impressive retention of 5%, I imagine something is better than nothing. So I hear along to the meta-philosophy cookbook, where bloom points out the origin of the western liberal thought. I haven't read a lot of philosophy, so I think it wasn't the most wise decision to read the book. Anyhow, as I need to complete my 50 books aim of the year, I don't really care.

This force feeding has actually worked out pretty well. Bloom points out the contradictory nature of the modern man, with ruthless economics on the mind, like a protestant, but also he understands the healing property of nature on man. This contradiction doesn't arise until you're a woodcutter. Otherwise, it keeps at bay. 

We, in our mind, have great capacity to keep contradictory notions in our mind, without actually noticing them. We might need to force ourselves to be crystal clear on what we actually think. 

Also Bloom brought out one more good point. In a democratic society each person is equal. So they don't have anyone above them to tell them what's right or wrong, and therefore they need to make a decision on what is to be done, and how is it to be done. This takes work. What food to eat? What stock to buy? What does the news actually mean? Whom to vote? All these questions, and we need to make choices for them. Not everyone can go and collect information, and think deeply and then make a rational choice. We just don't have enough time for that. Then what do we do in that case? Fallback to the next best choice. What are the others doing? What are the others saying about the current incidents? What are the sectors that will grow in the next 5 years? These decisions are deferred to whatever person we think is good enough to answer these. Unless, obviously, we get bitten back for some, and decide to research and then make an informed decision. 

So the freedom of choice is not exercised, just because we don't have the time to understand what the right choice is for us. 

This is intriguing. 

There was some discussion in the book around science being the prominent source of truth, with are mixed with liberal thought in America, leads them to think that they're non-religious, and thus can go meta, and judge each religion. Each being inferior to the scientific truth obviously. Then (not really sure on this) Bloom points out that, that's what the other religions do. They don't accept other religions. They just tolerate them. So maybe what Americans do isn't meta religion, instead it's a religion in itself. I didn't totally get the point there, as my, again impressive, 5% retention didn't allow me to listen and understand much. 

Why getting a very good voting system will bring dictator ship

 Why getting a very good voting system will bring dictatorship - Arrow's impossibility theorem

Premise

Imagine that we want to improve how the world works, and as a start, we look at our voting system and decide that it does not represent the real preferences of people. So we decide that our voting system should have two conditions:

1. Pareto Efficiency : If every voter says that the party A is more preferred that the party B, then in that case, the result, after voting should also have A ranked above over B.
2. Independence of irrelevant alternatives: Let's say there are two parties, A and B, and there is an output A > B, in the results. Now let's shuffle the other preferences of voters for every other party than A and B. Then in that case, the relative ranking of A and B shouldn't change in the result, even though there absolute position in the output ranking can change.

So what? Our system already has these two conditions!

So these two perfectly reasonable conditions are the ones that we want to have in our voting system. 

Now let's say our system is a majoritarian system, that is, the one person getting the maximum amount of votes will be the winner. I will try to prove, that in this voting system, there are cases when the second condition, independence of irrelevant alternatives (IIA), will be broken. 

Let's say we have a total of 21 voters, and 3 parties in the election, with the following configuration:

• 10 voters have the preference : $ A > B > C $
• 2 voters have the preference : $ C > B > A $
• 9 voters have the preference : $ B > C > A $

Now in accordance to this voting system, A would win. But let's remove C from this. In that case, this would be the situation :

• 10 voters : $ A > B $
• 2 voters : $ B > A $
• 9 voters : $ B > A $

That is, 11 votes to B, and 10 votes to A, which would mean that B would win.

This means that the presence of C, actually changes the results, and the relative position between A and B. Which means that this system doesn't follow the IIA condition.

Then let's improve things?

Okay. So this means let's go towards a system which actually has these two conditions followed. But there is a mathematical proof which says that if both of these conditions are followed, there will be dictatorship. Wow. Pretty dark theorem.

Less go.

Why can't we improve things?

Let's prove one important theorem, which is a secret tool that will help us later.

Theorem 1: If some preference B is either at the top or bottom of everyone's preference sequence, then it would be either on the top, or at the bottom of the result's sequence.

Let's prove this using contradiction. Let's have $n$ voters with different preferences, each represented by a line. As shown in the  diagram. Now, let's say that in the output, the output $B$, is in the middle. It's neither at the top, nor at the bottom. So this would mean that it has some output $A$ above $B$, and $C$ below $B$. 

Now as you can see in the output $ A > B $, and $ B > C $. Now IIA says that only the relative positions of A, and B determine $A > B$, and the relative positions of B and C, determine $B > C$. So let's just change the positions of A and C in each of our voter's preferences, and put C above A. This would mean that $C > A$. 

This neither changes the relative position of A and B, nor the relative positions of B and C, since A and C are either both above or below B.

Now the condition looks some thing like:

Now using Pareto efficiency (PE)  which was our first condition : Since everyone agrees that $C > A$, therefore, in the results, we should also see that $C >_{Result}  A$.

Now let's go to our original conditions, that is : $A > B$, and $B > C$. Due to transitivity, $A > C$. 

So we have a contradiction, and $B$ can never be in the middle. It can either be at the top, or at the bottom of the result.

[X] (The box that comes in math books, after the "hence proved").

Theorem 2: In the voting system where PE and IIA both are present, there always will be a dictator.

Slow down death. 

Let's say initially we have $B$ at the bottom of every voter's preference. And gradually we move $B$ from the bottom to the top of the preference sequence. Let's say there is some voter $n^*$ who, as soon as moves $B$ from bottom to the top of his preference sequence, the result also puts $B$ back to the top. 

Due to theorem 1, we can say that $B$ will go directly to the top, because $B$'s are either at the top or at the bottom of the preferences of every voter.

The process can be described here:

Now let's prove that this guy, $n^*$ is a dictator for every pair A and C (excluding B). This means that, $n^*$ decides the result directly for every pair of outputs.

Let's put A above B for $n^*$ only, and let all the preferences for all the other voters remain same. For all the other preferences the relationship between A and B's positions is random.

The output should look something like this:

Now, let's compare this to the time, when $n^*$ had decided to put $B$ at the bottom of their preferences. 

In that case, the result had $B$ at the bottom. That is $A > B$. But notice that in this case too, the relative positions between A and B are the same. A is above B, for $n^*$'s preferences too. 

So using IIA we can say that for this case too, $A>B$.

Now let's see the case, when $n^*$ has shifted B to the top of their preferences. In that case, C is below B for $n^*$, and also in the result.

In this case too, C is below B. Moreover, the relative positions of B and C are the same, so we can directly say that $B > C$.

Using this, we can say that $n^*$ is a dictator for all pairs A and C.

Now let's see whether this works for B too. Well obviously! Since B's position at the top, for $n^*$'s preferences determines that the result has B at the top, and vice-versa, we can say that $n^*$ is a dictator for B too.

Using this information, we can say that $n^*$ is a dictator for all pairs.

Improving at competitive coding

 How not to bitch about solving lesser problems

Whenever I ask anyone about how to improve, all they say is "Solve more problems". Bitch. I know that. Do you have anything else to offer? 

So just like other things, this too we have to do on our own. And what does that mean? Analysis!

So I thought of all the phases of solving a problem and made this shitty diagram : 

My main aim to find the most time taking stages. What I understood was, that the main "problem solving" stage, where we think the hardest is the most important one. It might not be the one that takes most of the time itself, but surely is the one that directly affects the total time taken.

Let's just say that I as a person can solve some problem in $T$ minutes, when no re-implementation or code changes are made.

Now let's consider the case that I ran some tests on the code, and it failed for some case. Either we did some silly mistake, so we need to debug it, or our solution is wrong, and we need to rethink the problem.

In this case, again, intense thinking is required, and immense time is wasted. What we can do is make sure that the solution we're implementing is the best version we can give. It's well tested, via dry runs, and should have the least amount of bugs. 

This is a doable thing. It's more of a good habit, than a skill upgradation, as it doesn't require you to improve problem solving, but just stopping a while, testing the solution well in mind, creating some edge cases, and then  implementing it. 

This should save massive amounts of time in competitions. Let's see if this actually improves anything, or is just the intuitive thing people anyways follow. Big eh time.

1335C - Counting Triangles - How to be a bitch of a problem 101

 1355C - Counting Triangles

Problem:

So here is this 1800 rating question acting like it's not that  hard by being a Div2 C. No, fuck you 1355C, we know your truth.

The problem can be found here.

What does it say?

Well you have to make a triangle. You're given four values $A,B,C \text{ and } D$, such that $A \leq B \leq C \leq D$. Now you have two create a triangle, such that the sides of the triangle are of lengths $x,y,z$ such that, $A \leq x \leq B \leq y \leq C \leq z \leq D$. 

Initial thoughts

A triangle can be created when $ x + y \geq z $, $y + z \geq x$ and $x+z \geq y$. Since we have to have only those triangles which have a positive are, thus the equal to sign is removed. Thus $$ x+y > z \\ y + z > x \\ x+z > y $$.

But since $z$ is already greater than $x$ and $y$, you only need to consider $ x+y > z$.

Delving deeper

So, since the differences between A,B,C and D can go upto $5 \times 10^5$, forget brute force. If you have something $O(n^2)$, forget it too. We need something better here.

We have three areas (A to B for x, B to C for y and C to D for z) to cover. So, we might lean into pointers or something, I don't know I'm illiterate.

Since, I know the answer now (which I definitely knew after reading the editorial only), I don't have any other option than to delve straight into the solution.

The Solution

We take two pointers, $xp$ for $x$ and $zp$ for $z$. Now we can calculate minimum $y$, that satisfies our condition by $yp = zp-xp+1$ (remember that we had removed the equals sign).

We can check whether this $yp$ value is sane or not, by checking if it's between B and C only, but more on this later.

Now we know that a triangle can be formed by $xp$, $yp$ and $zp$. Notice that if we increase $yp$, the condition would still hold. That is, $$xp+yp > zp \\ xp+yp+1>zp$$.

Glad to know that. So, that means if we keep $xp$ and $zp$ constant, all eligible y's, from $yp$ all the way to $C$ are eligible triangles.

We can get their number as $C-yp+1$

Noice.

Now, what if we increased $xp$ by one? 

So if $$xp+yp>zp \\ xp+1+(-1)+yp>zp \\ xp + 1 + yp -1 > zp $$

What does it mean? It means that if we increase $xp$, then we can still satisfy the same condition by decreasing the value of $yp$. So, notice that in this case, our eligible triangles would increase by one from the previous case.

To be precise, it would be $C-yp+2$.

Do we see a pattern here? 

Yes. As we increase $xp$, we can decrease $yp$ and increase the number of eligible triangles.

But there should be some limit right. Yes. So you can only decrease $yp$ until $B$. Anything less is not acceptable.

So we increase the value of eligible triangles till then, which would be $C-B+1$, in the end.

But what if we still have some x's left? That is, we traversed by increasing $xp$ and decreasing $yp$. Our value of $yp$ could not be reduced anymore because it had reached it's limit $B$, but $xp$ still had not reached it's upper limit $B$. So since the current value of $xp$ satisfies the condition $xp+yp>z$, we know that if we increase $xp$, then the constraint would still be satisfied.

Notice, that in that case, the total number of eligible triangles would be limited to $C-B+1$ for each such "extra" $xp$.

Now you're getting a hang at it don't you.

Now coming back at the sanity of $yp$ initially. you should know that if intially $yp$ is greater than $C$, then no valid $y$ value satisfies that. So in that case, your $x$ was not good enough to create something greater than $z$, so you increase $xp$.

If in some case $yp$ is less than B, then voila! For all the remaining x's, you can take whole range of $y$. That is $C-B+1$.

I think this should be enough to make the core concept of the problem clear.

Anyhow, if you don't get something or something is wrong here, please leave a comment.

Here is my submission.

Gradient of a softmax classifier and it's loss function

Softmax classifier and it's gradient

The function itself:

The softmax function can be seen as:

$$ S_i = \frac{e^{a_{i}}}{\sum_{k}{e^{a_{k}}}}$$

and the loss function can be seen as:

$$L_i = -log(S_i)$$

The partial derivative as taken from here  :

If you do a partial derivative of $S_i$ w.r.t the $a_{i}^{th}$ element, then, remember the quotient rule:

If $$f(x) = \frac{g(x)}{h(x)} $$

Then,

$$f'(x) = \frac{g'(x) \times h(x) - h'(x)\times g(x)}{h(x)^2} $$

Here, $g(x) = e^{a_i}$, and $h(x) = \sum_{k}{e^{a_k}}$,

So $g'(x) = e^{a_i}$, when partially derived w.r.t $a_i$, and $g'(x) = 0$, when partially derived w.r.t $a_j$, where $j \neq i$, because in that case it'll be a constant.

$h'(x) = e^{a_j}$ always, because only one of all the terms in the summation will not be treated as a constant. All others will be treated as a constant, and thus they'll be diminished to zero after being derived. 

Thus when $i=j$, our 

$$\frac{\partial {S_i}}{\partial a_i} = \frac {e^{a_i}\times \sum - e^{a_i}\times e^{a_i}}{\sum^2} = S_i(1-S_i)$$

And when $i \neq j$

$$\frac{\partial S_i}{\partial a_j} = \frac {0 \times \sum - e^{a_i} \times e^{a_i}} {\sum^2}= -S_i \times S_j$$

Now keep these in mind. Now following this stackoverflow solution : 

$$\frac{\partial L}{\partial o_i}=-\sum_ky_k\frac{\partial \log p_k}{\partial o_i}=-\sum_ky_k\frac{1}{p_k}\frac{\partial p_k}{\partial o_i}\\=-y_i(1-p_i)-\sum_{k\neq i}y_k\frac{1}{p_k}({\color{red}{-p_kp_i}})\\=-y_i(1-p_i)+\sum_{k\neq i}y_k({\color{red}{p_i}})\\=-y_i+\color{blue}{y_ip_i+\sum_{k\neq i}y_k({p_i})}\\=\color{blue}{p_i\left(\sum_ky_k\right)}-y_i=p_i-y_i$$

And thus we have our solution! 

I took some time to understand this, so a good I decided to create some resource for me too look back to.

[Persistent Segment Trees] 351 - D Jeff and Removing Periods

351 - D - Jeff and Removing Periods

Problem statement:

Problem can be found here.The gist of the  problem is:

We're given an array $a$ of length $n$. With that we're given $q$ queries too, which are of the form $l r$. We have to answer the total number of unique numbers in that range plus if any number present with a constant gap (AP of indices) in the range then add 0, else if no such element is present then add 1.

Notice that if an element is present two or one times, then it automatically satisfied the condition mentioned

Approach:

There are two approaches to solve this question:

1. Mo's algorithm for sorting the queries and finding the answer. I have done this question and here is the solution. We will not discuss it here, since it is already discussed in the editorials.
2. Persistent segment trees. 

I followed rlac's submission to understand  this method.

Description:

You need to know about persistent segment trees, in order to understand this solution. If you don't then you can find material regarding them at these places:

• GKCS-MKTHNUM (SPOJ)
• This codechef tutorial

In my opinion, the idea isn't that much hard to understand. It's just like version control. Once you update some value, which is a leaf, you create new parents till the root, for each update.

Now to solve this problem we need two persistent segment trees:

1. To find the total number of  unique numbers for a range $l,r$
2. To find whether any element's indices follow any AP for a range $l,r$

For the first tree, the logic can be like:

We go from 0, to n-1 in this case.

You create an array pos, which saves the element's last occurrence. So, if the current element has been seen before, then we update that position with a zero (this creates a new root). Then for the current number, we update the for the current index with a 1. We can save this root as the ith root.

When queried, we can query the segment tree root r. The operation performed is the sum operation. That is, the parent's value is calculated by adding the values of the left child and the right child.

Initially, I thought why not just use a regular segment tree? But actually changing the one to zero, might affect some answers, so a regular segment tree might have not worked in that case.

Now, for the second tree, we can again use pos to save last occurrences, next_pos, to save the next position of an element in the array, and last to save the element after the last element of the AP.

Note that we go from n-1 to 0, in this case.

So, if the current element was present before, then it's index' value is set to zero for this segtree. Now if we have a next->next element, then we check if an AP is being formed. If not, then we know that two elements always form an AP, so the last value of the current element is the next->next value.

If next->next indeed follows an AP, then we have atleast three elements in AP, maybe more, maybe less. A check must have also been done for the next element which was saved in pos[a[i]]. We can save it's last value as last[i].

Now when queried, we go for the lth 2nd segment tree, and check for (l,r). Here the operator is the max. That is, the value for the parent node is calculated by taking the max of the left and right child.

If the queried result is less that r, then we know the AP got fucked up before the query r. So we know no AP is present in the range.

If the queries results is more than r, then we know that there is an AP in the given range, as the last element after the AP is more than r.

I hope this explanation helps

Here is my submission which I absolutely stole from rlac.

[Trees][Binary Lifting] 501 - D Misha And Permutations

Misha and Permutations

Problem statement:

The problem can be found here. The gist of the problem is:

You're given two permutations of $n$. Their order might be $a$, and $b$. By "order" I mean, the index of the permutation. That is, for $n=3$, $0,1,2$ is the first permutation, whereas, $2,1,0$ is the last (=6th). 

So, we have to find $(a+b)%(n!)$th permutation. 

This is it. Simple. Isn't is? 

No. Nope. Nada. Wtf is this question man?

Okay, rants apart.

Ideas:

The major point here is the conversion of order, and the permutation. We convert the initial permutations into the order. Add them. Mod by n!, and then find the resulting permutation.

There are many problems with this approach. You'll have to find ways to interconvert order and permutation, plus the mod by n!. Since $1 \leq n \leq 200000$, thus $n!$ can be a very large number. You don't want to solve, this way.

Enter the [Factorial Number System](https://en.wikipedia.org/wiki/Factorial_number_system).

As the name suggests, it's the factorial number system. Instead of bases, you have factorials! An example would be:

$$ 4\times 4! + 3\times 3!+2\times 2!+1\times 1!+0\times 0!$$.

This number in factorial representation (or factoradic) can be written as $43210_{!}$, which in decimal representation can be written as $119_{10}$. 

A good property is that, we can represent permutations using factoradics. This way, we can ignore the decimal representation altogether. But you'll say what about the n! mod? We'll come back to that later. I promise.

Pemutation to factoradic conversions:

Given a permuation $3,0,2,1$. It's factoradic representation is not simply $4132_{!}$. We have to put some mind to it. 

We have $n=4$. So we have $A\times 3! + B\times 2! + C\times 1! + D\times 0!$. We have to find the values of $A,B,C$ and $D$. 

A = We can simply do that by finding number of elements smaller than 3. That is 3. 

Now we remove 3 from consideration, and have $\{0,1,2\}$. 

B = Number of elements smaller than 0? None. So 0. Remove 0. We're left with $\{1,2\}$.

C = Number of elements smaller than 2? 1. So 1. We have $\{1\}$ left.

D = 0

So, we have $3010_{!}$, representing $3,0,2,1$. To double check, you can find the decimal value, which comes out to be: $3\times 3!+0\times 2!+1\times 1!+ 0 \times 0! = 18+0+1+0 = 19$

As you can see here, 3021, is indeed the 19th permutation for n=4. 

The basic inference you can get from here is that factoradic representation gives the rank of the number, for the remaining numbers. 

This can be done by fenwick trees / Binary indexed trees, easily.

Modulus n!: 

Thanks to Ecenerwala's solution.

So, until now, we have converted the permutations into their factoradic representations, and added them. How do we do the mod n!?

A factoradic representation is of the form (for n):

$$X_{(n-1)!}\times (n-1)!+X_{(n-2)!}\times(n-2)!+...+X_{2!}\times 2!+X_{1!}\times 1!$$.

Thus if $X_{(i-1)!}>=i$, then it can be written in the form: $X_{(i-1)!} = i+(X_{(i-1)!}-i)$. Thus, the term $$X_{(i-1)!}\times (i-1)! = i \times (i-1)! + (X_{(i-1)!}-i\times (i-1)!) \\= i!+(X_{(i-1)!}-i\times (i-1)!$$

This extra $i!$ can be added to the next greater element. So, if we percolate up to $(n-1)$, then we might get an extra $n!$ element. That's it!

Isn't that what we want? We know that there can be at most one n! extra. You can check that. (Kuchh to krlo khud se)

Code for that:

Good. Beautiful.

Factoradic to permutation conversion:

So, as I mentioned earlier, factoradic-> Rank (well actually, number of elements smaller). So you have ranks, and  you want the permutation now. Consider $3010_{!}$ again. You want the (3+1)th element, amongst $0,1,2,3$ = 3. Remove 3. $\{0,1,2\}$ left. You want the (0+1)st element. You get 0. Remaining $\{1,2\}$. You want the (1+1)nd element. You get 2. Remaining $\{1\}$. You want the (0+1)st element. You get 1. So $3,0,2,1$ formed.

This can be done by binary searching on he fenwick tree, or doing binary lifting on it. 

My submission can be found here.