## Posts

Showing posts from December, 2009

### Why are people attracted to religion?

Quote of the day 2009-12-19

SW: Why do so many FOBs get attracted to religion when they come here?
After someone is twentysomething years how can they still be tricked
like that?
WE: I think it is because they have weak minds
SW: I think they get tricked into coming to the meetings for the free
food

### Problems using iPhone in NYC

What NYC lacks in iPhone connectivity it makes up for in sexy women.

### Interesting properties of the entropy function

Update 2017-07-03: Corrected equation for associative definition, thank you /u/Syrak.

This may not be the first time someone recognized this, but I have recently discovered some interesting and useful properties of the entropy function and now share them.

First a definition:

Entropy — H(p1, p2, ..., pn) — is a function that quantifies surprise in selecting an object from a set where the probability of selecting each object is given: {p1, p2, ..., pn}. This has utility in communications, information theory and other fields of math.
Hb(p1, p2, ..., pn) = Σ(i..n)-pilogb(pi)where b is normally 2, to express entropy in bits.

Other definitions of H() use expected values and random variables. As an analog to the definition above, I will discuss entropy of a set of frequencies. pi = fi / Σ fi.

Entropy defined without bits:

A definition that doesn't use bits is:
H(p1, p2, ..., pn) = Π(i..n) pi-pi And you can convert between the definitions with Hb = logb H.

A recursive (associative) definitio…