Posts

Showing posts from December, 2009

Why are people attracted to religion?

Quote of the day 2009-12-19

SW: Why do so many FOBs get attracted to religion when they come here?
After someone is twentysomething years how can they still be tricked
like that?
WE: I think it is because they have weak minds
SW: I think they get tricked into coming to the meetings for the free
food

Problems using iPhone in NYC

What NYC lacks in iPhone connectivity it makes up for in sexy women.

Interesting properties of the entropy function

Image
Update 2017-07-03: Corrected equation for associative definition, thank you /u/Syrak.

This may not be the first time someone recognized this, but I have recently discovered some interesting and useful properties of the entropy function and now share them.

First a definition:

Entropy — H(p1, p2, ..., pn) — is a function that quantifies surprise in selecting an object from a set where the probability of selecting each object is given: {p1, p2, ..., pn}. This has utility in communications, information theory and other fields of math.
Hb(p1, p2, ..., pn) = Σ(i..n)-pilogb(pi)where b is normally 2, to express entropy in bits.

Other definitions of H() use expected values and random variables. As an analog to the definition above, I will discuss entropy of a set of frequencies. pi = fi / Σ fi.

Entropy defined without bits:

A definition that doesn't use bits is:
H(p1, p2, ..., pn) = Π(i..n) pi-pi And you can convert between the definitions with Hb = logb H.

A recursive (associative) definitio…