Strange Loops

Written by Chirag on Sunday, July 12, 2015 at about 2:35pm

Over the past week, I have been more aggressive in trying to keep up with my schedule.   I have added one more book to my weekly reading list and have defined a key question that needs to be answered by science.

1) I have added Thinking Recursively by Eric S. Roberts.

2) The key question is why does our brain think it is alive?  To me it is endlessly fascinating that you can put together bunch of chemicals (as in what’s in our brain) and if put together exactly right, you create a system that thinks it is alive? How does this happen?  To me only person, who has even tried to ask this question and come up with a solution is Douglas Hofstadter in his book GEB. More on this a bit later!

Recursions and Strange Loops

Recursion: Separately, my view is that Strange Loops (Douglas Hofstadter’s Theory) and Recursion are quite related.   Also, as I read more about HTM, I don’t think the Brain uses Bayes’ Theorem directly.  It is coincidental as in the Brain is a memory system, so it is constantly using priors to make future predictions.  So it gives us a feeling that it is using Bayesian Inference to come up with a most likely outcome.

It’s worth emphasizing that Vicarious (AI company) algorithms are defined as Recursive Cortical Networks (RCNs). As per wikipedia, RCN is a visual perception system that interprets the contents of photographs and videos in a manner similar to humans. The system is powered by a balanced approach that takes sensory data, mathematics, and biological plausibility into consideration. On October 22, 2013, beating CAPTCHA, Vicarious announced its AI was reliably able to solve modern CAPTCHAs, with character recognition rates of 90% or better.

Also, I think Recursion by itself is a pretty cool tool to have in your arsenal as it is a super awesome problem solving technique.  I have coded my first recursion example in Python. Yes, am getting little more comfortable with Python and have pushed it out on my Github account here: https://github.com/g402chi

Strange Loops:As I am reading GEB by Douglas Hofstadter and his theory of Strange Loops. Example of Strange Loops are Catch-22s or Which came first the Chicken or the Egg. I like to use examples first because that is usually the easiest way to get a point across.

As per wikipedia:”A strange loop arises when, by moving only upwards or downwards through a hierarchical system, one finds oneself back to where one started.

Strange loops may involve self-reference and paradox. The concept of a strange loop was proposed and extensively discussed by Douglas Hofstadter in Gödel, Escher, Bach, and is further elaborated in Hofstadter’s book I Am a Strange Loop, published in 2007.

A tangled hierarchy is a hierarchical consciousness system in which a strange loop appears. In short, a strange loop is a paradoxical level-crossing feedback loop”

I realize Douglas Hofstadter, in his book GEB, is absolutely trying to answer the right question.  As in how does the brain of any given animal think it is alive.  This is the most key, tremendously important, question that modern science should be trying to answer.  My conjecture is that we can not build truly intelligent machines until we answer this question. My guess on the future is that if the nature is able to build this brain that thinks it is alive, it must be possible to do it.  We need to figure this out. I am not even talking about a human brain, take a C. Elegan brain, ant brain.  However small you would like it to be. All these insects, they are self aware and are acting on their behalf only.

As per Douglas Hofstadter,

“the psychological self arises out of a similar kind of paradox. We are not born with an ‘I’ – the ego emerges only gradually as experience shapes our dense web of active symbols into a tapestry rich and complex enough to begin twisting back upon itself. According to this view the psychological ‘I’ is a narrative fiction, something created only from intake of symbolic data and its own ability to create stories about itself from that data. The consequence is that a perspective (a mind) is a culmination of a unique pattern of symbolic activity in our nervous systems, which suggests that the pattern of symbolic activity that makes identity, that constitutes subjectivity, can be replicated within the brains of others, and perhaps even in artificial brains.

Learning Bayes’ Theorem

Written by Chirag on June 21, 2015 at 8:00PM Eastern Time

This is week three of my personal goal of learning/implementing all that is Brain and Neuroscience related.  It’s going pretty well.  I have been quite diligent in keeping up with my nine self-assigned tasks (see previous post table).  This week, what stuck out to me is Bayes’ Theorem.  It is used quite heavily on neuroscience modeling. In fact, a book I am reading “Probabilistic Models of the Brain” noted this

“There is now substantial evidence showing that humans are good Bayesian observers.”

Also a post by Vicarious cofounder, Dileep George, on his blog, references some paper that uses Bayesian Inference.  I have printed out the paper and its on my reading list.  But the key question here again is What the heck is Bayesian Inference???

So from the above statement and other papers I have been reading, it has become pretty clear that I must read all about Thomas Bayes and his Theorem.

In Graduate School and most probably in undergrad statistics classes, I learned about Bayes’ Theorem.  Prior to my readings, I had vague memory of it being something related to conditional probabilities.  I think it’s best to drop an example right about now. I think we all learn from examples, not sure why people start with abstract thinkings.

Bayes’ Theorem basically gives you what is called the posterior probability. A way of reversing conditional probability.  If you’ve got probability of symptom based on disease, how can you get Given a symptom, what is the likelihood you have X disease?  The reverse probabilities are almost always more useful.

P(A|B)  apply Bayes Theorem and you get P(B|A)

One of the most famous mathematician, Carl Jacobi, used to repeat to himself Invert always Invert.. as in solve problems backwards.  Bayes Theorem lets you invert. So it’s a useful tool.

Here is a seriously made up example..

Probability of Heartburn given that you’re an Investment Banker is 90%

Probability of Heartburn given that you’re a Techie is 1%

now what is the probability that you’re an investment banker given that you have a heartburn?? (posterior probability)

or Probability (Investment Banker|Heartburn)

Well it turns out you can figure that out by knowing some probability that you’re an investment banker and percentage of the population that has heartburn.

Let’s say 3% of the population has heartburn

Let’s say 1% of the population is investment bankers.

Probability(Investment Banker|Heartburn) = Probability(HeartBurn|InvestmentBanker) * Probability(Investment Banker)/ Probability(Heartburn)

= 90%*1%/3%  = 30%

This means that if you have a heartburn, there is almost 30% chance that you’re an investment banker.  I know, in this made up world, it would suck to be an investment banker.

Now let’s see if we can figure out probability (Techie|Heartburn)

We need some additional information.  What is the probability that you’re a techie of the population.  In this Utopian world, 30% of the population is Techies.

Probability(Techie|Heartburn) = Probability(HeartBurn|Techie) * Probability(Techie)/ Probability(Heartburn)

= 1% * 30%/3%

= 10% Chance You’re a Techie, if You’ve gotta heartburn.

So basically, you can use above kind of trickery to figure out what someone’s career might be.  Not really, this is just an example. In real world, one can use this kind of inference techniques to figure out what is the probability you have certain disease based on symptoms you show for example.  Two diseases may have similar symptoms  but one is extremely rare.  In which case, the correct conclusion (or inference) may be, with a high degree certainty, that the person has the more common disease and not the super rare deadly disease.  We draw these kind of conclusions in our everyday analysis without realizing.

For example the heartburn example..bankers are notorious for working long hours and eating late at night regularly at a young age..my doctor recently told me.. that almost all young people that come to his office with heartburns are usually investment bankers from a certain firm.  So if you’re a young person, with an heartburn, the doctor might infer that you’re an investment banker.  Even though, investment bankers aren’t a large group of population.

I am sure, I will come up with more relevant example to brains as we go along. But just wanted to give you all a flavor of what is to come.