Coding up Augmented Spatial Pooler

As of now, I am focusing all my efforts on coding up Augmented Spatial Pooler as defined in this paper (working on it as of 20 September 2015)

http://www.ict.griffith.edu.au/~johnt/publications/IJMLC2012.pdf

We have put a binary version of spatial pooler here.

Our Spatial Pooler implementation on Github

we have also put a Augmented Spatial Pooler version of the code here.

Our Augment Spatial Pooler implementation on Github

Augmented Spatial Pooling and Spatial Pooling for Greyscale Images

I am trying to better understand spatial pooling and am wanting to code up a working version of it on my own.  I found two works (shown below) by Professor John Thornton in Australia as quite useful in this regard.  I hope to implement these in time.

I had trouble keeping up with my overall schedule this week but I plan to finish it all up by tomorrow.

 

Onto Spatial Pooler

Written by Chirag on Sunday, July 26, 2016 (around: 6:30pm)

This week I am working on the Spatial Pooler from Numenta’s Cortical Learning Algorithm white paper.  Instead of typical CLA (Cortical Learning Algorithm) reading, I will be focused on implementing and testing my own version of CLA-Spatial Pooler (page 34 has the pseudocode).  So that I can  understand it better.

CLA has two key input pooling components. Effectively, in Jeff Hawkins words, pooling is the mapping of a set of inputs (visual, auditory, smell, sensorimotor) onto a single output pattern. There are two basic forms of pooling.

1) Spatial Pooler: “Spatial Pooling” maps two or more patterns together based on bit overlap. If two patterns share sufficient number of bits they are mapped onto a common output pattern.

2) Temporal Pooler: “Temporal Pooling” maps two or more patterns together based on temporal proximity. If two patterns occur adjacent in time they are likely to have a common cause in the world.

Quite honestly, given my limited python skills, I am finding it very difficult to code up the Spatial Pooler.  I am thinking it will take me at least six months. But, I think this will be worth the deep dive.

 

Learning Bayes’ Theorem

Written by Chirag on June 21, 2015 at 8:00PM Eastern Time

This is week three of my personal goal of learning/implementing all that is Brain and Neuroscience related.  It’s going pretty well.  I have been quite diligent in keeping up with my nine self-assigned tasks (see previous post table).  This week, what stuck out to me is Bayes’ Theorem.  It is used quite heavily on neuroscience modeling. In fact, a book I am reading “Probabilistic Models of the Brain” noted this

“There is now substantial evidence showing that humans are good Bayesian observers.”

Also a post by Vicarious cofounder, Dileep George, on his blog, references some paper that uses Bayesian Inference.  I have printed out the paper and its on my reading list.  But the key question here again is What the heck is Bayesian Inference???

So from the above statement and other papers I have been reading, it has become pretty clear that I must read all about Thomas Bayes and his Theorem.

In Graduate School and most probably in undergrad statistics classes, I learned about Bayes’ Theorem.  Prior to my readings, I had vague memory of it being something related to conditional probabilities.  I think it’s best to drop an example right about now. I think we all learn from examples, not sure why people start with abstract thinkings.

Bayes’ Theorem basically gives you what is called the posterior probability. A way of reversing conditional probability.  If you’ve got probability of symptom based on disease, how can you get Given a symptom, what is the likelihood you have X disease?  The reverse probabilities are almost always more useful.

P(A|B)  apply Bayes Theorem and you get P(B|A)

One of the most famous mathematician, Carl Jacobi, used to repeat to himself Invert always Invert.. as in solve problems backwards.  Bayes Theorem lets you invert. So it’s a useful tool.

Here is a seriously made up example..

Probability of Heartburn given that you’re an Investment Banker is 90%

Probability of Heartburn given that you’re a Techie is 1%

now what is the probability that you’re an investment banker given that you have a heartburn?? (posterior probability)

or Probability (Investment Banker|Heartburn)

Well it turns out you can figure that out by knowing some probability that you’re an investment banker and percentage of the population that has heartburn.

Let’s say 3% of the population has heartburn

Let’s say 1% of the population is investment bankers.

Probability(Investment Banker|Heartburn) = Probability(HeartBurn|InvestmentBanker) * Probability(Investment Banker)/ Probability(Heartburn)

= 90%*1%/3%  = 30%

This means that if you have a heartburn, there is almost 30% chance that you’re an investment banker.  I know, in this made up world, it would suck to be an investment banker.

Now let’s see if we can figure out probability (Techie|Heartburn)

We need some additional information.  What is the probability that you’re a techie of the population.  In this Utopian world, 30% of the population is Techies.

Probability(Techie|Heartburn) = Probability(HeartBurn|Techie) * Probability(Techie)/ Probability(Heartburn)

= 1% * 30%/3%

= 10% Chance You’re a Techie, if You’ve gotta heartburn.

So basically, you can use above kind of trickery to figure out what someone’s career might be.  Not really, this is just an example. In real world, one can use this kind of inference techniques to figure out what is the probability you have certain disease based on symptoms you show for example.  Two diseases may have similar symptoms  but one is extremely rare.  In which case, the correct conclusion (or inference) may be, with a high degree certainty, that the person has the more common disease and not the super rare deadly disease.  We draw these kind of conclusions in our everyday analysis without realizing.

For example the heartburn example..bankers are notorious for working long hours and eating late at night regularly at a young age..my doctor recently told me.. that almost all young people that come to his office with heartburns are usually investment bankers from a certain firm.  So if you’re a young person, with an heartburn, the doctor might infer that you’re an investment banker.  Even though, investment bankers aren’t a large group of population.

I am sure, I will come up with more relevant example to brains as we go along. But just wanted to give you all a flavor of what is to come.