Saturday, July 21, 2012

Memoize in python and html5

Memoization is a concept from algorithms which finds heavy use in dynamic programming. It is an optimization technique which gains time at the cost of space. So yes, this is an useful technique to know and implement in your algorithms.

The key idea is to have a data store which stores values computed and reuses them to compute the current iteration. The equation is particularly of the form

T(n) = T(n-k1) operator T(n-k2),

In case of the fibonacci series, f(n) = f(n-1) + f(n-2). If you see a computation of this form, you can readily apply a DP technique. Memoization provides a good infrastructure for writing such computations.

The implementation in python is really simple. Apply a decorator, hash the input parameter and if the function is called on the same input just return the already computed value. This is of course assuming all else is stationary and there are no side effects in the program.

Here is a small example from python which is from http://wiki.python.org/moin/PythonDecoratorLibrary#Memoize

import collections
import functools

class memoized(object):
'''Decorator. Caches a function's return value each time it is called.
If called later with the same arguments, the cached value is returned
(not reevaluated).
'''
def __init__(self, func):
self.func = func
self.cache = {}
def __call__(self, *args):
if not isinstance(args, collections.Hashable):
# uncacheable. a list, for instance.
# better to not cache than blow up.
return self.func(*args)
if args in self.cache:
return self.cache[args]
else:
value = self.func(*args)
self.cache[args] = value
return value
def __repr__(self):
'''Return the function's docstring.'''
return self.func.__doc__
def __get__(self, obj, objtype):
'''Support instance methods.'''
return functools.partial(self.__call__, obj)

@memoized
def fibonacci(n):
"Return the nth fibonacci number."
if n in (0, 1):
return n
return fibonacci(n-1) + fibonacci(n-2)

print fibonacci(12)

This is pretty neat if you ask me, saves you a lot of time when dealing with computations.

Now the fun part, HTML5 provides localStorage which is basically a dictionary into which you can write values and read back across page refreshes, AKA semi-persistence. This is sought to be a sort of replacement to cookies and most of the computation is done client side. So you could fetch data from the server and store it here, so you are paying for the network latency everytime.

The access usage goes somewhat like this

if(localStorage){
if(localStorage['myvalue1'])
return localStorage['myvalue1']
else
localStorage['myvalue1'] = value1

This is readily accessible structure, and should dramatically improve your performance. But as with anything web, there are security concerns. This is visible to anyone or anything with machine access, so

1. Encrypt your keys.
2. Don't store anything extra sensitive here.
3. Ask the user before doing this. Its only fair.

 Happy memoization.

Sunday, July 15, 2012

Dependency Injection

Figuring out dependency injection manually is hard, I mean it really really hard. linkedIn has a library for depenecy injection https://github.com/Jakobo/inject. This is really great, but considering I don't care much about doing a require for others to load its not my cake.

underscore.js has unique features _bind to

Thursday, July 5, 2012

Understanding currying

As functional paradigms are increasingly finding ways into my non-functional programs, I felt it was time to upgrade my skills.

I have been dabbling with haskell lately and will update some functional concepts.

Currying: What is it?

Currying is parameter application to functions. In c++, we can have arguments passed to a function.

int foo(type par1, type par2, ...){

    //perform operations on the par's

}

This is really great, but in functional programming paradigm; every function takes only a single argument. Oh no! what are we to do? Well here is where currying comes to the rescue. If you pass a function multiple parameters in functional programming languages, such as scheme and haskell, they perform currying of the function.

How it works

The function takes the first parameter, then another function is applied to the 2nd parameter.
>let x3 x y z = x*y*z 

>x3 2 3 4

24

The above function takes 3 parameters and multiplies them together. This is just repeated application of the * function, right? I could have said (x*y)*z.

This is exactly what currying is doing. Internally I look at it like this

func1 = x3 (x)

func2 = func1(y)

func3 = func2(z)

The function is applied to one argument, the applied function is then called on the 2nd argument and so on.

This can be used with partial functions to create simple and intuitive data flow mechanisms.

Wednesday, July 4, 2012

haskell primitives

Haskell the lazy purely functional programming

1. The basic primitives
This is the greatest

Matlab Parfor

In the parallelization sphere of things, matlab is pretty up-to the mark. There are a lot of ways to parallelize in matlab.

The easiest is to create a matlabpool of workers and run you for loops in parallel using parfor. (the upper limit is 12 on a local machine) This is great if you using this on a local machine to prototype some application, but if you do want to deploy your application (compile and make an executable); you run into problems.




Friday, June 29, 2012

Matlab Profiling

Matlab comes built in with a profiler. It is much akin to the profiler in valgrind, only that in true matlab fashion it generates a html report of the profiler.

Profiling can be used to look at specfic runs of functions and the call tree, it generates statistics like the number of times a function was called and how much time a function takes to execute.

This kind of information can provide key insight into blockages, especially on the IO front. For example, in C++ it is better to use "\n" than endl to mark end of lines.endl flushes the buffer and adds consdierable overhead. In matlab, you don't figure out why some functions are slow, you can use profiling to pinpoint the exact line causing the bottleneck. This can be crucial especially if you deal with big datasets and/or heavy computation.

So, if you now have code which you want to profile and say save the stats and measure, simply run the following commands and it will be done.


Saturday, March 31, 2012

Reverse engineering the ising model

I was trying to solve this problem in image segmentation. The key idea in segmentation is to group pixels which seem spatially similar owing to properties of color, region and body. There is no one single segmentation which is correct, it is a matter of perception. Perception of images varies by individual; what I perceive in an image, you may not. Its like looking at monet or chagall, my opinions of masterpiece may be lost on you.

So, starting with this how does end up with the ising model. One starts by applying a markov random field to the image and running a max-flow on the image graph to get the segmentation. Standard fare combinatorial optimization problems some say.

My introduction to markov process is from a NLP standpoint, pretty informally again (understanding the process, and not reading a whole bunch of literature). I understood the conditional inference scheme depended only on the previous state(s) and not all the past states. With this understanding I embarked to solve this problem (maybe I needed better accoutrements, but this too shall pass).

How does one group anything? A general consensus is to put all things similar in a heap and call it a group.  The question then arises, "how do you define similarity in images?". This led me down the path of wave particle duality, although we only see images as composed of particles (aka pixels). It took me a little while to understand what I was solving was not a computer science question but a physics question. I imagined there should be a force applied on the pixels to conform them to some shape, in essence changing their basic property (read color intensity value). If you think of a child grouping his toys into jedi and dark forces for a epic battle, he pushes all the dark ones into a group. The act of "pushing" is a force, which is the same concept for the image. Aha, but there is catch, you cannot move the pixels spatially like the little boy. A little confusing I agree.

So then, the story thus far looks good but we still don't have a solution. Now I can use the markov knowledge from a previous class. Imagine a bunch of patches or markers hanging in the air above the ground, and you image is painted for you by monet . For simplicity let's consider 2 patches, white and black. If we have a background and foreground in an image, lets say the white patches are attracted to the background and black patches to the foreground. Now if we can define an attraction force between the pixel and the patch we are home scott free. Herein comes the spatial arrangement problem, pixels close to each other should have the same patches. (I agree with this notion in a general sense, but not completely, occlusion and a few other factors like spatial incoherence, which can be seen in illusions make me a little uncomfortable with this generalization).

Steaming ahead with this idea, we now have 2 forces, one force between the patch (in the air) and the pixel in the image on the ground and another force between the pixels on the ground making them stick close together (I also think of a force between the patches and regions, making up a complete field but that too shall pass for now).

With this scheme of things, we can define a function to to either minimize or maximize over all the variables we have. The norm is to choose a energy function here and the general idea is to have a energy minimization or maximization,

E(x) = Ed + Es.

Here Ed is the data term and Es the smoothing term. The data term makes makes up some energy but only the pixel data and no neighborhood function. The smoothness term uses the neighborhood function and ensures we get the same labeling for pixels within the same region.

You can read more about this scheme here - http://www.cis.upenn.edu/~jshi/GraphTutorial/

So now that we have that down, we need equate the force to energy. If we assume a gravitational force, it will be attraction and we will have higher force for correct labelling and we can maximize the energy function to get a stable labeling of the pixels. On the other hand, if we consider a magnetic field, where like particles repel and unlike don't. We need to minimize the function to get a correct labeling of the images, which is the accepted norm in vision.

The concepts here are grossly oversimplified to help me understand the workings and make a intuitive sense of the process underlying the markov field. As with everything, the disclaimer, I do not know everything and I can get things wrong.