Machine Learning Problem

Hi there, I have an optimization problem and I could really use your help.

The problem is related to computer vision and machine learning and it consists in identity matching.

I have a person in a video which I represent by a set of features, like color, position in the video, compactness etc. I can follow this person in the video for a while, but then the person interacts with another person and it is impossible to distinguish which person is which. At a later point in time, the two people part ways and I am able to distinguish again the individuals and extract the new set of features for both of them. Now I want to establish the pervious identities of the newly detected people. For this I implement a Bayesian Network and I use the Bayes probability to determine the similarity between the 'old' and 'new' persons.

Time T T+1 T+2

Person A X

O

Person B Y

Problem X = A, Y = B or X = B, Y = A?

My probability is: P(X = A| NewObservations) = ( P(NewObservations|X = A) * P(X = A) ) / P(NewObservations)

which reduces to: P(X = A| NewObservations) = P(NewObservations|X = A), as the priori probabilities are equal (1/2).

I compute the probability as a weighted product of the features probability:

P(NewObservations|X = A) = Weight_Height * P(Height_A| X = A) * Weight_Color * P(Color_x|X = A) * Weight_compactness * P(Compactness_x|X = A) *Weight_position * P(Position_x|X = A)

My problem is to determine the weights for each feature in order to obtain the best results.

For this I took a training set consisting of features of the same person at time T and T+5(frames). Then I tried to maximize the probability of P(X = A| NewObservations), knowing that I am looking at the same person. The constrains for the weights are: 0< weightX < 1 and weight1+weight2+weight3+weight4 = 1. I used the Nelder-Mean optimisation algorithm. After a couple of iterations (3-5) the algorithm simply gives me the highest weight allowed by the constrains (0.999999..) for the highest feature probability, while the other weights go to minimum (0.00..01). Am I missing anything? Is there another approach for this problem?