Why i get negative values of eigenvalues for covariance matrix

[Adeline]

I suppose to collect Array of Covariance Matrix. However, the last few values of eigenvalues I get negative values.

Can anyone who experience in covariance matrix give me some suggestion on what's wrong with my code?

The code is has no error.

IplImage**  ArrCov (IplImage* in, int patch_size, CvRect window,IplImage Gab0,IplImage Gab85,IplImage Gab180,IplImage Gab265  ){

int block_start_x, block_start_y;
int block_width = patch_size, block_height = patch_size;
int startc = 0;

CvSize sz = cvGetSize( in );
IplImage **ArrayCov;  //Declare Covariance matrix array
ArrayCov = (IplImage**)malloc(25 * sizeof(IplImage *));
for (int i = 0; i < 25 ; i++) {
ArrayCov[i] = cvCreateImage(cvSize(12,12), IPL_DEPTH_32F,1);
cvSetZero(ArrayCov[i]);
}



IplImage* avgImage = cvCreateImage( sz,IPL_DEPTH_32F, 1);  //Declare average image 
IplImage* SingleF= cvCreateImage( sz, in->depth, 1); 
//cvZero(SingleF);


IplImage **ARCO = Feature(in,Gab0,Gab85,Gab180, Gab265); //get total Features 
//showImage(ARCO[11], 0, "bubu");



for (block_start_y = window.y; block_start_y <= window.y + window.height - block_height; block_start_y += (block_height/2))
{
for (block_start_x = window.x; block_start_x<= window.x + window.width - block_width; block_start_x += (block_width/2))
{
	
    IplImage **temp;
	temp = (IplImage**)malloc(12 * sizeof(IplImage *));
	

for(int ArC=0; ArC<12; ArC++){

    /* get properties, needed to create ROI image */
	
    cvZero(SingleF);
	SingleF= cvCloneImage(ARCO[ArC]);
	cvSetImageROI( SingleF , cvRect(block_start_x, block_start_y, block_width, block_height));
	IplImage* tempINT = cvCreateImage(cvGetSize(SingleF),SingleF->depth,SingleF->nChannels);
	temp[ArC] = cvCreateImage(cvGetSize(SingleF), IPL_DEPTH_32F,1);
	//temp[ArC] = cvCreateImage(cvGetSize(SingleF),SingleF->depth,SingleF->nChannels);
	//cvCopy(SingleF, temp[ArC], NULL);
	cvCopy(SingleF, tempINT, NULL);
	cvConvertScale(tempINT,temp[ArC], 1, 0);
//	if(ArC==9)
//	showImage(temp[ArC], 0, "bubu");
	cvReleaseImage(&tempINT);
	cvResetImageROI(SingleF);
	
	}

	//cvCalcCovarMatrix((const CvArr**)temp, 12, ArrayCov[startc], avgImage, CV_COVAR_NORMAL); //calculate covmatrix
    cvCalcCovarMatrix((const CvArr**)temp, 12,ArrayCov[startc], avgImage, CV_COVAR_SCRAMBLED+CV_COVAR_SCALE); //calculate covmatrix
	cvReleaseImage(temp);
	cvZero(avgImage);
     startc++;
}
}
cvReleaseImage(&avgImage);
cvReleaseImage(&in);
cvReleaseImage(ARCO);
cvReleaseImage(&SingleF);


return (ArrayCov);
}

[nathan10]

Hi Adeline,

I am not a professional programmer and am teaching myself C++. Your code is too complex for me to follow.

However, my field uses math and I have limited experience with eigenvectors. Though I needed to dust off my old linear algebra text, I confirmed that eigenvalues can be either positive or negative. In my field, the negative eigenvalues are ignored because they have no real world interpretation or application.

What I am saying is that there may be NO problem with your code.

That said, if the real world issue you need to solve precludes negative eigenvalues then there may be a problem with the design of your program.

Have you written pseudocode or drawn flowcharts in which you might be able to follow the computation and identify the circumstances that generate negative eigenvalues? If so then use it to find your 'error'. If not, then make them.

[mik2718]

I think the eigenvalues should all be positive since a covariance matrix is symmetric with positive entries.
(also the eigenvectors are orthogonal)

What method are you using? I have always used Jacobi transformations. In fact the code I use is lifted from numerical recipes in C which can be found here

http://apps.nrbook.com/c/index.html

Look under Jacobi transformations of a symmetric matrix

[nathan10]

mik2718 wrote:

I think the eigenvalues should all be positive since a covariance matrix is symmetric with positive entries.

Only the diagonal entries in a covariance matrix must be positive. The off-diagonal entries can be positive, negative, or zero.

However, I don't know (or think) that a negative covariance will necessarily cause a negative eigenvalue.

[mik2718]

Sorry, you are correct.
My reading on the eigenvalues suggests that covariance matrices should be positive semidefinite. i.e. eigenvalues are positive or zero.

[Adeline]

I have check my input images which used to compute covariance matrix, their individual result shows no problem.

In opencv, they are a function to calculate covariance matrix:

http://books.google.co.jp/books?id=seAgiOfu2EIC&pg=PA54&lpg=PA54&dq=covariance+matrix+opencv&source=bl&ots=hSI35leBNf&sig=OxTSueeNk59euUXZ7d91CWoX4uE&hl=en&sa=X&ei=3xqmUOejMaSPmQXI-IDQCQ&ved=0CGwQ6AEwCQ#v=onepage&q=covariance%20matrix%20opencv&f=false

I used this function and CV_COVAR_SCRAMBLED+CV_COVAR_SCALE to get the image_sample * image_sample dimension of covariance matrix. However, I am curious that I get the negative values for the smallest eigenvalues.

[Adeline]

However, one thing that I don't understand is:

http://docs.opencv.org/modules/core/doc/operations_on_arrays.html#calccovarmatrix

mean – input or output (depending on the flags) array as the average value of the input vectors.

So, is it means that if I am using 'CV_COVAR_SCRAMBLED' flag, I need to make mean the input instead of output?

[mik2718]

My reading of the docs is that the mean is an input variable only if
CV_COVAR_USE_AVG is set

In your code I think you should have
CV_COVAR_SCRAMBLED|CV_COVAR_SCALE (bitwise or)
instead of plus. These look like a bitset of flags to me.

Are these negative values small in magnitude (compared to the positive ones)

I can't see where in the code the eigenvalues are calculated

are you using cvEigenVV to calculate them?

http://www.cs.iit.edu/~agam/cs512/lect-notes/opencv-intro/opencv-intro.html#SECTION00063000000000000000

[Adeline]

mik2718, yes, I am using cvEigenVV to calculate. Below is my code:

IplImage** AllCov =  ArrCov(img, patch_size,window, Gab0,Gab85,Gab180,Gab265 );
ofstream outdata; // outdata is like cin
outdata.open("testgraph.txt"); // opens the file
 if( !outdata ) { // file couldn't be opened
      cerr << "Error: file could not be opened" << endl;
      exit(1);
   }

	 
CvMat* EigenV  = cvCreateMat(25,12,CV_32FC1);
for (int Ch= 0; Ch<25;Ch++){



CvMat* E  = cvCreateMat(12,12,CV_32FC1);
CvMat* l  = cvCreateMat(12,1,CV_32FC1);
cvEigenVV(AllCov[Ch], E, l);

cout<<cvmGet(l,0,0)<<endl;
 outdata << cvmGet(l,0,0)<<endl;
cout<<cvmGet(l,1,0)<<endl;
 outdata << cvmGet(l,1,0)<<endl;
cout<<cvmGet(l,2,0)<<endl;
 outdata << cvmGet(l,2,0)<<endl;
cout<<cvmGet(l,3,0)<<endl;
 outdata << cvmGet(l,3,0)<<endl;
cout<<cvmGet(l,4,0)<<endl;
 outdata << cvmGet(l,4,0)<<endl;
cout<<cvmGet(l,5,0)<<endl;
 outdata << cvmGet(l,5,0)<<endl;
cout<<cvmGet(l,6,0)<<endl;
 outdata << cvmGet(l,6,0)<<endl;
cout<<cvmGet(l,7,0)<<endl;
 outdata << cvmGet(l,7,0)<<endl;
cout<<cvmGet(l,8,0)<<endl;
 outdata << cvmGet(l,8,0)<<endl;
cout<<cvmGet(l,9,0)<<endl;
 outdata << cvmGet(l,9,0)<<endl;
cout<<cvmGet(l,10,0)<<endl;
 outdata << cvmGet(l,10,0)<<endl;
cout<<cvmGet(l,11,0)<<endl;
 outdata << cvmGet(l,11,0)<<endl;
}

And, yes, is CV_COVAR_SCRAMBLED|CV_COVAR_SCALE.
However, after i changed it, the result of eigenvalues stilll show negative value.

These are the result of 12 eigenvalues for 25 Array of covariance matrics with dimension of 12*12 :

434941
476.05
13.0408
0.229729
0.165433
0.0475896
0.0149984
0.0052374
0.00138277
0.0002789
-7.8337e-005
-0.000732685




425351
168.372
12.0928
0.555679
0.232708
0.0415843
0.0219215
0.0109292
0.000686594
0.000145359
-0.000149034
-0.00550065




412835
14.2377
3.27641
0.423957
0.113699
0.0717227
0.00970036
0.00706829
0.00244494
0.000125004
-0.000146919
-0.00641104




405901
55.1256
18.2479
0.39818
0.0936296
0.0103858
0.0082194
0.00330535
0.00101555
0.000240717
-3.4435e-005
-0.00446303




423553
395.481
19.3271
0.381515
0.127133
0.0109391
0.00778359
0.00433625
0.00137635
0.000514039
0.000201521
6.5035e-005




406594
253.512
18.0674
1.12895
0.404416
0.205142
0.0509044
0.00160985
0.000886909
0.000166033
0.000136379
-0.00218203




421015
71.9299
13.9852
1.66826
0.494467
0.260163
0.0413086
0.00230316
0.00131747
0.000556026
6.03594e-005
-0.00542986




398189
14.4084
3.28126
0.899483
0.392052
0.115644
0.0648547
0.00529257
0.00243065
0.00136099
0.00053307
6.19668e-005




381595
29.3141
14.0963
0.259062
0.178134
0.0889774
0.0393558
0.00612524
0.00174532
0.0007768
0.000360144
0.000134801




403684
220.271
18.6915
0.154635
0.0724717
0.0300317
0.00756427
0.00472055
0.000912798
0.000530488
0.000178738
-2.8791e-007




320101
111.354
20.0723
2.26008
0.837895
0.276076
0.106024
0.00485219
0.00321501
0.00127061
-0.0001065
-0.000299802




355022
26.3255
8.42884
1.97657
0.397327
0.229303
0.152046
0.00318979
0.00104527
0.000574519
1.61162e-005
-0.00292458




365020
14.1555
4.11375
1.41778
0.566796
0.239538
0.103347
0.00258068
0.00205743
0.00109074
-0.000263602
-0.000367105




324397
34.6255
10.7816
1.65484
0.486502
0.268105
0.078191
0.00581639
0.00256835
0.000597202
0.000102063
-0.000161507




327339
220.296
35.5536
0.314017
0.125374
0.123498
0.0544792
0.00208204
0.00101765
0.000396772
-0.000218469
-0.000423893




285562
69.0832
2.7687
2.32759
1.08951
0.482043
0.129677
0.00369158
0.00173254
0.00128481
0.000192729
-0.000144794




309886
12.4452
6.47967
2.76693
0.497366
0.340031
0.122041
0.00529095
0.00378099
0.000295806
-0.000208341
-0.00416839




339965
20.94
2.63587
1.83145
0.479637
0.213125
0.0804762
0.0047008
0.00148618
0.000282981
5.31619e-005
-0.00450524




321481
46.4337
7.07921
2.37857
0.382175
0.309306
0.0524365
0.00406964
0.00272736
0.00134508
-0.000239855
-0.00147243




311869
217.59
29.2934
0.425372
0.110061
0.0902323
0.071882
0.00396984
0.00254475
0.000557988
4.63215e-005
-0.000112443




361451
17.913
5.02295
0.844934
0.511684
0.428535
0.0625116
0.0185659
0.00149073
0.00051608
0.000412829
-0.00435842




356800
7.84342
4.38512
1.88364
0.845922
0.204255
0.115104
0.0064319
0.0027864
0.000328387
-0.000332731
-0.00177415




370989
16.0856
2.15688
1.19757
0.56437
0.222107
0.171882
0.00557853
0.00317736
0.000151364
-0.000196121
-0.00235805




376488
42.6005
3.94158
1.11649
0.334485
0.219659
0.1274
0.00573928
0.00351351
0.000787268
-0.000337997
-0.000437277




360386
173.666
16.104
0.228053
0.141407
0.0628892
0.0234155
0.0042185
0.00260964
0.000489347
-9.98956e-005
-0.000252862

[mik2718]

I don't have much experience of what you are doing here but I do notice that the difference in magnitude of the most negative eigenvalue and the largest positive one is about 10^8

You may just have to set these negative eigenvalues to 0 for whatever further work you are doing

Its possible these errors are just a fact of life, or maybe they arise from limited precision used in calculating the covariance matrices?