I have a problem with the graphical output of a ROC curve I am calculating.

I am sure it is very silly but I cannot figure out where I am going wrong.

I estimate the probability of damage (damage levels 1 to 5) to a number of buildings given one continuous explanatory variable taking n values (Xi), using ordinal regression - prop odds.

I will call my cumulative fitted probabilities: cyfit, and my actual cumulative probabilities: pDS.

Using Matlab, I want to assess the goodness-of-fit of my model by plotting the ROC curves (one per damage level),

I calculate the number of true positives TP as follows:

TP=pDS (if cyfit-pDS > = 0);

otherwise TP=cyfit

similarly, for false positives FP

FP=cyfit -pDS (if cyfit-pDS > = 0);

otherwise FP=0

Then I sum FP and TP over number of values of X by doing

TotFP=sum(FP) : i=1 to n,

TotTP=sum(TP) : i=1 to n,

Then calculate rate

RateFP=FP / TotFP

RateTP=TP / TotTP

Finally I calculate the cumulative rates

cumulFP(i)=cumulFP(i-1)+RateFP(i)

cumulTP(i)=cumulTP(i-1)+RateTP(i)

So this is where I have an issue, I plot cumulFP on the x-axis and cumulTP on the y-axis and end up with ROC curves way below the random guess line, basically telling me that my fir is really bad,

when really when I look at my fitted model my curves seem to give a relatively decent fit...

(see attached files)

I have loads of data and I have this problem with every single case, so I think there must be some issue with my approach in plotting the ROC curve.

If anyone has an idea of where I am going wrong, please let me know...

Thanks!