CFA Level 3 MPS by Linear Discriminant Analysis – 2013
This posting is for my fellow CFA (Chartered Financial Analyst) level 3 exam candidates. I am a a bio-statistician with a strong interest in finance. I have cleared the first two levels and like most level 3 candidates concerned about passing the highly competitive third and final level. The third level results will be released on August 12th 2014.
There have been many threads posted in the Analyst Forum on the MPS (Minimum Passing Score), a statistic which is not released to the public by the CFA administrators. I thought I would try a more rigorous statistical methodology to assess the MPS than those at the Analyst Forum threads. One of the analysts had kindly compiled a list of scores in the morning essay exam and afternoon exam for 91 2013 candidates who had posted their scores at the forum. The 91 scores were those of 80 passing candidates and 11 failing candidates. So the sample was quite limited, especially for the failing candidates and is also subject to possible differences in a cohort that chooses to report their scores.
Further the linear discriminant analysis technique used here assumes equal covariance matrices in the pass and fail groups, an assumption which is not testable for small cohorts. The linear method, as opposed to the quadratic method, is easier to implement and display graphically.
The graphic below the calculator provides a scatter plot of the pass and fail scores in the 2013 CFA level 3 exam as well as a yellow line which attempts to separate the passing and failing groups. The green dots in the scatter plot are the 80 passing scores and the 11 red dots in the graphic are the failing scores. The green triangle is the mean passing score and red triangle is the mean failing score. The blue triangle is the author’s mean score in the level 3 practice tests. Likely to be biased upwards as the scores (unlike the 91 2013 scores) are actual scores and not based on the discrete 80/60/40 mapping that needs to be done for the CFA results. Further the scores were based on subjective grading of AM practice tests and could again be biased upwards. So, all in all, I am not as confident of passing as the location of the score on the graphic would suggest. The yellow line is obtained by the linear discriminant analysis. The precise formula for it is
You have a greater probability of passing than failing if
0.154*(PM Score)+0.115*(AM Score) – 16.5 >= 0
and you have a greater probability of failing than passing if
0.154*(PM Score)+0.115*(AM Score) – 16.5 < 0
This is what I call a centered discriminant score as the value of 0 on this determines estimated group membership. The calculator below provides the centered discriminant score, the estimated likely outcome and the percentile score based on your AM and PM scores. The second tab in the spreadsheet provides the raw data on the 91 2013 level 3 scores. 49% of the 2013 level 3 candidates passed. So a percentile above 50 or so estimates success. A candidate expecting a score of 55 in AM and 70 in PM is estimated as likely to pass and he is estimated to be about a 66.3 percentile among those tested. All this presumes the distributions of scores in future exams are like those in 2013 in addition to all other assumptions made. Note that the inputs need to be your estimated 40/60/80 score – so the scores need to be between 40 and 80. Hope you find this interesting and entertaining. Good Luck!
Edit the blue cells in the spreadsheet and enter your data and the calculations will refresh.
From the graphic, we see that the discriminant analysis incorrectly classified 3 of the 2013 fails as a pass (one was on the line) and incorrectly classified 9 of the 2013 pass’s as fail (two were on the line).
A more accurate separation would have occurred if I had ethics scores and could fit a 3 factor model of the form
(Coefficient_PM)*(PM Score)+(Coefficient_AM)*(AM Score)+(Coefficient_E)*(Ethics)
Post results update
I passed level 3 by the skin of my teeth! My AM essay 80/60/40 score was 66% and my PM score was 58%. By my own calculator I am about 53 percentile among all candidates. 54% of all level 3 candidates passed and I might have flunked if the pass rate was only 47%. But I think I will brag about it to the extent permitted by the CFA program:
“I passed all three levels of the CFA program in consecutive attempts and could qualify for the Chartered Financial Analyst Charter if and when I complete 4 years of relevant experience.”
The graphic is now updated with a black triangle which is the authors actual score on the exam.