CFA Level 3 MPS by Linear Discriminant Analysis
This posting is for CFA (Chartered Financial Analyst) level 3 exam candidates and those with results who want to estimate their relative standing among candidates . I am a bio-statistician with a strong interest in finance. In August 2014 I cleared all three levels. Like most level 3 candidates I was concerned about passing the highly competitive third and final level. I did a calculator to assess my chances of passing based on available 2013 scores. That is available at this page. Details on the updated calculator and scatter plot of passing and failing scores on the pooled 2013 and 2014 level 3 scores are provided below.
There have been many threads posted in the Analyst Forum on the MPS (Minimum Passing Score), a statistic which is not released to the public by the CFA administrators. I thought I would try a more rigorous statistical methodology to assess the MPS than those at the Analyst Forum threads. One of the analysts had kindly compiled a list of scores in the morning essay exam and afternoon exam for 91 2013 candidates who had posted their scores at the forum. The 91 scores were those of 80 passing candidates and 11 failing candidates. I also obtained the 74 scores from the results thread for 2014 which had 67 passing scores and 7 failing scores. So the sample was quite limited, especially for the failing candidates and is also subject to possible differences in a cohort that chooses to report their scores. However this is an improvement on the previous calculator — together we have 18 failing scores and 154 passing scores.
Further the linear discriminant analysis technique used here assumes equal covariance matrices in the pass and fail groups, an assumption which is not testable for small cohorts. The linear method, as opposed to the quadratic method, is easier to implement and display graphically.
The graphic below the calculator provides a scatter plot of the pass and fail scores in the CFA level 3 exam as well as a yellow line which attempts to separate the passing and failing groups. The green dots in the scatter plot are the 80 2013 passing scores, the green crosses are the 67 2014 passing scores, the 11 red dots in the graphic are the 2013 failing scores and the 7 red crosses are 2014 failing scores. The green triangle is the mean passing score and red triangle is the mean failing score. I apologize for the red, yellow and green traffic light coding – just using it to make it easier to read. I am sure there are a number of people who have passed (green – go) who are not really going anywhere and people who have failed (red-stop) who are unstoppable in many ways. The blue triangle is the author’s mean score in seven level 3 practice tests. The yellow line is obtained by the linear discriminant analysis. The precise formula for it is
You have a greater probability of passing than failing if
=0.563(AM Score) +0.827*(PM Score) – 86.84 >= 0
This is what I call a centered discriminant score as the value of 0 on this determines estimated group membership. As can be seen from the graphic there is considerable variability and overlap in the two groups. So it is healthy to consider confidence intervals on the discriminating line. The green line is the upper one sided 90% confidence interval (same as 2 sided 80% upper confidence interval). The red line is the lower two sided 80% confidence interval. The region between the red and the green lines is a region of ambiguity — you might end up in this region and either pass or fail due to the ethics adjustment or due to low/high scores within the broad intervals that the CFA divulges. The “Confidence in Passing” measure in the calculator below is the actual one-sided percent confidence interval line that passes through your input scores. The calculator also provides the centered discriminant score, the estimated likely outcome and the percentile score based on your input AM and PM scores. I assume a 50% probability of passing. You can check your estimated percentile if you expect a different percentage to pass your exam. In addition in the middle of calculator is the comparable estimated MPS combination. This is the score you get when you drop a perpendicular onto the yellow line from from your input AM and PM scores and tends to be roughly proportional to your input scores. This provides the estimated combination of AM and PM scores to beat given your tendency to score better in PM instead of AM or vice-versa. The second tab in the spreadsheet provides the raw data on the 2013 and 2014 level 3 scores. A candidate expecting a score of 60 in AM and 70 in PM is estimated as likely to pass and he is estimated to be about a 66.5 percentile among those tested with a 83.07% confidence in passing. The relevant estimated MPS for this candidate is 57.31 on the AM and 66.05 on the PM. All this presumes the distributions of scores in future exams are like those in 2013 and 2014 in addition to all other assumptions made. The inputs need to be your average scores on practice tests or on CFA mocks and estimated 40/60/80 score if it is an actual result. So we do assume that actual mock test scores will be close to 40/60/80 scores. The scores I used to generate the scatter plot and the calculator were 40/60/80 scores – hence between 40 and 80. Hope you find this helpful when you are doing your practice tests, waiting for your results or gauging your eventual CFA level 3 result. Good Luck!
Edit the blue cells in the spreadsheet and enter your data and the calculations will refresh.
I passed the 2014 level 3 by the skin of my teeth! My AM essay 80/60/40 score was 66% and my PM score was 58%. A little below the yellow line. My ethics score wasn’t very high either — so in all likelihood I scored high within the three interval grades the CFA institute provides. Despite the close call I think I will brag about it to the extent permitted by the CFA program:
“I passed all three levels of the CFA program in consecutive attempts and could qualify for the Chartered Financial Analyst Charter if and when I complete 4 years of relevant experience.”
The black triangle is my actual score on the exam.