Financial Model Mistakes Can Cost Millions of Dollars

 

It's been awhile since I've blogged. Work has been busy and then I took off ten days to visit Paris. The City of Lights is amazing indeed. Now that I'm back, I will try to blog more frequently. There is certainly no shortage of topics about risk, governance, litigation, valuation and so on.

For those who don't know, I created a sister blog a few months ago. See GoodRiskGovernancePays.com. Nearly all of the time, the posts on each blog are different. However, I decided to reprint a post from GoodRiskGovernancePays.com here since the topic is hugely important. After all, for those defined benefit and defined contribution plans that are exposed to "hard-to-value" investments, leverage and perhaps higher than expected volatility, model risk could be the hidden alligator that bites if left unchecked. As always, I welcome your comments at contact@fiduciaryleadership.com.

Here is the post that was originally posted on June 2, 2011 by Dr. Susan Mangiero.

In a recently published article about financial models entitled "Financial Model Mistakes Can Cost Millions of Dollars" (American Bar Association, Section of Litigation, Expert Witnesses, May 31, 2011), Dr. Susan Mangiero defines model risk and explains why it is so important. Referencing the recent $242 million enforcement action by the U.S. Securities and Exchange Commission as a result of model mistakes made by a well-known asset management firm, this financial expert cites the heightened regulatory and litigation imperatives with respect to risk and valuation models. She concludes the article by listing some of the ways to mitigate risk. These include, but are not limited to the following:

  • Hire knowledgeable programmers with capital market experience;
  • Create and follow a set of policies and procedures that govern how and who will validate financial models over time and what will trigger revisions in a model(s);
  • Avoid conflicts of interest that would reward managers for ignoring problems and would potentially preclude an independent and objective assessment of problems and related corrective action(s);
  • Test assumptions for validity in stable markets as well as extreme circumstances;
  • Stress a model using a sufficient number of economic scenarios to gauge its predictive power and whether results can be relied upon in both good or bad times;
  • Educate personnel about how a particular model is supposed to work;
  • Establish a response strategy should a problem occur and investors need to be informed before things get out hand;
  • Scrap models that are overly complex and expensive to replicate;
  • Don't be afraid to ask questions about inputs, data quality, results, and concerns; and
  • Invite informed outsiders to offer an independent and regular critique on a confidential basis.

 

Public Pension Risk Management and Fiduciary Liability

A few weeks ago, Attorney Terren B. Magid and Dr. Susan Mangiero jointly presented on the topic of pension risk management and fiduciary liability with a particular emphasis on public plans. Attorney Magid's insights reflect a particularly unique perspective inasmuch as he served as executive director of the $17 billion Indiana Public Employees' Retirement Fund ("PERF"). Dr. Mangiero shares her views as an independent risk management and valuation consultant, author, trainer and expert witness.

Click to download the 25-page webinar transcript for public pension fiduciaries entitled "Are You Properly Mitigating Risk? Assess Your Fiduciary IQ" with Attorney Terren B. Magid (Bingham McHale LLP) and Dr. Susan Mangiero (Fiduciary Leadership, LLC). Comments about ERISA plans are provided when applicable.

Topics discussed include, but are not limited, to the following:

  • Public Pension Transparency Act
  • Discount Rate Choice
  • Dodd-Frank Wall Street Reform and Municipal Advisor Registration
  • Expanded Definition of ERISA Fiduciary
  • Fee Disclosure Under ERISA 408(b)(2)
  • Failure to Pay and Actuarially Required Contribution ("ARC")
  • Benefit Reductions
  • RFP Process
  • Fiduciary Audits
  • D&O Policy Review
  • Vendor Contract Examination
  • Qualitative and Quantitative "Investment Risk Alphabet Soup"
  • Interrelated Risk Factors
  • Key Person Risk
  • Hard to Value Investing
  • Model Risk
  • Stress Testing
  • Pension Litigation
  • Fiduciary Breach Vulnerability
  • Characteristics of a Good Model
  • Side Pockets and Investment Performance.

Comments are welcome.

Model Risk Costs One Asset Manager $242 Million

According to a February 3, 2011 document released by the U.S. Securities and Exchange Commission ("SEC"), AXA Rosenberg Group ("AXA") and various related entities have settled a matter relating to model risk for $242 million in economic damages and penalties. In a company letter dated April 15, 2010, several AXA executives describe an error with an investment model as having been "discovered in late June 2009" and "corrected between September and mid-November." While there are issues about the disclosure of said problems, the official message to investors at that time was a continued commitment to risk management.

Investor fallout has occurred nevertheless with various press accounts reporting the withdrawal of monies from AXA by pension plans such as the School Employees' Retirement System of Ohio, Los Angeles Fire and Police Pensions, the City of Fresno Retirement System, Florida State Board of Administration, the Marin County Employees' Retirement Association and the Montana Board of Investments. 

As I've written before, valuation (and by extension, model risk assessment) is a key element of the due diligence of asset managers. For a list of some of the "must ask" questions about model reviews, click to read "Asset Valuation: Not a Trivial Pursuit" by Susan Mangiero, PhD, CFA, FRM (FSA Times, The Institute of Internal Auditors, Q1-2004).

Email contact@fiduciaryleadership.com if you want to further discuss model review best practices.

Pension Fiduciaries - Have You Asked Your Bankers About Their Risk Controls?

In a November 5, 2007 statement, Citi announced that current CEO Charles Prince will step down. Robert E. Rubin will become Chairman of the Board and a search for a new leader will begin immediately. In a related story, Wall Street Journal reporters Carrick Mollenkamp and David Reilly describe Citigroup's struggles to estimate trading losses, in large part due to the fact that internal quantitative models relied heavily on credit ratings assigned to securities that were used in structuring Collateralized Debt Obligations (CDOs). With recent downgrades (to arguably better reflect risk levels), the value of Citi's sub-prime holdings similarly sank. Credit fears dampened already limited trading interest, forcing a heavy reliance on a mark-to-model approach.

As this blog's author has stated many times before, model risk is real. Bad or inappropriately used models lead to imprecise outputs. Decisions based on poor information can only lead to trouble. According to "Why Citi Struggles to Tally Losses Swelling Write-Downs Show Just How Fallible Pricing Models Can Be" (Wall Street Journal, November 5, 2007), modelers projected future expected payments for then high-rated sub-prime backed CDOs on the basis of how similar credit rated corporate bonds were trading. By not recognizing that default experience for corporate versus sub-primed backed securities differed dramatically, Citi's rocket scientists painted too rosy a valuation picture.

In a related article ("Where Did the Buck Stop at Merrill? November 4, 2007), New York Times reporters Graham Bowley and Jenny Anderson describe oversight problems at Merrill Lynch. Following a $8.4 billion charge and the recent resignation of CEO O'Neal, questions have arisen about whether board members should be more aware of daily operations, especially those areas that are likely to present problems if things go awry. Quoting Meredith Whitney, CIBC World Markets financial analyst, the point is made that Merrill had no one with sub-prime experience to serve on any of the committees charged with risk oversight and auditing. Despite creating a post for Chief Risk Officer in early September 2007, other experts cited in the article decry the lack of board/oversight committee independence from senior management, at the same time that large trading books were "hard to value."

By extension, this notion of oversight applies to pension fiduciaries. As this blog's author has repeatedly emphasized, plan sponsors MUST do a thorough job of vetting service providers (including banks) with respect to their "red flag" controls. How many pension fiduciaries ask about the existence of a Chief Risk Officer (or lack thereof)? How much detail do pension fiduciaries demand to know about each bank's risk management function, certainly for key parts of trading operations? Do pension fiduciaries ask to speak to members of the valuation team and/or those responsible for collateral management? Have pension fiduciaries asked banks about their progress with respect to preparing for Basel II and related model requirements? (Click here to read the November 2, 2007 press release from the Federal Reserve Board which describes their approval of new risk-based capital rules.) The list of other "must know" queries is long but nevertheless essential to proper due diligence.

Will clever attorneys make a case for poor process if pension fiduciaries have allocated monies to any or all of the banks now making headlines, citing breach if they failed to dig deep about risk and valuation policies and procedures?