REFERENCE NO.: 96-CR-7

DATE: May 1, 1996

SUBJECT: Model Credit Program Methods and Documentation for Estimating Subsidy Rates and the Model Information Store

Over the past five years, credit agencies have worked to establish loan program subsidy estimates for the budget and for financial accounting. Agencies have had varying levels of success in constructing and documenting historical databases to support their subsidy estimates. As a result, some agencies have been unable to provide adequate assurances that the cash flow assumptions that underlie their subsidy estimates (such as default and recovery rates) are reasonable, and they have failed to receive unqualified audit opinions on their financial statements.

This paper outlines the structure of reasonable methods for subsidy estimation and the model database, or information store.¹ The methods build upon the standards set forth in Statement of Federal Financial Accounting Standards (SFFAS) No. 2, Accounting for Direct Loans and Loan Guarantees (August 23, 1993), paragraphs 33-36. Establishing a general structure is difficult, because of the unique characteristics and purpose of each Federal credit program. This paper sets out elements critical to subsidy rate estimation (at either the risk category or cohort level) and information stores. The first section outlines reasonable methods for subsidy rate estimation. The second section explains the intended use of information stores. The remaining sections outline the appropriate number of loans to include in an information store and the type of data elements which might be included. Any method of estimation requires that the data used in constructing an information store is reliable. A separate paper will address "reliability."

Given the diversity of Federal credit programs and, therefore, the wide range in data required to support subsidy estimates, an information store should be tailored to the data needs of each loan program. The data elements in an information store should be selected to allow for more in-depth analysis of the most significant subsidy estimate assumptions.

Section I: Reasonable Methods for Estimating Subsidy Rates

The passage of the Federal Credit Reform Act on November 5, 1990, presented tremendous challenges. OMB had to provide guidance to credit agencies on how to implement credit reform in the budget that was to be published only three months later. In a similar condensed time frame, agencies were required to implement this guidance and transform their financial systems to meet the new requirements. As a result of this tight time frame, early subsidy estimates were often based on limited historical data and simplifying assumptions. Over the past five years, estimation procedures have become more sophisticated, but estimates and methods still suffer from a lack of historical data to support cohort estimates.

Methods of estimating future cash flows for existing credit programs need to take account of past experience. SFFAS No. 2 states: "Actual historical experience of the performance of a risk category [or cohort] is a primary factor upon which estimation of default data is based. To document actual experience, a data base [or "information store"] should be maintained to provide historical information."² The estimation methods currently used by agencies fall along a continuum in terms of the degree to which they provide a basis for supporting the estimates. They can be divided broadly into three categories of which the first provides the most support and the third provides the least: 1) econometric modeling of key economic indicators from an information store to predict future cohort performance; 2) use of an information store to forecast performance based on "averages" of past performance; and 3) informed opinion.³ In many instances, a combination of the three methods is used. Often, documentation of key assumptions is missing whichever method is used. Documentation of all assumptions should be provided in order to make these assumptions transparent enough for an otherwise uninformed second party to reproduce the model results. Effort to collect and update an information store is needed to evaluate the model assumptions and to improve predictions.

Econometric modeling: The term "econometric modeling" is meant to include any estimated quantitative method of analysis. Econometric modeling is desirable for all cohorts that are significant to the agency's budget request. SFFAS No. 2 states that "each credit program should use a systematic methodology, such as an econometric model, to project default costs of each risk category [or cohort]."⁴ Econometric modeling should use the type of loan cash flow, economic, and loan characteristics data outlined in Sections IV-VI of this paper to be used to project cash flows. This approach creates an auditable procedure that formalizes and documents loan performance assumptions. In addition, it defines key relationships between loan performance and economic and other indicators. For example, if the interest rates that borrowers pay on their loans are a function of their incomes, a model that predicts changes in borrower income and corresponding interest rates paid by the borrower over the life of the cohort should be superior to informed opinion or simple averages from the past. Or if defaults are affected by the initial loan-to-value ratio, a model estimate can take account of any differences from past experience that are expected in the future due to a change in the percentage distribution of borrowers with different loan-to-value ratios. It can also take account of change in policy regarding loan-to-value rations; and it can be used in policy formulation to estimate how alternative changes in policy regarding variables such as the loan-to-value ratio would affect future cash flows and thereby the subsidy cost of the loans or guarantees. A documented econometric model can also be more readily observed and commented upon by others, and it can then be reestimated to take account of their comments. Finally, a documented model allows for easy transferability between analysts. If the agency's experienced staff leaves, the model and its key assumptions remain in place.

It should be understood that econometric modeling needs to use informed opinion -- people with knowledge of the program -- to develop the model in the first place; to make use of the data in the information store and other sources of information in estimating programmatic or economic variables that the model uses to predict cash flows; to test the model against experience; and to update it. The difference between the first method of estimation, econometric modeling, and the third, informed opinion, is thus not whether people with knowledge of the program are needed in making the, estimates. The difference is that in econometric modeling people with program knowledge are used in the construction and application of the econometric model, whereas the method of informed opinion uses such people to make a judgmental forecast.

An econometric model will not be sufficient if a program consists of one or a small number of loans, or if it has a few loans that are large relative to the size of the portfolio. It might or might not be possible to estimate an econometric model for such programs, but in either case the equation probably could not take into account events affecting individual borrowers and their ability to service their debt. Therefore, SFFAS No. 2 states: "If individual accounts with significant amounts carry a high weight in risk exposure, an analysis of the individual accounts is warranted in making the default cost estimate for that category."⁵

Historical averages: Existing data may not be sufficient to estimate econometric models. If an agency's information store contains historical cash flows, but no other data elements for use in econometric models, averages of historical cohort performance on a year-by-year basis (but not averages of the portfolio as a whole)⁶ may be used to predict future cohort behavior. Averages may also be used if the recorded cash flows do not extend over enough years to estimate a model. This should be used on an interim basis as additional data elements are added to the information store. For loan programs with stable loan terms and borrower characteristics, historical averages may be acceptable for projecting cash flows on more than an interim basis. Similar to econometric models, use of historical averages creates an auditable procedure that formalizes and documents loan performance assumptions.

 Informed opinion: If historical cohort information is not available, informed opinion is often used to estimate the subsidies. "Informed opinion" refers to the judgement of agency staff or others who make subsidy estimates based on their programmatic knowledge and experience without using an econometric model or a satisfactory information store.

Informed opinion is the least data intensive of the three general methods of estimation. Agencies that currently rely on informed opinion should strive to build an information store, as outlined below, in order to establish the basis for better estimates in the future -- initially by using historical averages and eventually by estimating econometric models.

A separate paper will address how estimates should be made and documented during the interim period while an information store is being developed.

Section II: Purpose of Information Stores

A loan program information store should provide three types of information. First, the information store should maintain key loan characteristics at the individual loan level. Second, it should track economic data which influence loan performance. Third, an information store should track historical cash flows on a loan-by-loan basis. These data will allow for econometric analyses of risk. This information will also ensure that the historical data remain useful, even as the loan program, borrower characteristics, and economic conditions change. The data on cash flows for individual loans should be aggregated to calculate the total cash flows for the cohort as a whole. An information store may also be needed for reasons other than subsidy estimation, such as servicing loans and tracking loan program performance measures.

Section 502(5) of the Federal Credit Reform Act (FCRA) requires that agencies estimate the subsidy cost to the Government of making direct loans or extending loan guarantees. "Cost" is defined as "the estimated long-term cost to the Government of a direct loan or loan guarantee, calculated on a net present value basis, excluding administrative costs and any incidental effects on governmental receipts or outlays." Cost includes all cash flows directly related to the loan, such as disbursements, fees, interest subsidies, repayments, prepayments, and payments for default claims.

Agencies are required to project the expected cash flows to and from the Government for each cohort of loans. Therefore, the purpose of information stores is to collect data which allows managers to predict the timing and amount of cash flows. Data not affecting cash flows, such as loan charge offs, are only useful for estimating subsidy cost to the extent that this information is predictive of actual cash flows (although they may be useful for other purposes of the information store).

Section III: Information Store Size

Analysis based on the entire population of loans is generally believed to be more accurate than analysis based on a sampling of loans. However, for programs that annually disburse or guarantee a large number of loans, such as some of those in the Departments of Agriculture, Education, and Veterans Affairs, a statistically reliable sampling may be preferred due to cost considerations. Properly selected random samples can provide data representative of the population of loans, reducing the cost of developing, maintaining, and analyzing the information store. The sampling procedure should be well documented and reviewed by the users and auditors before accepting any analysis based on the information store. The aggregate cash flows for the cohort as a whole would be calculated from the entire population of loans regardless of whether a sample is used to estimate an econometric model for projecting cash flows.

Section IV: General Data Elements in Information Store

A loan performance information store should maintain three types of data. First, information regarding loan characteristics should be maintained. Second, relevant economic data should be gathered. Third, data on the actual timing and amount of all cash flows related to each loan in the information store should be collected. It is important that the data collected accurately reflect actual terms of the loan agreements. For example, the loan maturity at time of approval may change by the time the loan is disbursed. The information store needs to track the maturity in the final contract terms.⁷

Loan characteristics: This information is critical for identifying the factors which are predictive of subsidy costs, such as default and recovery rates. Loan characteristics maintained in information stores will vary greatly from program to program. For example, while the value of collateral pledged may be highly predictive of recovery rates in one program, collateral may not be required in another program.

Outlined below are examples of loan characteristics which managers may choose to collect. This information should be drawn from internal information stores described in JFMIP Direct Loan System Requirements and Guaranteed Loan System Requirements, such as application information, loan information, and program criteria. Information stores external to the loan system may also provide useful data, such as credit information and external organization information.

• Loan number. Cash flows must be maintained at the individual loan level, even though analysis might often be done at the cohort level. Cash flows, such as receipts from property disposition, must be tracked back to the original loan via the loan number or other data element used to identify the original loan.

• Date of obligation. This information is necessary for reviewing historical cohort data, since cohorts are defined by year of obligation, and for relating loan behavior to other dated variables.

• Loan terms and conditions. An information store should maintain the actual loan terms, including maturity, interest rate, repayment schedule (including any grace periods), and up-front and/or annual fees. These data are critical for comparing actual payments to scheduled payments. These data are also important for measuring the relationship between default risk and loan terms and conditions. For example, low-interest loans may have a lower incidence of default. Since subsidy cash flows are discounted at the rate on Treasury securities of comparable maturity, loan maturity data must be collected. This information will help determine the appropriate discount rate for estimating the subsidy cost of the grouping of loans analyzed (at either the cohort level or a lower level of detail).

• Changes in loan terms and conditions. Loan terms and conditions may be changed due to (a) modifications as defined by credit reform or (b) workouts of troubled loans that are designed to improve the Government's returns or reduce its losses. This information is needed for the same reasons as information on the original terms and conditions. Any change in terms and conditions needs to be recorded in addition to the original terms and conditions, not in place of them. The original and revised terms and conditions are both part of the history of the original loan or guarantee and affect its ultimate cost to the Government. This is needed both to reestimate subsidy cost and to establish a basis for estimating new subsidies.

• Borrower location. This data is important for regional analysis. Managers may choose to collect several location elements, such as zip code, congressional district code, approving office code, and servicing office code. If data is tracked by field office, this data may be used to support performance measures for regional offices.

• Borrower creditworthiness. While creditworthiness may not influence the decision to extend credit, this data may be a strong predictor of net defaults. For example, a rise in the number of borrowers who have previously defaulted on loans may increase the risk of future defaults. Borrower creditworthiness includes measures of the financial condition of the business or individual as well as past experience with credit.

• Loan use. For certain loan programs, borrowers may be able to use Government credit for a wide variety of purposes. If this is true, tracking the intended loan use may reveal a significant variance in cost. For example, working capital loans may have a much higher incidence of default than construction loans.

• Program-specific data. The above items provide only a few examples of the type of data agencies may choose to maintain in their information stores. Other characteristics relating the loan may also be important in predicting default. For example, the loan-to-value (LTV) ratio is a critical data element for predicting housing loan defaults; for student loans, the type of educational institution is important; the value of collateral is important in a number of programs. As a general rule, the wider the range of loan characteristic data collected, the more useful the information store will be as the loan program changes.

Economic data are also useful for evaluating portfolio management. Unless economic influences can be isolated, it is difficult to measure how well a loan portfolio is managed.

Historical cash flows: All cash transactions (i.e., all transactions which pass through the cash account) related to each loan should be maintained in the information store. Since transactions may be identified by a wide variety of transactional codes, cash flows should be grouped by the type of cash flows that are projected in loan program subsidy estimates. These groupings will vary from program to program, depending on the way cash flows are projected for subsidy estimates. Listed below are suggested groupings which should be modified to fit actual loan programs. The first two categories must be used by all information stores.

• Approval amount. This is the first step in tracking the cash flows related to each loan. Following cash flows from the approval stage allows for predictions of the cancellation rate of future cohorts. Since the cash account is not affected by a loan approval, these data must be drawn from another source.

• Disbursement amount. For direct loans, the initial disbursement is the first cash transaction. For loan guarantees, the initial disbursement is not a transaction of the program and the data must be collected from the lending institution. The disbursement amount is an important direct element in subsidy cash flows for direct loans. The disbursement rate -- the percentage of the total disbursement made in different years -- is an important element in calculating the subsidy for all direct loans and loan guarantees, since all cash flows are discounted to the year of disbursement. For example, if a cohort disburses evenly over five years, twenty percent of each remaining cash flow (other than the disbursements) will be discounted to each of Years 1-5. The cohort cancellation rate can be calculated by comparing the gross approval amount to the disbursement amount.

• Up-front fee. Many Federal loan programs charge an up-front fee. If the up-front fee is displayed separately in the subsidy cash flows, the accuracy of up-front fee collections is easy to validate in future years with actual data.

• Annual fees. If annual fees are charged, they should also be tracked separately. When projecting the collection of fees for future cohorts, subsidy models should incorporate the reduction in collections as a result of projected loan cancellations, defaults, and prepayments.

• Interest subsidies. Many loan programs provide credit at subsidized interest rates. For loan guarantee programs, this results in payments to lending institution. For direct loan programs, interest subsidies are calculated on a present value basis as the difference between the present value of the contractual cash flows using the interest rate charged to the borrower and the present value of the contractual cash flows using the rate on Treasury securities of comparable maturity.

• Prepayments. For direct loans programs which charge an interest rate other than the rate on Treasury securities of comparable maturity, prepayment projections can have a large impact on the subsidy rate. For loan guarantees, prepayment data is important if the borrower interest rate is Federally subsidized or if the borrower pays an interest premium which is passed on to the Government. The total effect of prepayments must be projected in subsidy cash flows estimates. Not only must the timing and amount of prepayments be tracked, but also the subsequent annual effect on payments to the Government.

• Defaults/claim payments. Depending on the loan program, defaults may be tracked as a single category, or may be divided into more specific subcategories. For loan guarantees, claim payments are relatively easy to track. A default can be defined as the purchase of the agency's share of a guaranteed loan, including a certain amount of accrued interest. Tracking defaults for direct loan programs requires a comparison of scheduled to actual payments. For credit subsidy estimates, agencies must project how the expected cash flows to the Government will differ from the borrowers' scheduled payments.

• Recoveries. Agencies may choose to divide gross recoveries into smaller categories, for example, the number of loans that return to good standing and the recovery rate (and timing of recoveries) from collateral liquidations. Recoveries should be net of costs paid from the financing account.

• Scheduled principal and interest payments. The importance of scheduled payments cannot be over emphasized. Unless scheduled. payments are accurately maintained, and cash flows are updated with the most recent repayment schedule, the comparison of scheduled to actual payments in the reestimate process cannot be validly made.

• Actual principal and interest payments. Actual borrower payments should be tracked for each loan in order to compare with scheduled payments and determine the prepayment and default rates.

• Delinquencies. This is the period in which payments less than those scheduled are made by the borrower. Patterns in historical delinquency rates should support delinquency estimates included in subsidy cash flows.

• Lending institution and guaranty agencies. For loan guarantee programs, agencies should track the performance of its lending institutions and, if applicable, by guaranty agency. For example, if a particular institution has a high level of defaults relative to its peers, the institution might be suspended from the program. An improved quality of lending institutions could serve as the justification for lowering future default assumptions.

• Interest subsidies. For programs that extend interest subsidies, these payments should be tracked separately. For fixed rate loans, the cost of the interest subsidy will be strongly affected by changes in interest rates.

 

¹"Information store," as opposed to "database," will be used throughout this paper. This usage follows JFMIP, Direct Loan System Requirements, pp. 15-20, and Guaranteed Loan System Requirements, pp. 15-21. The JFMIP used "information store" to avoid any confusion with the technical or physical characteristics of the data storage medium. Each agency must determine the appropriate hardware and software for physically storing and then manipulating the data.
²Paragraph 36. If the cohort is not divided into risk categories, the standard applies to the cohort as a whole.
³"Informed opinion" refers to the judgement of agency staff or others who make subsidy estimates based on their programmatic knowledge and experience without using an econometric model or a satisfactory information store.
⁴Paragraph 35. If the cohort is not divided into risk categories, the standard applies to the cohort as a whole.
⁵Paragraph 35.
⁶Although an average for the portfolio as a whole does not provide the year-by-year information needed to estimate the effect of defaults on the present value of loan repayments, for example, an average for the portfolio as a whole may be useful for estimating some elements of cash flow such as recoveries from foreclosed property.
⁷This section and the following two sections of the paper cover the types of data that SFFAS No. 2, paragraphs 34 and 36, says ought to be considered as risk factors in estimating default costs or for other reasons included in the information store to estimate cash flows.