Public Comment Regarding: Additional comments on OMB Guidance on "Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies," on paragraphs V.3.B, V.9, and V.10, on the "capable of being substantially reproducible" standard.
Date: 11/ 2/2001
|Brian V. Greenberg,Social Security Administration
Text of comment:
Social Security Administration Comments and Recommendations on Capable of Being Substantially Reproduced
Thank you for the opportunity to comment on the capable of being substantially reproduced standard in sections V.3.B, V.9 and V.10 of the guidelines on information quality. SSA has one comment and two suggested remedies.
There are three general principles embodied in these guidelinesto ensure information quality, public access to information, and protections of privacy and security of information. The principles of public access and protections of privacy and security present competing demands, especially with respect to making ones results capable of being substantially reproduced.
SSA publishes a wide range of statistical information and policy analyses on its beneficiary populations and on workers covered for benefits. Most of this information is generated from SSAs program records on benefits and earnings or from survey data that have been linked to SSAs program records. Reproducibility would require access to the underlying data as well as the computer programs that generated the information. SSA program records have limited access under the Privacy Act, SSAs Regulation 1 and Section 6103 of the Treasury legislation covering individual earnings records maintained by SSA. Survey data such as the Census Bureaus Survey of Income and Program Participation that has been linked to SSA records is another major source of data for SSAs policy analyses. These data also have limited access under the Census Bureaus Title 13 legislation. Thus, in many cases, data confidentiality requirements would prohibit SSA from providing the data files (except to persons qualified under the relevant legislation) that would allow someone to reproduce statistics or analyses. Legislation covering access to these program records and survey data would take precedence over this guideline to make ones results capable of being reproduced.
Making supporting data available is reasonable, where possible, and has been SSAs general practice. However, it is not generally practical to make complex models, such as those used by the Office of the Chief Actuary (OCACT), publicly available in their full detail. These models are so varied and dynamic that it would be impractical to make them generally available. These models change very frequently in response to changing demands and the burden of continual disclosure of fully detailed models would result in an inability to maintain productivity.
A more reasonable model for disclosure would be that applied in actuarial practice for insurance in the private sector. In this case, actuarial memoranda are produced identifying data sources and attesting to the use of methods consistent with standard actuarial practice. OCACT complies with this level of disclosure, and more. OCACT additionally provides, where requested, descriptions of actuarial methodologies that permit qualified individuals, with expertise in the subject area, to produce estimates that will be the same for all practical purposes.
The first sentence of V.3B.ii should be amended to change the last word from models to methods as follows:
ii. In those situations involving influential scientific or statistical information, the results must be capable of being substantially reproduced, if the original or supporting data are independently analyzed using the same methods.
The first sentence of V.3B.iii should be amended from:
iii. Making the data and models publicly available will assist in determining whether analytical results are capable of being substantially reproduced.
iii. Making the data and either the models or a description of the model methodologies publicly available will assist in determining whether analytical results are capable of being substantially reproduced.