Public Comment Regarding: Additional comments on OMB Guidance on "Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies," on paragraphs V.3.B, V.9, and V.10, on the "capable of being substantially reproducible" standard.
Date: 11/ 2/2001
Signer
Organization
Brian V. Greenberg,Social Security Administration
Text of comment:
Social Security Administration Comments and Recommendations on “Capable of Being Substantially Reproduced”
Thank you for the opportunity to comment on the “capable of being substantially reproduced” standard in sections V.3.B, V.9 and V.10 of the guidelines on information quality. SSA has one comment and two suggested remedies.
There are three general principles embodied in these guidelines—to ensure information quality, public access to information, and protections of privacy and security of information. The principles of public access and protections of privacy and security present competing demands, especially with respect to making one’s results capable of being substantially reproduced.
SSA publishes a wide range of statistical information and policy analyses on its beneficiary populations and on workers covered for benefits. Most of this information is generated from SSA’s program records on benefits and earnings or from survey data that have been linked to SSA’s program records. Reproducibility would require access to the underlying data as well as the computer programs that generated the information. SSA program records have limited access under the Privacy Act, SSA’s Regulation 1 and Section 6103 of the Treasury legislation covering individual earnings records maintained by SSA. Survey data such as the Census Bureau’s Survey of Income and Program Participation that has been linked to SSA records is another major source of data for SSA’s policy analyses. These data also have limited access under the Census Bureau’s Title 13 legislation. Thus, in many cases, data confidentiality requirements would prohibit SSA from providing the data files (except to persons qualified under the relevant legislation) that would allow someone to reproduce statistics or analyses. Legislation covering access to these program records and survey data would take precedence over this guideline to make one’s results capable of being reproduced.
Making supporting data available is reasonable, where possible, and has been SSA’s general practice. However, it is not generally practical to make complex models, such as those used by the Office of the Chief Actuary (OCACT), publicly available in their full detail. These models are so varied and dynamic that it would be impractical to make them generally available. These models change very frequently in response to changing demands and the burden of continual disclosure of fully detailed models would result in an inability to maintain productivity.
A more reasonable model for disclosure would be that applied in actuarial practice for insurance in the private sector. In this case, actuarial memoranda are produced identifying data sources and attesting to the use of methods consistent with standard actuarial practice. OCACT complies with this level of disclosure, and more. OCACT additionally provides, where requested, descriptions of actuarial methodologies that permit qualified individuals, with expertise in the subject area, to produce estimates that will be the same for all practical purposes.
Recommendation
The first sentence of V.3B.ii should be amended to change the last word from “models” to “methods” as follows:
“ii. In those situations involving influential scientific or statistical information, the results must be capable of being substantially reproduced, if the original or supporting data are independently analyzed using the same methods.
The first sentence of V.3B.iii should be amended from:
“iii. Making the data and models publicly available will assist in determining whether analytical results are capable of being substantially reproduced.”
to read:
“iii. Making the data and either the models or a description of the model methodologies publicly available will assist in determining whether analytical results are capable of being substantially reproduced.
