Already a member?

Sign In
Syndicate content

United States

Council of Professional Associations on Federal Statistics (COPAFS) meeting notes

I was lucky enough to be able to sit in on the most recent COPAFS meeting in place of our regular liaison Judith Rowe.  While the topics were very different than the issues I usually deal with at work, I found the presentations really interesting. Here's an abridged version of my notes.

 

News:

Ed Spar will be stepping down as Executive Director at the end of 2012.  The board will be launching a search and will be engaging a search firm.

Director's update:

The budgetary situation is grim to worse and outlook isn't any better. Every agency will wish they had last years budget. Census numbers reflect a very bad year coming up. The meeting dates for next year are: March 16, June 1, Sept 14, December 7.

Update on National Center for Education Statistics (NCES)- Marilyn Seastrom
NCES is the statistical agency within Dept of Education.  They have a small staff but lots of contractors and may be lucky enough to be level funded next year.

Assessment: it was the busiest year in the history of national assessment.  They are ready to release the state mapping report.  This compares assessment measures across states - map state assessments to National Assessment of Educational Progress (NAEP). For example, there is only one state (MA) where a 4th grader who is deemed is proficient on the state exam is proficient on the national level.  There are many states where they are proficient at the state level but they don't even make the "basic" cut for the national assessment. The are also ready to Release the Reading and Mathematics report card


Elementary and Secondary update: They've done an expansion of NCES Geo-mapping application which works with the ACS to provide data by school district boundaries.


Miscellaneous: there's a new OECD adult literacy study (PIAAC - first international assessment done on laptops in the home) and the national household education survey (what goes on outside of school) is no longer random digit dial sample due to deterioration in response rates, now address based sample (mail) .
There's new stuff on the horizon:  a middle school study, NAEP-TIMSS (Trends in International Mathematics and Science Study) link which will be an ambitious study using 8th grade level achievement in math and science.

American Demographic History: Campbell Gibson (demographer retired from Census)
Website of demographic history : www.demographicchartbook.com
Developed over a few years with David Kennedy and Herbert Kline (Stanford) - about 130 graphics through 2000 for both state and national charts which are freely available and can be downloaded.
Source:  all decennial census - some drawn from compendia of ipums files.
He showed a variety of slides - all of which are available on the website and most of which were fascinating.  Can you guess the changes in the set of the top five languages spoken in the home of non-US born residents?

Rural Statistical Areas: Mike Radcliffe, Geography Division, Census
The presentation described a three year joint research project with 23 states. The goal was to define Rural Statistical areas - geographic areas defined using counties, county subdivisions and census tracts a building blocks. The goal was to be able to tabulate ACS 1 year estimates for areas of 65K+ people. These areas would be based on rural focus - not like pumas which used 100K but mostly urban areas.  They started with most rural parts and build from there - urban is really the residual.


RSA delineation process - counties with 65K+ would be standalone RSAs if rural focus.  Used the urban influence codes (UIC from USDA) to get to "ruralness" and grouped counties with some boundary tweaks made by State Data Center Steering committee. He showed maps of UIC ratings then discussed how to aggregate counties:  they created an aggregation net using state boundaries, interstate highways and rivers to create a lattice work to think about how to group counties.  They started with UIC category 12 and aggregated up by county until you hit the 65K+ measure. It's an imperfect measure and there were some problems with adjacent county differences and sometimes had to sacrifice resolution.


The resulting definitions for RSAs by state were sent to the state and they were able to move things around a bit to help smooth out some of the initial classification imperfections. Some states suggested alternative definitions; for example, Vermont wanted to use their planning regions.


Questions on the table:

  • Should RSAs be contiguous? Census has a preference for yes but states disagree - eg Alabama might have similar demographics between north and south counties that would match better for an RSA than using geography.
  • Can a variety of building blocks be used to form RSAs?  Initial proposal was counties but they may not be the best units to start with.  States found that in some cases sub-county divisions or census tracts worked better.
  • Why not cross state lines?  Makes sense for some questions but State data centers need to address rural areas withing their states?
  • Should counties of 65K+ be split into multiple areas?

Next steps:
State data centers have asked Census to define these as statistical areas but Census has said that in some cases (like Los Angeles) you just can't call them rural.  What do you call them? The project needs to get wider review including public comment through a Federal Register notice.


Research on measuring same sex couples - Nancy Bates - Census
Motivation: definition of marriage has changed; new terms and different state recognition and no federal recognition of same sex couples. According to 2008 ACS, there are about 150,000 self described same-sex married couples but only around 32,000 same-sex legally married couples.

Possible causes:

  • Classification error:  maybe people think of themselves as married even if they aren't.
  • First response: on ACS the husband/wife category is first in list but unmarried partner is 13th
  • Errors elsewhere: false positives due to incorrect gender response

Research: some based on focus groups - 18 groups in 8 different areas with different legal recognition of same sex marriage.  Mostly gay couples but some unmarried straight couples.  Most people interpreted the question on federal form as indicating "legal status".  Some thought it meant "legally married anywhere".  Many groups noted they were missing categories for civil unions or domestic partnerships. And there is the "function equivalence" problem that couples had the equivalent of a marriage but no where to put themselves.


Research: some based on cognitive interviews - 40 interviews both gays and straights across different legal jurisdictions. Participants filled out forms then were debriefed afterwards and showed alternative form and asked for preference.
Results: most survey results aligned with "true" legal status.  Specifically calling out same sex or opposite sex in the marital status question was preferred but also was flagged as potentially sensitive. Would this delineation increase unit non-response? Also, there was some confusion about defintion of civil union/domestic partnership.  Most people found it useful to have a cohabitation question.
Next steps:  interagency group review, piggyback on an ACS test for a larger trial which is mail only and they need to test in other modes and would love to be able to have a re-interview component.

Research on measuring same sex couples - Martin O'Donnell - showing some data
Showed a comparison of ACS data and census stuff - but comparability may not be perfect.
Changes in ACS forms and editing caused a drop of self reported same sex spouses from 350K+ to 150K+.

2010 Census results showed much higher level of same sex households than the 2010 ACS.  There was a huge difference between mail forms and non-mail forms.  Approximately 3 times as many households reported themselves as same sex households in mail forms as non-mail forms for ACS where the non-mail were nonresponse follow up (NRFU). On the pre2008 ACS and 2010 Census NRFU form, the matrix format for the form didn't yield consistent results.  ACS 2008+ and 2010 Census form had a person based column format which had much more consistent responses.  This is truly non-sampling error for populations: you only need 4 errors per 1000 of opposite sex households to generate the 250K+ error in the same sex spouses because there are 60 million of them.


Problem: bad matrix form was approved and printed before these results where available. Now short form data wave 1 is published including one table with one table about same sex couples but they can't stop the processing of the entire 2010 Census to allow for the correction of one table. Now how do they fix it?


They tested the quality of the reporting on sex.  Used name index to match the probability that a person has a name associated with a male (John or Thomas has very high index, Virginia or Elizabeth is very low) with state controls for cultural differences (Jean may be more likely to be a male in French areas).  Index value of 0-50 were likely to be female and those with 950-1000 were likely to be male.  Couples with a female partner with a name at the highest index value or a male partner with a name at the lowest index value where then considered to have incorrectly marked the sex item on the question and they were dropped from the same sex couples category. Ex: 9000 male-male couples in Texas out of 31,000 have names that indicate they are probably male-female couples - nearly one third of the same sex marriage stats in American Factfinder may be incorrect.  


Geographic distribution with inconsistent name reporting: swath from Florida north west to ND - matches high rate of NRFU forms.
Summary: They reissued the numbers which matched the 2010 ACS better once the name mismatched folks where thrown out. Spousal household estimate is most improved. American Factfinder page shows people where to go to get preferred estimate. Census PUMS is based on edited data.  They aren't recalculating the entire Census data but they are published the edit data and there will be a flag on data that are affected.

Stephen S. Clark Library for Maps, Government Information, and Data Services is open for business!

Three cheers for Jen Green!!! 

When not keeping IASSIST finances in check as the IASSIST Treasurer, Jennifer Green, director of the new Stephen S. Clark Library for Maps, Government Information, and Data Services, at the University of Michigan has been busy getting the library in shape for the recent opening day! 

Check out the announcement of the grand opening festivities in the Record Update (a publication of the Office of the Vice President for Communications at the University of Michigan) and don't miss the brand new website of the Setphen S. Clark Library

Green says the new library’s unique combination of collections, government information expertise, and data services will provide scholars and researchers with unprecedented opportunities for exploration, discovery, and collaboration.

“Before the Clark, there was a large degree of interaction among these three units,” Green says. “Our new proximity, in a purposefully designed and equipped space, means that we can more effectively collaborate with each other, which in turn really enhances our ability to creatively collaborate with students, faculty, and researchers.”

From the Record Update

Open Access to Federally Funded Research

Got something to say about "ensuring long-term stewardship and encouraging broad public access to unclassified digital data that result from federally funded scientific research"?

 

The White House Office for Science and Technology Policy (OSTP) released two public consultations today, one on OA for data and one on OA for publications arising from publicly-funded research. Responses are due in early January. Please spread the word. Submit your own comments and/or work with colleagues to submit comments on behalf of your institution.

(1) "[T]his Request for Information (RFI) offers the opportunity for interested individuals and organizations to provide recommendations on approaches for ensuring long-term stewardship and encouraging broad public access to unclassified digital data that result from federally funded scientific research....Response Date: January 12, 2012...."
http://goo.gl/L1jn3

(2) "[T]his Request for Information (RFI) offers the opportunity for interested individuals and organizations to provide recommendations on approaches for ensuring long-term stewardship and broad public access to the peer-reviewed scholarly publications that result from federally funded scientific research....Response Date: January 2, 2012...."
http://goo.gl/vTP18

ANES Announcement: The ANES 2012 Time Series Study

On June 30, 2011, the American National Election Studies (ANES) began accepting proposals for questions to include on the ANES 2012 Time Series Study.  Proposals may be submitted through the ANES Online Commons. The following describes the goals of this study and the opportunity to include questions on it.

About The ANES 2012 Time Series Study

The ANES’s core mission is to promote cutting-edge and broadly-collaborative research on American national elections. The heart of the ANES is its presidential year time series surveys. The time series legacy is well known, serving as a model for election studies around the world and having generated thousands of publications. Every four years, a large representative sample of American adults has been interviewed on two occasions, first between Labor Day and Election Day, and again between Election Day and the onset of the winter holidays. The two face-to-face interviews will last approximately one hour each in 2012. Pre-election interviews focus on candidate preferences and anticipated vote choice; an array of possible predictors of candidate preferences, turnout, citizen engagement; and an array of indicators of cognitive and behavioral engagement in the information flow of the campaign. Post-election interviews measures a variety of behavioral experiences people might have had throughout the campaign (e.g., turnout, mobilization efforts), plus additional posited predictors of candidate preferences, turnout, and citizen engagement.

Some of the questions asked during these interviews are categorized as standard (also known as core) items, meaning that they have been asked regularly over the years.  These questions are scheduled to appear on subsequent editions of the ANES Time Series in order to permit comparisons across elections.  The purpose of categorizing items as standard is to assure scholars who conduct longitudinal analyses that they can continue to depend on ANES to include variables that have been shown to perform well in the past.

Although recognizing the importance of continuity, ANES has also sought to develop the time series in innovative ways. The non-standard component of each questionnaire has routinely focused on matters of interest to the current election cycle. These items are often selected from an "ANES Question Inventory," which includes the standard questions and questions that have been asked in past ANES surveys but are not part of the standard battery of questions.  Researchers can access the question inventory at:

ftp://ftp.electionstudies.org/ftp/anes/OC/CoreUtility/ALT2010core.htm

The non-standard content of questionnaires has varied over the years. For example, candidate positions on issues of government policy are recognized as predictors of candidate preferences, but two one-hour interviews do not permit measuring positions on all of the many issues enjoying government attention at any one time in history. So from year to year, different choices have been made about which issues to include in the questionnaire.

As in the past, ANES will continue to emphasize best practices in sample design, respondent recruitment, and interviewing.  As always, we aim to provide top-quality service in many respects, including: (1) the careful and extensive planning that must be done before the field work begins, (2) the hard work that will be done by interviewers, supervisors, and study managers during data collection to monitor productivity and make adjustments in strategy to maximize the quality of the final product, and (3) the extensive data processing efforts (including integration of an extensive contextual data file) that will be required to assemble and document the final data set.

 

About the Online Commons

Content for the ANES 2012 Time Series Study will primarily evolve from two sources:  previous ANES Time Series questionnaires and new proposals received via the ANES Online Commons (OC).  The OC is an Internet-based system designed to promote communication among scholars and to yield innovative proposals about the most effective ways to measure electorally-relevant concepts and relationships. The goal of the OC is to improve the quality and scientific value of ANES data collections, to encourage the submission of new ideas, and to make such experiences more beneficial to and enjoyable for investigators. In the last study cycle, more than 700 scholars sent over 200 proposals through the OC.

Proposals for the inclusion of questions must include clear theoretical and empirical rationales. All proposals must also clearly state how the questions will increase the value of the respective studies. In particular, proposed questions must have the potential to help scholars understand the causes and/or consequences of turnout or candidate choice.

The ANES Online Commons will accept proposals until 3:00pm Eastern Time on August 30, 2011. The deadline for members of the Online Commons community to comment on proposals is September 8, 2011. The deadline for revisions to proposals is at 3:00pm Eastern Time on September 14, 2011.

For additional information about how to submit a proposal, please visit:

http://www.electionstudies.org/

 

Proposal Evaluation Criteria

The following criteria will guide the PIs and the ANES Board in evaluating proposals made through the Online Commons. We strongly encourage anyone who is considering making a proposal to read the following carefully.

1. Problem-Relevant. Are the theoretical motivations, proposed concepts and survey items relevant to ongoing controversies among researchers? How will the data that the proposers expect to observe advance the debate?

What specific analyses of the data will be performed? What might these analyses reveal? How would these findings be relevant to specific questions or controversies?

2. Suitability to ANES. The primary mission of the ANES is to advance our understanding of voter choice and electoral participation. Ceteris paribus, concepts and instrumentation that are relevant to our understanding of these phenomena will be considered more favorably than items tapping other facets of politics, public opinion, American culture or society.

3. Building on Solid Theoretical Footing. Does the proposed instrumentation follow from a plausible theory of political behavior?

4. Demonstrated Validity and Reliability of Proposed Items. Proposed items should be accompanied by evidence demonstrating their validity and reliability. Validity has various facets: e.g., construct validity, concurrent validity, discriminant validity and predictive validity. Any assessment of predictive validity should keep in mind criterion 2, above.

Reliability can be demonstrated in various ways; one example is test-retest reliability. We understand that proposals for novel concepts and/or instrumentation will almost always lack empirical evidence demonstrating validity and/or reliability. Proposals for truly "novel" instrumentation might be best suited for the series of smaller, cross-sectional studies ANES will field in the period 2010 through the summer of 2012; as a general matter, we are highly unlikely to field untested instrumentation on the Fall 2012 pre-election and post-election surveys.

5. Breadth of Relevance and Generalizability. Will the research that results from the proposed instrumentation be useful to many scholars?

Given the broad usage of ANES data, we may be unable to accommodate requests to include items that are relevant for one -or only a few- hypothesis tests. Ceteris paribus, items that are potentially relevant for a wide range of analyses will be considered more favorably than items that would seem to have less applicability.

When the 2012 questionnaires are designed, the status of the standard questions will be central considerations. Standard questions do not have an infinite shelf life -- Science advances and new insights can reveal more effective ways of asking important questions or can show that some questions do not in fact meet the requirements of remaining a standard question.  However, proposed changes made to standard questions will be scrutinized with recognition of the value of continuity over time.  While we will welcome proposals to change standard questions, the burden of proof required for making such changes will be high. We will take most seriously arguments that are backed by concrete evidence and strong theory.

All proposals that include a change to a particular question (standard or non-standard) should name the specific question that would be altered and provide a full explanation as to why the ANES user community will benefit by such a change.

Tools To Assist Your Proposal Development

As previously mentioned, researchers can access the ANES Question Inventory at:

ftp://ftp.electionstudies.org/ftp/anes/OC/CoreUtility/ALT2010core.htm

This Inventory provides the list of standard and non-standard questions that have been part of the Time Series, and includes frequencies for the most recent studies.

We have also created a second resource to review questions that have been asked previously.  The ANES Time Series Codebook Search utility searches existing codebooks from studies in the ANES Time Series.   You can access the utility at http://ftp.nes.isr.umich.edu/backup/searchhelp.htm  

(Please note that there are some limitations to the utility that are documented on the search help page, the link to that page is at the top of the utility page.)

We hope that you will find these tools useful as you prepare your proposals.

The opportunity to submit proposals is open to anyone who wants to make a constructive contribution to the development of the ANES 2012 Time Series Study. Feel free to pass this invitation along to anyone (e.g., your colleagues and students) who you think might be interested. We hope to hear from you.

For additional resources and information on how to submit a proposal, please visit http://www.electionstudies.org/onlinecommons/

 

Darrell Donakowski

Director of Studies

American National Election Studies (ANES)

Roper Centre recognition

To iassist:

It isn't every day that an incoming university president envies data over basketball!
Congratulations to our colleagues at the Roper Center for this well deserved recognition.

Susan Herbst, incoming President at the University of Connecticut:
What about UConn drew you here? more...

Regional Report 2008-2009: United States

IASSIST Regional Report 2008-2009
United States
Pam Baxter
Cornell University
May 2009 more...

Regional Report 2006-2007: United States

IASSIST Regional Report 2006-2007
United States
Pam Baxter
Cornell University
May 2007

français

more...

Regional Report 2007-2008: United States

IASSIST Regional Report 2007-2008
United States
Pam Baxter
Cornell University
May 2008 more...

SPARC Digital Repositories meeting includes session on open data

Kathleen Shearer of the Canadian Association of Research Libraries organized and chaired a panel on Open Data was held at the SPARC Digital Repositories meeting on November 8, 2010.  IASSIST members Gail Steinhart and Chuck Humphrey were two of the three members on this panel.  Kevin Ashley, Director of the Digital Curation Centre (DCC), was the third. more...

  • IASSIST Quarterly

    Publications Special issue: A pioneer data librarian
    Welcome to the special volume of the IASSIST Quarterly (IQ (37):1-4, 2013). This special issue started as exchange of ideas between Libbie Stephenson and Margaret Adams to collect

    more...

  • Resources

    Resources

    A space for IASSIST members to share professional resources useful to them in their daily work. Also the IASSIST Jobs Repository for an archive of data-related position descriptions. more...

  • community

    • LinkedIn
    • Facebook
    • Twitter

    Find out what IASSISTers are doing in the field and explore other avenues of presentation, communication and discussion via social networking and related online social spaces. more...