Charity Lays Off Survey Architect
The chief architect of the controversial National Jewish Population Survey, Jim Schwartz, has been laid off by the survey’s sponsor, United Jewish Communities, according to sources close to UJC.
The philanthropy announced last week in an internal communication that Schwartz’s position as director of the North American Jewish Data Bank was being eliminated as part of a reorganization, several sources said. The data bank is jointly operated by UJC and Brandeis University.
The announcement came just days after the release of an external review that detailed a series of errors and missteps in the $6 million population survey. Questions about the survey’s reliability have mired UJC in controversy for the past year.
Schwartz declined to be interviewed for this article.
Schwartz had overseen the survey as UJC’s director of research, a position he assumed in 1997. Last January, however, the organization announced that he had been reassigned to the data bank. The announcement coincided with the release of a preliminary review of the survey by an internal review panel, chaired by the principal and vice-chancellor emeritus of McGill University, Bernard Shapiro, that found “limitations and qualifications” in the survey.
The population survey was initially released in October 2002 but was withdrawn three weeks later, after researchers raised questions about support data that appeared to be missing. Shapiro’s review committee was appointed in November. It reported in January that “a range of serious issues of conception, of data collection, and of analysis were present in the project.”
In April a new review committee was formed, made up of outside experts headed by Marc Schulman, past chairman of the American Association for Public Opinion Research.
The survey itself was finally released this past September 10. Its main finding, widely quoted in the press, was an American Jewish population totaling 5.2 million, which is 5% less than the 5.5 million total found in a similar survey in 1990.
The survey also reported that the Jewish community showed signs of increased involvement in Jewish activity and ritual, including a Jewish day school attendance of 29%.
The published survey noted that its estimate of a 5.2 million total population may represent an undercount, and UJC officials cautioned in interviews that its findings did not necessarily indicate a population decline. However, UJC’s Web site continues to feature news articles reporting the decline as fact.
The Schulman review, submitted to the UJC September 19, points to serious methodological flaws that may have led to an undercount and damaged the survey’s comparability with the 1990 survey. It also cited flaws in data collection and analysis that may “skew” the Jewish sample “toward being more strongly identified with Judaism” than the actual Jewish population, undercutting the survey’s report of a more observant community.
Researchers said this week that one little-noticed calculation error cited in the Schulman report appeared by itself to have reduced the Jewish population by as much as 100,000. The glitch amounts to just 2% of the 5.2 million total, but researchers say it may be a sign of things to come as they begin to comb through the errors described in the review.
According to the review, the survey erred in calculating the percentage of Jews in the overall population by misinterpreting the responses of those who declined to give their religion. Such respondents were classified as non-Jews and included in the overall sample, rather than being deducted as nonresponsive, as is customary in such surveys. As result, Jewish respondents were calculated as part of an artificially large total sample, reducing their proportion and leading to an undercount.
“If it was assumed they were all non-Jewish, I think it was a mistake,” said Ira Sheskin, who was a member of the survey’s National Technical Advisory Committee. “But it’s not a big mistake. These surveys were never made to be accurate to the 100,000.” Sheskin is a geographer at the University of Miami.
Schulman also cited other errors in the design of the questionnaire, data collection and calculation of results, all of which he said could have resulted in undercounting the Jewish population. In addition, he cited the data whose loss, due to a faulty computer program, set off the review process last year, saying it, too, may have caused an undercount.
Brandeis University is scheduled to hold a colloquium later this month to gauge the values and limitations of the survey, according to scholars connected to the data bank.
Sheskin said he regretted Schwartz’s departure. “Jim is a personal friend of mine,” he said. “He put a lot of work into this study. I wish the study could’ve finished with him at the head of it. I don’t know all the reasons why the decision was made. I don’t know where the blame really should lie.”
A message from our CEO & publisher Rachel Fishman Feddersen
I hope you appreciated this article. Before you go, I’d like to ask you to please support the Forward’s award-winning, nonprofit journalism during this critical time.
We’ve set a goal to raise $260,000 by December 31. That’s an ambitious goal, but one that will give us the resources we need to invest in the high quality news, opinion, analysis and cultural coverage that isn’t available anywhere else.
If you feel inspired to make an impact, now is the time to give something back. Join us as a member at your most generous level.
— Rachel Fishman Feddersen, Publisher and CEO
