Trivia Pursuit

...There are so [many] bad statistics,” says Gabriel Weinreich, an acoustical physicist at the University of Michigan...

...Everett Ladd, a professor at the University of Connecticut, told Boyer that the emphasis on research and publication is “seriously out of touch with what faculty actually do and want to do...

...And the initiative for such change will have to come from the universities themselves...

...search activity is actually useful, the Association of The situation has American Universigotten so bad that even ties’s Robert Rosenthe normally lethargic zweig shied away from academic establish- the question...

...And you thought it was just a news program...

...As a result, universities have every incentive to comply with whatever strings the government decides to attach to their money...

...For instance, the literature chronicling recent research in the social sciences includes the following: . >"An Empirical Methodology for the Ethical Assessment of Marketing Phenomena Such as Casino Gambling" (Journal of the Academy David P. Hamilton is a reporter for Science of Marketing), in which University of Detroit I professor Oswald Mascarenhas explains not only that gamblers are more favorably disposed toward gambling than non-gamblers ("teleological and deontological justifications 1 of casino gambling were decisively low"), but that people look more favorably on gambling if they think they can get rich at it ("distribuI tive justice related conditional acceptance of casino gambling was higher...

...IS1 only indexes roughly 10 percent of all journals-this sample included about 4,500...

...But there would hardly be so many journals if there wasn’t money to support them-and that money comes almost wholly from the public...

...But most journals-85 percent, by some estimatesare distributed by commercial firms that usually make a healthy profit off government largess...

...Would-be reformers could do worse than to look into a recent report by Ernest Boyer of the Camegie Foundation for the Advancement of Teaching...

...Such analyses are far from foolproof: some scientists are unscrupulous and don’t cite their colleagues’ papers, while others can be Pass through the current periodical section, and you’ll find titles of “scholarly” research journals like School Food Service Journal, Bee World, and The Journal of Band Research...

...Rustum Roy, a material science researcher at the Pennsylvania State University, told me about his experience as a session chairman at a meeting of the Materials Research Society: "We received 500 papers on superconductivity, and I told the conference organizers that we could eliminate 90 percent without hurting the session...

...First, tenure candidates must convince their departmental peers that their research is up to the standards of the field, and the easiest way to do that is to amass a pile of impenetrable research articles, most of which are never read...

...above, three floors are devoted to classrooms and offices, while two more hold the library's collection of 1.5 million books...

...Many journals are put out by professional societies, like the American Chemical Society, which enjoy the advantages of nonprofit status...

...psychologists have the option of publishing in Psychosomatic Medicine, Psychosomatics, and Psychotherapy and Psychosomatics...

...Instead, it might be useful to consider ways of improving the quality of research by eliminating the perverse incentives that skew the present system...

...After all, who needs 138 journals of economics...

...Surprisingly, for a community of supposedly open-minded scholars, academics generally prefer not to be challenged in their views...

...The National Institutes of Health (NIH), for instance-the primary source of biomedical funding, and the target of Rosenberg’s campaign-has grown by an average of 8.1 percent a year for the last five years...

...Proliferate or perish Less substandard work would make it out of the academy, of course, if there were fewer outlets within which to be published...

...Or take the experience of University of Michigan President James Duderstadt...

...But Brown adds that a mediocre teacher with an outstanding research record probably would get tenure...

...For instance, although scientists like to portray themselves as inquiring, dynamic researchers who strike out wherever their curiosity leads them, in reality most of the same criticisms levied against "pack journalism" in the political world can just as easily be aimed at "pack research...

...But even if increases were feasible, it’s not clear they would be a good idea...

...In fact, in one of the famous frauds of science, a medical researcher named Elias Alsabti published nearly 60 of his colleagues’ old articles in obscure journals under his own name...

...The control academic departments hold over promotions would necessarily be diminished, since Boyer also quite sensibly recommends rewarding interdisciplinary scholarship...

...Even then, "difficulties in measuring variation in autonomy and interdependence" will make policy decisions "imprecise...

...Drexel University science historian David Noble suggests that democratic control of research funding would go a long way toward asserting the needs of society over the often misplaced priorities of scientists and other academics...

...But political control of science has an unhappy history, one filled with horrors such as T. D. Lysenko’s fraudulent crop genetics in the Soviet Union and the barbaric medical experimentation of Josef Mengele in Nazi Germany...

...For Rosenzweig and others, there’s just one fly in the ointment-researchers aren’t getting as much money as they should...

...One way trapolating the preceding trends into the bottom tier of journals in another crude calculation suggests that perhaps only three scientific articles out of every hundred are worth reading at all...

...And "our lack of skill at converting research findings into diagnostic recommendations for organizations" will also hinder the search for concrete solutions...

...Even with these caveats in mind, however, the results of a recent citation analysis are rather startling...

...News and World Report...

...The federal government is in a unique position to pressure universities to change, thanks to the leverage provided by the research grants it parcels out to institutions...

...Of course, page charges are usually paid out of government grants, too...

...Unfortunately, Duderstadt isn’t exaggerating...

...Leon Rosenberg, dean of the Yale University School of Medicine, is one of the leading doomsayers...

...Universities should consider a broad range of writing in evaluating their faculty--everything from traditional research articles to textbooks and popular writing, such as the books and magazine articles of Stephen Jay Gould...

...But once you begin to look with a critical eye through the material kept there, some of your awe might begin to wane...

...publishable units . ” Some institutionsparticularly state universities, where “publication is seen as the road to respectability,” in the words of Harvard education professor Vito Perroneare likely to resist...

...But the agencies can go one step further, and require institutions that take federal money to do likewise in their promotion and tenure decisions...

...Any guess I could make would surely underestiment is waking up...

...Sturgeon’s law Although there’s a significant amount of money wasted on useless research, the financial cost is only part of the reason to be concerned...

...Yet such questions bear asking...

...There are a number of enthusiasts for just this approach...

...As a result, it’s hardly surprising that a sufficiently determined researcher can get his study, no matter how flawed it may be, into print somewhere...

...I is to discourage “sala- can’t think of a better mi science,” the prac- way to operate the systice of breaking re- tem than by having the search into the “lowest people involved judge “I’ve never met a scientist who didn’t bel ieve that 80 percent of the scientific literature was nonsense,’’ says an NIH researcher...

...4) not use simple methods...

...Maybe not...

...As someone who has to read a couple of hundred casebooks a year for tenure decisions, I can say it varies significantly from discipline to discipline,” Duderstadt says...

...Spending time with students is suicide in this competitive atmosphere...

...Spending more money on research isn’t the answer...

...Those sorts of judgments, it turns out, require the systematic assembly of data on "the relationship between autonomy, interdependence, and social control in diverse types of regulatory settings...

...The crisis in academic research is real, even if it’s not the one that critics such as Yale’s Leon Rosenberg like to complain about...

...Pick up one of these journals and actually try to read it, and you can make an even scarier discovery: that an unfortunately large percentage of what passes as the bedrock of academic achievement more closely resembles intellectual quicksand...

...Although the Too much of America's research building itself, a squat seven-story slab of conJ mone-y R- oes to studies cprrieztees ,a int dd ogelas spso, swseosns 'ta cweirnta iann yso laermchniittyec...

...In 10 years, only one subunit vaccine has ever protected people against disease-a failure that Young attributes to vaccinologists’ reluctance to really dig into the complex immunological systems that govern the body’s response to disease...

...And a quick statistical look at the nation’s research effort helps explain the smugness...

...Just standing in the presence of so much painstakingly assembled research is humbling...

...Presumably, Rosenzweig thinks the alternative to the current system would be some draconian setup whereby a panel of bureaucrats attempts to judge whether research into the immune system of rats is more worthy of federal money than high-energy physics...

...Redirecting the energies of the academy away from self-promoting activity and back toward its primary goals-educating and advancing the state of knowledge-will require fundamental changes in academic culture...

...tJuurset wants to read inside is a rook filled with the modem equivalent of the card catalog-flashing terminals before which anxious students are busily compilby David P. Hamilton ing lists of the books they need for footnotes in their current research paper...

...In 1986, when IBM researchers in Switzerland discovered a ceramic that became superconducting at much higher temperatures than previously thought possible, material scientists all over the world piled onto the superconductivity bandwagon...

...Even so, there are a number of good reasons to suspect that things are about as bad in the hard sciences as in social science, even if it's harder to figure out just from reading the published studies...

...Entitled Scholarship Reconsidered: Priorities of the Professoriate, it begins with the startling fact that 60 percent of today’s faculty believe that promotion should be based primarily on teaching ability rather than research prowess...

...Three tiers of teaching evaluations-self-assessment, student evaluations, and peer reviews-should also be incorporated into promotion decisions...

...Overall, the United States has the best research universities and graduate education in the world,” says Robert Rosenzweig, president of the research universities...

...There are more than 30,000 journals in the hard sciences alone, and well over 100,000 journals for all fields...

...Helfand argues that tenured job security attracts too many scholars seeking a respite from performance reviews and often “locks in” whole generations of academics-many of them untalented-during periodic “hiring frenzies” brought on by the retirement of faculty hired during the last such wave...

...But at heart, the idea of a research funding “crisis” borders on the fraudulent...

...You won’t get far with such concerns if you approach the guardians of the’ research establishment, of course...

...Class consciousness One obvious reform would be to have all research funding agencies-NIH, the Department of Defense, NASA, the Department of Agriculture, and the Department of Energy-follow the National Science Foundation’s lead and limit the number of papers applicants can include with their grant applications...

...Education has become a bottom-drawer priority for most academics...

...Spending it smarter is...

...and 6) not write clearly...

...While it’s indisning to limit the num- putable that “some reber of papers they’ll search results are less accept from faculty interesting than others,” members up for pro- it’s “difficult to tell in motion or tenure, as is advance which bets the National Science will turn out successful, Foundation...

...An earlier IS1 study considered papers that were cited once or more, and found that only 46 percent of such papers were cited more than once...

...For those of you who can’t smell the smoke, I say, please blow your nose and inhale again...

...Combining the two figures (an admittedly imprecise calculation, but one that provides some sense of the problem’s magnitude) leads to the conclusion that about 81 percent of all the published work in the sciences has next to no impact on the work of other scientists...

...Expointments...

...More often these days, publication is not regarded as a way of communicating knowledge, but of promoting faculty,” says Armstrong...

...The highly regarded commercial biology journal Cell, for instance, charges researchers $15 a page to publish papers which often run 15 or 20 pages...

...These simple measures could go a long way toward curbing the more flagrant failings of the current research system...

...With the exception of a few iconoclasts, however, no one spends much time figuring out what, exactly, society reaps from its substantial investment in academic research, or whether it might be better served by distributing government resources differently...

...Securing the Middle Ground: Reporter Formulas in '60 Minutes' " (Critical Studies in Mass Communication), in which University of Michigan professor Richard Campbell analyzes 154 of the show's episodes and concludes that its meaning lies in "story formulas" in which reporters "construct a mythology for Middle America...

...influenced by a study they forgot to credit...

...Some universities, led by the Harvard Medi- mate its value,” he cal School, are begin- says...

...It's much harder to make a similar judgment about work in the hard sciences, such as an AIDS study published a few years ago in which researchers mistakenly claimed to have found a viral cousin to HIV, unless you've spent years doing similar research yourself...

...In addition to whatever advertising revenue they can scrape up, commercial journals thrive on subscription rates nearly twice those of nonprofit journals, mostly paid by university libranes whose operating costs are largely covered by the government...

...The answer lies in three famous words: publish or perish...

...Within scientific circles, horror stories of studies rendered useless by bad methodologies, improper uses of statistics, shoddy data, sloppiness, or fraud are legion...

...Rosenzweig fairly well summarizes the conventional wisdom of the American research establishment: We’re doing just fine, thank you...

...In 1988 there were 356 accredited “research universities” supporting about 250,000 professors in some 275 disciplines ranging from applied mathematics to sociology to art history and criticism...

...It’s fair to argue that journal publishers are merely responding to demand...

...Sandwiched between offices and the stacks is the periodicals floor, which holds both the popular magazine collection .and perhaps as many as 10,000 scholarly journals-the published record of the world's past and present academic research...

...And what’s worse is that while they’re stumbling, the government is picking up most of the tab...

...The idea and which won’t...

...Similarly, it wouldn’t hurt for funding agencies to be more aggressive about the extent to which they reimburse library costs-a strategy that could help indirectly cut back on the number of journals in circulation...

...But Rosenzweig’s answer might be called the Sturgeon’s Law defense, in honor of the maxim-credited to the late writer Theodore Sturgeonthat “90 percent of anything is crap...

...The success of this approach has already been demonstrated-NIH, for instance, promulgated a set of guidelines for research practices and the handling of misconduct allegations...

...The Institute for Scientific Information (ISI), a nonprofit organization in Philadelphia that keeps a large citation database, recently found that among papers published between 1981 and 1985 (in the hard and social sciences), just over half were not cited ut all for five years after they were published...

...Pass through the current periodical section, and you'll find titles of "scholarly" research journals like School Food Service Journal, Bee World, and The Journal of Band Research...

...It was, like, just publish whatever you can to get more grants and more money...

...It wasn’t like the ultimate outcome was scientific knowledge,” said one researcher of her work at a Georgetown University medical school lab...

...They could also set guidelines for evaluating other scholarly contributions, such as the development of educational software or audiovisual materials...

...2) not challenge existing beliefs...

...It’s worth remembering that every hour faculty members spend conducting experiments, taking surveys, or “deconstructing” June Eyre is an hour they’re not spending with students...

...At research universities, a smaller-but still significant-2 1 percent of faculty agree that current priorities are scrambled...

...Trivia Pursuit Too much of America's research money goes to studies nobody wants to read by David P. Hamilton ike most academic libraries, the Gelman Library at George Washington University is an impressive place if you're easily awed by thinking about the accumulated weight of human knowledge...

...Right now, only those economists seeking a home for the article no one else will print...

...I’ve never met a scientist who didn’t believe that 80 percent of the scientific literature was nonsense,” says Walter Stewart, an NIH researcher with a longstanding interest in the integrity of scientific research...

...At Stanford, which has the highest indirect cost rate in the country, a $100,000 research grant actually costs the government $178,000-$100,000 to cover the researcher’s salary, his graduate students, and equipment, and $78,000 for the university’s purposes...

...As if that weren’t enough, many commercial journals resemble vanity presses by making researchers pay for the privilege of seeing their articles in print...

...In other words, since we can’t figure out in advance who will have the bright ideas, we’d better fund everybody...

...Among the publications indexed by ISI-and remember, this is only about 10 percent of everything in print-are 312 titles in psychology, 73 in sociology, 369 in mathematics, 126 in botany, 293 in literature, 61 in food and science technology, 12 in ergonomics, 55 in library science, and 18 in parasitology...

...These societies fund their publishing operations thrmgh a combination of advertising and membership dues-the latter paid largely out of government research grants...

...And those three did research in areas very idea...

...American science is drawing more money than ever before from the federal government, and even the budget convulsions last year left research spending growing faster than just about any other segment of the federal budget...

...Obviously, a great many scholars in fields ranging from psychology to immunology to chemistry do make significant contributions, both to scientific understanding and to society at large...

...When I asked him how much academic rewhere the information is a professional necessity...

...Finally, according to David Helfand, chairman of the astronomy department at Columbia University, tenure “can exclude productive, energetic scholars from the system, maintain unproductive, unmotivated teachers in our universities, and discourage our best young minds from pursuing academic careers...

...Surveys reveal that the average number of hours faculty spend in the classroom each week has fallen from 10.5 in 1980 to only 8.7 in 1989...

...But, spurred on by all the bogus publishing opportunities, academics have become so specialized that they can sometimes barely talk to their counterparts, much less students or their colleagues in other disciplines...

...The price-nine mediocre researchers for every brilliant scholar-is one society had better just shoulder...

...Mutters: Achieving Scientific Literacy, recently This suggests that anything that shrinks the found that in a group of 24 physicists and geologists, amount of time and energy devoted to marginal reonly three could explain the difference between DNA search would have to be an improvement...

...Although hard evidence for such assertions is difficult to come by, there are a few indirect indications that such criticisms are, if anything, conservative...

...In the long run, however, more systemic change is needed...

...3) not obtain surprising results...

...Beyond the tip of this iceberg, we aren’t even beginning to get our money’s worth...

...Young adds that 80 percent of scientific articles could “just vanish” without affecting the scientific enterprise...

...The conventional wisdom in the field is that 10 percent of the journals get 90 percent of the citations,” says IS1 analyst David Pendlebury...

...It’s not a minor violationit’s really rather horrifying...

...By this formula, authors wishing to increase the chances of getting their research published should “1) not pick an interesting topic...

...Much of this research is very pedestrian,” says MIT molecular biologist Richard Young, who did vaccine development himself until he became disgusted with the field...

...It's easy to dismiss Richard Campbell's study of "60 Minutes" as gasbaggery if you've watched the show...

...It might seem unfair to pick on the social sciences, which have long suffered by comparison with the more glamorous and better-funded "hard" sciences (physical and life science, medicine, and engineering...

...Below, the basement holds several million pounds of govemment documents...

...But the reputation of social science as a haven for work that tells us 1) nothing we need to know, or 2) nothing that we didn't know already, has resulted at least in part from the accessibility of the subjects that social scientists like to study...

...We are selecting those with the greatest need for security, and the least confidence in their ability to hold a job on merit,” he says...

...Despite unprecedented growth in federal research budgets over the past 10 years, complaints about a “funding crisis” in American research are picking up steam...

...This argument suits legions of mediocre academic researchers just fine...

...For those of you in Washington who are unable to see the flames, 1 say, wake up and open your eyes...

...We have created a faculty of scholars frequently so narrow in their studies and specialized in their scholarship that they are simply incapable of teaching introductory courses,” William Shaefer, a professor of English at UCLA, told US...

...Our nation’s health research program is burning,” he announced at an Institute of Medicine meeting last June...

...James Trefil and Robert Hazen, two George Mason University science granting institutions now spend an average of 19.3 professors and the authors of the new book Science hours a week on research-related activities...

...In order to allow professors to bloom as teachers, Boyer makes a series of simple, yet sensible, recommendations...

...Some academics claim that the proliferation of journals and studies is merely evidence of a healthy scholastic enterprise...

...5) not provide full disclosure...

...Similarly, when Douglas Peters of the University of North Dakota and Steve Ceci of Cornel1 University found that psychology journals tended to accept articles based on the perceived status of the authors-regardless of the contributions of the paper itself-their study was rejected by the prestigious journals Science and American Psychologist before it found a home in a journal devoted to controversial topics...

...As ostensibly nonprofit organizations, universities save billions more in exemptions from taxes on patent income and property...

...That’s a pretty stunning figure-but it gets worse...

...They agreed, but said, 'Then [the authors] won’t come to the meeting!’ ’’ Association of American Universities, the WashingSimilarly, in the field of vaccinology, researchers ton representative for 56 American and two Canadian for the past 10 years have devoted themselves to making what are called “subunit vaccines,” work that has next to nothing to do with producing vaccines for real people...

...Second, the tenure process creates a need to win the approval of academic peers and tends to reinforce conventional wisdom in a field, stifling innovative research...

...And the publish-orperish principle is firmly rooted in scientific competition for grants and positions-a process exacerbated by the practice of granting faculty members lifetime apto measure the im-pact of someone’s research is through “citation analysis,” a process which basically amounts to counting the number of times a published study is footnoted in other scientific articles...

...This is not to say that academics are all stodgy, incompetent researchers who do derivative research while hiding their inadequacies with specialized jargon, statistics, and a lousy command of English...

...Just over 1 million students toil away in the nation’s graduate schools, and 34,000 of them receive doctoral degrees each year...

...The reand RNA-a fundamental piece of information in the search establishment, of course, is resistant to the life sciences...

...Wait, there's more-"60 Minutes" has the power "to transform and deform experience, to secure a middle ground for audiences, and to build unified meanings in and for a pluralistic culture...

...The public investment in research is huge-almost $16 billion in direct federal, state, and tax-deductible industrial support...

...By cutting back on the number of journals carried by libraries and subscribed to by researchers, the government can stop subsidizing the useless sectors of the academic publishing industry...

...It’s true that a person with a good teaching record and a mediocre to poor research record will not be promoted [at MIT],” says Gene Brown, MIT’s dean of science...

...But many of their colleagues are falling down on the job, or unable to do it in the first place...

...The amount of repetition is mind-numbing: anesthesiologists can choose between Anesthesia and Analgesia, Anesthesiology, and Anesthesiology Clinics of North America...

...Smoke or mirrors...

...You simply don’t get rewarded for teaching students,” says Robert Collins, an English professor at Florida Atlantic University...

...In fairness, Rosenberg’s complaints can’t be dismissed out of hand: Biomedical research costs have risen faster than inflation, and it is harder now than ever before for young scientists to get funding to start their own labs...

...Helfand, by the way, is virtually unique among academics for rejecting his own tenure offer nine years ago in favor of a five-year renewable contract [see “I Turned Down Tenure,” June 19861...

...After reviewing the research on scholarly publication, University of Pennsylvania marketing professor J. Scott Armstrong observed several common themes that led him to devise an “author’s formula...

...The resulting mediocrity of the professoriate goes a long way toward explaining the generally dismal state of academic research...

...But it’s not exactly a guide for a defensible public policy...

...Nobody noticed for almost three years, and even then the fraud was discovered only because Alsabti asked a colleague to review a paper that still contained clear references to the real author’s identity...

...Originaily intended to protect the freedom of academics to study controversial topics, tenure has devolved into a sevenyear review of junior faculty members that encourages useless publication in three ways...

...By removing the incentive to publish as many papers as possible in the traditional seven-year race for tenure, tenure committees would encourage professors to concentrate on producing only the best papers possible...

...Autonomy, Interdependence, and Social Control: NASA and the Space Shuttle Challenger" (Administrative Science Quarterly), a 32-page dissection of the NASA mistakes leading to the Challenger accident, after which Boston College professor Diane Vaughan concludes that "this case study does not generate the sort of comparative information on which definitive policy statements can be made...

...Why do so many unremarkable articles-which usually reflect equally unremarkable academic studiesend up in print...

...And even IS1 indexes such outofthe-mainstream journals as the Annuls of Saudi Medicine, the Ethiopian Medical Journal, and the Journal of the University of Kuwait-Science (whose publication is, presumably, temporarily suspended...

...Richard Young agrees: “I frequently have to go into the ‘deep literature’-those journals I no longer have time to read on a daily basis-and it is often a waste of time...

...Stanford is also currently under investigation for inflating its reported costs-less an argument for cutting back such reimbursements across the board than for letting federal auditors keep a closer eye on the money...

...It’s not widely known outside the research community, but universities actually collect a fair amount of money from these grants through indirect “overhead costs” ostensibly related to utilities and upkeep of research facilities...

...If universities really want to do something about improving the quality of research, they had better pay attention to suggestions like Boyer’s...

...Larger increases in federal research spending are politically unlikely in the future...

...Despite lip service to the importance of teaching, few academic departments these days really consider teaching ability to be on the same plane as grant-getting and publishing...

...By contrast, faculty at research and doctoratewhat’s the best line of inquiry...

...And although the success stories of academic research are undeniable-advances in computing technology, a variety of medical treatments ranging from vaccines to cancer therapies, and a better understanding of the geophysical forces that cause earthquakes, to name a few-these successes, and the foundation of basic research that has made them possible, represent only a tiny fraction of all the research performed each year...

...And even that isn’t the end of the story...

...No one’s come up with a better system yet...

...But it is clearly most out of control in the medical sciences, where if a person doesn’t have over 100 publications listed in [his] biography you think there’s something wrong with [him...