Peer Review 2.0
Peer review is an absolute necessity for recognizing good science and rejecting the false, but the traditional method of journal based peer review is not keeping up with modern needs. The more prestigious journals are more concerned with the potential a paper has to enhance their impact factor. With so many papers to choose from they can happily reject many, not because they are wrong but because they are not sufficiently mainstream to attract quick citations. When they do accept they place the final version behind a paywall and charge the taxpayers who funded it $30 to read each paper. Is this right?
Different areas of science have different needs and the resilience of journal based peer review can in part be attributed to its flexibility. The needs of maths and physics surpass what the journals offer and as a result the peer review process has been largely replaced by internal reviews, submission to open archives. Where work is more theoretical and speculative the journals do little to decide the validity. This is determined by citations, open discussions and further research leading eventually to experimental tests (we hope). But even here the journals have not disappeared. They remain because students and postdocs need the official stamp of approval that the journal offers in order to move to their next job. Can this role be replaced?
The role of open discussion on the web is surprisingly controversial. In a recent post I queried a response to a question put to Brian Cox and Jeff Foreshaw in the Guardian. They were basically saying that blogging about science that is not yet peer-reviewed undermines the system. a littler later there was a similar article in the Guardian itself in which astronomer Sarah Kendrew defends blogging. But Cox and Foreshaw are far from isolated in their opinion. A link from that article leads to an interesting story about a question in a course about “Responsible Conduct of Research”. The question was as follows:
A good alternative to the current peer review process would be web logs (BLOGS) where papers would be posted and reviewed by those who have an interest in the work, true or false?
The correct answer according to the course is false. Lose a point if you thought otherwise. Well it is indeed the case that blogs alone cannot replace the current peer review system, but they are becoming increasingly important in discussing and judging some questions. Could it be possible to construct a system of peer review based on open web-based appraisals that would replace the journals? Nearly a year and a half ago I asked this question and suggested that a system based on something like stack exchange might be possible. It would not be easy and one thing is clear: It would have to be backed by people with more clout and credibility than me.
Happily some people who do have that kind of clout are now starting to think of the same idea. In particular Tim Gowers has been asking similar questions for peer review in mathematics (see here and here) As we have seen above, such a system is likely to be highly controversial as well as difficult to put together effectively, but at least it is starting to be discussed by people who matter. Mathematics is an area where it might work most easily because correctness in mathematics is very cut-and-dried. This is one reason why MathOverflow has been so much more successful than Physics Stack Exchnge. But as I said earlier, journal based peer-review holds its place because it is so flexible. To replace it we need a web-based peer-review system that can work across all disciplines.
<script src="http://pagead2.googlesyndication.com/pagead/show_ads.js" type="text/javascript"></script>
Maths has The “Mathematical Reviews” and the “Zentralblatt”. The physicists (or other sciences) equivalent of these public peer review of papers never worked.
Back in 2005-2006, I run physcomments.org, you can go to internet archive if you are curious. My first approach was to colen MathReviews approach kind of electronically: I analized the bibliography of each eprint, selected from it two authors likely to work in the same topic at with a similar citation or h-index that the eprint authors, and emailed them asking if they wanted to volunteer a public review, or suggest other reviewers.
The answer of the arxiv was to cut the server machine of any access to email information; I have had the mistake of not asking to arxiv admins, and when I explained the purpose, they told me to do not use the arxiv as source of reviewers email addresses. Ok, then I asked SPIRES if I could use their HEPNAMES for this purpose they gladly allowed me to do it… and next week arxiv admins though that I was pirating somehow some backup server and asked my university colleagues to cut my access to computers in the physics department… which they did, without asking me, and driving me near to nervous breakdown. I am sure that even today, some of my teachers and colleagues believe that I pirated the servers.
After this, I abandoned the idea of requesting reviews and the server evolved to a friends-only site, just a blog more, as you will notice if you browse across the site. Amusingly, I think that some renewed administration in the arxiv even allowed for trackback pings, but at that time I had completely lost the momentum and the server was not seriously cared of.
(sorry the cut-paste-edit typo, read “My first idea was to clone MathReview approach, but electronically:&rdquo
That was a good attempt but it is hard to get support for such a project. Another one that has struggled is http://scirate.org.
Ginsparg’s own version of a proposal for a new peer-review system (2003) can be found at http://people.ccmr.cornell.edu...rg/blurb/pg02pr.html
Yeah, it makes even more puzzling why Ginsparg did not attempted to contact me more smoothly, say via the same common friends he -or his sysadmin- got to influence to cut my access to my alma mater computers. I was practically volunteering to implement MathReviews for free, and it was clear that there was more points of agreement than disagreement. Perhaps he changed mind, really, after exposing their ideas in 2003. Perhaps the arxiv had open some conflicts with editors (think, the $30 to read each paper), perhaps bad memories of other parthners, perhaps some other big plan. Well, I have not the record of exchanges here just now, but I can not recall any sign of encouragement in the emails from Cornell.
Another piece of the puzzle is, perhaps, JHEP and its inception from babbage sissa it.
Interesting thoughts, but you are missing much of the harsh reality. I am now being censored for having posted on viXra, the justification being that it proves I am a crackpot.
This was not about posting on viXra, but it is an interesting point. Of course there is no reason to think someone is a crackpot because they post on viXra but people who post here face that prejudice.
I have been getting the impression that arXiv are making their submission criteria more stringent. There have been cases where they have asked authors about their academic credentials before making a decision. Where in the past they would move endorsed articles to graveyard categories if they don’t like them they are now rejecting them outright.
This makes viXra necessary for more people, but institutions forbid their members to post here and others are put off because people assume that anything posted here must be not worth looking at. Never mind, if peoples work is worth reading it does not matter where it is posted so long as people can find it. viXra submissions are growing to the point where I have now implemented a form-based submission system that will allow us to process the requests much more quickly. viXra is flourishing despite its detractors.
You really sound like an innocent 5 yr old, that has no idea what serious censorship is like. Of course it matters where stuff is posted, especially if the horrific lack of professional ethics results in outright bans of material whose content does not get considered until years later, when some professional claims it as their own, so that the system can be perpetuated with ‘justification’.
A thiking… can we assume safely that nowadays every idea on physics, nor to say in science, is nowadays available in the internet ecosystem? If it is so, I’d judge that the “arxiv-less” is a lot smaller sector that the actual “referee-able” production anyway.
As usual, nobody actual listens to me. I’ll give another example. I used to have three or four papers on the arXiv. There is now only one remaining.
Sorry, that’s not quite true. They are still there, but not quite as obtainable as before.
You do a great job Phillip, thank you.
Looking to the future, when vixra becomes an accepted mainstream point of first communication, will it also become restrictively focussed on preserving the orthodox?
How will you prevent the ossification that has arguably happened to arxiv?
Some of what happened at arxiv started over a noticable increase in aether theory based articles that a few of us who used to post there had noticed at the time things changed. I am not really sure if the change and that actually where related. But it seemed like the organization started to close ranks at about the same time.
Personally I rather support the whole peer review process even if I suspect it sometimes degrades into if you are part of the accepted status quo or not. Politics of one form or another always rules. Several friends I used to know at NASA often remarked I had good ideas, but tended to forget they where politicians themselves in many ways and had top deal with the system as it was. I am glad this archive exists. But I do hope the fact that it is outside of the more established one’s does not get everyone who posts here labeled a crackpot.
My opinion is that it will take a long time for people to let go of their prejudices against viXra. After all, it is not unexpected in times like ours when establishment has strong political influence and outsiders are often being discarded “by default”.
But this unfortunate state of affairs should not discourage anyone to post at viXra. It is crucial for the development of science that new ideas/models are recorded in a truly open and unbiased repository.ViXra will continue to gain acceptance as time goes by.
Ervin
Yes, although Phil says this post is not about viXra, it really is, because the peer review system is so corrupt and the politics so Orwellian, that viXra is almost the only honest and stable preprint server in existence.
A scientist guards his or her re****tion almost as closely as they guard scientific objectivity (sometimes even closer). Peer reviewed journals consist of peer reviewers whose participation satisfies a consensus that is usually established within the academic community. As a result there is an inevitable amount of “groupthink” that occurs during the process. Even Einstein experienced this when he published his special and general theories of relativity. Any theory that is sufficiently outside of the “main stream” of scientific consensus will be criticized and justly so. Extraordinary claims require extraordinary proofs. The amount of “fringe” explanations for such things as dark matter or the OPERA experiment results has clogged any review process that could result from blogging. Someone once observed “ninety five percent of everything is trash”. An important function of the peer review process is to separate that ninety five percent from the five percent that is actually good science. A process that performs that function within the blog-o-sphere remains to be designed but it would be a worthy goal and a powerful tool.
Criticism is healthy debate, yes. But outright deletion and murder are not.
Kea,
I understand how frustrating must be for anyone to get rejected just because he/she posted at viXra. Don’t get demoralized though. You are not alone in this unfair fight. I maybe naive in my thoughts but I believe that, if your work is sound, it will gain attention some day and electronic track records will give you full authorship rights.
Cheers,
Ervin
Ervin, my work has already gained attention, only under other people’s names. I do not speak from a lack of experience.
To be clear: the electronic record is being altered.
I sometimes wonder whether that statement, ‘Extraordinary claims require extraordinary proofs’, is not becoming a bit overworked. Perhaps even a tool for defending the status-quo, do you think?
For example, some of the claims of fundamental physics really are extraordinary -wave particle duality for a start, and the many-worlds theory for a follow-up. There are no end of referred articles based on this type of concept, but just because they are mainstream somehow the requirement for ‘extraordinary proofs’ is waived.
As a strategy for the epistemic growth of a body of knowledge, that is problematic.
In psychology they have a name for this: confirmation bias!
So, effectively what that famous statement REALLY means is ‘Unorthodox claims require exhaustive proofs’.
Confirmation bias, yes, that’s it. As scientists, we would not argue with the need for exhaustive proof. But as you say, this is not being applied to accepted ideas, even after they have been esentially falsified. Science has become dogma.
Actually, the phrase reads “Extraordinary claims require extraordinary evidence,” coined by M. Truzzi and popularized by Carl Sagan. Truzzi later expressed wishes that he’d never created it, since it becomes a recipe for so-called “moving the goalposts” for the threshold of acceptance of discoveries which upset the status quo.
What’s the rigorous definition of “Extraordinary” xlaim? We cannot say, since it’s entirely subjective. Each of us has our own personal threshold where claims cross over from ordinary. Extraordinary may mean “violates well-tested physics,” or perhaps “goes against my unexamined assumptions.” Or it means “supports a theory which I personally dislike.”
Also Truzzi’s phrase lets us select evidence. We can refuse to accept any evidence for a claim, on the grounds that it’s not sufficiently “Extraordinary” At the same time we are maintaining our own hidden definition for what is “extraordinary” and what is not. It’s an excellent way to hide our own irrational subjective bias while at the same time appearing to be reasonable, even properly scientific.
His phrase also leads to circular reasoning: a claim cannot be accepted …because the evidence isn’t extraordinary enough. A claim is widely held to be ‘extraordinary’ …because it’s only supported by weak evidence (insufficiently extraordinary evidence!)![<img src=]()
" class="wp-smiley" src="http://s0.wp.com/wp-includes/images/smilies/icon_smile.gif?m=1309109404g" />
Workers in the disparaged field Parapsychology have for years been saying that their good solid evidence, evidence of higher quality than in all other fields, is being ignored and dismissed. The reason given? Extraordinary claims require extraordinary evidence …with the unspoken appendix that no evidence can ever be extraordinary enough to convince anyone that PSI is real. It’s data-selection in action.
Yes, thanks William, that was interesting and well-presented. You put it better that I did. As you say, the statement becomes a means to disparage the unorthodox.
In science, a really good novel idea that helps us to understand nature at a deeper level (eg, fractal geometry) will always overcome the prejudice against it and rise to its deserved status, given sufficient time.
In science, a novel idea that is enthralling but unlikely to have much connection to the physical world of nature (eg, string/brane theory) will always run its course and begin to look distressingly Ptolemaic, given sufficient time.
There are many different forms of peer review, including the gauntlets of the blogosphere, arxiv, viXra marketplace. It is a very imperfect and remarkably inefficient system, but science cannot prevail over pseudo-science without it.
Best, Albert Z
Well said!
Ervin
Really it is nature and time that ultimately decides the truth from the chaff. Brane theory may be novel in that it relies upon an endless set of solutions, one can postulate simular with different manifolds in our 4D space-time. But it is rather ptolemaic in many ways.
Fractual geometry and or chaios theory does seem to have an internal beauty and consistancy. It also comes closer to Einstein’s dream of a fuller geometric treatment of the forces of nature.
Nice reply!
The temporal issue is of course that ‘sufficient time’ might be longer than a professional career.
> In science, a really good novel idea that helps us to understand nature at a deeper level (eg, fractal geometry) will always overcome the prejudice against it
Cite?![<img src=]()
" class="wp-smiley" src="http://s0.wp.com/wp-includes/images/smilies/icon_smile.gif?m=1309109404g" />
I’d counter your assertion with a quote by WIB Beveridge:
“Many discoveries must have been stillborn or smothered at birth. We know only those which survived.”
Because many unconventional ideas experience extreme barriers to acceptance, this strongly suggests that some were suppressed entirely. Frequent near misses are the tail of the distribution of major automobile accidents. But if many good novel ideas are suppressed entirely, and they are assumed today to be pure crackpot, we cannot know this is happening. It’s because “Vanished” ideas take their evidence of suppression with them when they vanish. All that remains is a vast swamp of obvious crackpotism. Nobody knows if it conceals one major new physics idea, or hundreds. There may be diamonds in the sewage, but anyone touching it becomes contaminated, so nobody searches for them. (EXCEPT PHIL, VIA CROWDSOURCING THE TASK!)
@ Kea,
If your claims are true, why don’t you publicly denounce the person/persons who altered or plagiarized your work? This amounts to academic misconduct, to say the least…
Ervin, the internet makes anonymity possible. I do not know which persons are truly evil.
And when misogyny (racism, etc) is the accepted norm, there is really no point in writing a list of names. I definitely notice the exceptions.
I cannot but agree with Kea about the deep corruption of the academic world. It is incredible how unashamedly people with an academic position are lying.
The reason for the corruption is simple: academic researchers are not anymore explorers of the unknown but builders of a personal career.
Every scientists must make the ethical choice: genuine research or a career doing what you are ordered to do.
The first choice means a lost career, impossibility to communicate through academically respected channels, continual ridicule, endless problems with basic living, and so on. In this kind of situation it is only the knowledge that you are doing something that really matters that brings the satisfaction.
One must make a conscious decision of giving up all hopes about success. This is difficult since only “success” is what matters in our society. Successful mass murderer gains the same media attention as a successful artists or scientist and becomes a rich man if he realizes that tabloids are ready to pay almost anything for an interview!
What makes me however optimist is that communication is possible. One can have a blog and homepage and there are even archives like viXra. At least at this moment the complete silencing of scientific dissidents is not possible as it was for some decades ago: without internet no-one would had heard of my life work. I really believe that the ultimate winner is truth – even in recent day science.
The problem with the internet in general is that at times there are too many ideas a float. I regularly vist some of the so-called science replies on places like Yahoo, the New York Times, etc. Half the comments come from people who know nothing about real science or people with a religious drum to beat. But what I do notice in all the chat is a tendency to get their understanding of science and even nature out of the dissident side instead of the mainline.
What that tells me is there is a real disconnect between the real scientific world and the general population. While I would not expect the average Joe or Jane to be interested in all the math say in a published article I would expect that somewhere in all their early education they would have at least been explained basic science 101.
That leaves me with the big question about the backgrounds of those who do find their way into real scientific study. What I suspect is some of them come into the real mainline with some pseudo science ideals they then attempt to find support of in their research. Let’s say one of those just happens to do an article that is veiled enough in the right math and words to pass Peer Review. Now we have pseudo-science dressed up enough to pass as real science and the whole disconnect gets amplified and passed on.
If one will remember a bit of history, before Einstein and others laid the foundation for realitivity Newton’s Aether held sway. In some way what is now considered truth was at one time the crackpot fringe.
I think what really is needed in the world is a better explination of science at the lower levels that is itself subject to peer review of some kind. What I come across more and more is that the general public tends to speak louder about the fringe ideas than more solid ideas and these are the people through Taxes, etc that tend to eventually control the money needed for research in the first place.
It’s very easy to get drivel past peer review – one just needs the right multiverse buzzwords, and to cite the right people, and the paper can contain pretty well any half baked fairy tale.
Yes, and one only has to look at the ‘Conclusions’ section of physics papers, to see how poorly scientists communicate the implications of their results to practitioners.Good communicators to their academic peers, poor to practitioners, and hopeless to the general public.
The first choice means a lost career, impossibility to communicate through academically respected channels, continual ridicule, endless problems with basic living, and so on.
Yes, that’s right. The trouble is that even here on the viXra blog, the cool dudes don’t believe us.
And you are one of the worst misogynists in this crowd.
Well, that was really amusing![<img src=]()
" class="wp-smiley" src="http://s0.wp.com/wp-includes/images/smilies/icon_smile.gif?m=1309109404g" /> Of course you have an impact, Kea.
When you talk of blogs doing the peer review, then there are blogs and blogs. Like Science 2.0. We would need to know the policy behind those blogs too? The problem is to create a neutral blog.
In the end, nature is the ultimate, and only truly reliable, referee.
Best, Albert Z
Of the truth, yes. But in an Orwellian world, Big Brother owns the truth.
Sometimes the truth out of nature gets ignored in favor of tradition and dogma. One could call that Big Brother at work. But not everything is killed by the establishment either. Warp drives and time travel got over blown by the external media hype at one time to the point as one honest researcher and published article writer I know of at the time put it there was political pressure employed on him and others to avoid the subject. Sometimes Big Brother is controled itself by outside influences.
With respect to the Cornell arXiv, Brian Josephson (Cambridge Nobel Laureate) had email discussion this month (November 2011) in which:
David Ruddy of Cornell said: “… we do not believe that arXiv is blocking the transmission of any scholarship or stifling innovative ideas, primarily because arXiv is not by any measure the sole means for communicating scientific ideas. You have available to you many other channels by which you can communicate your ideas to the quantum physics community. …”.
Brian Josephson replied: “… Arxiv facilitates the transmission of ideas through a combination of two factors: avoidance of delays due to refereeing etc. (the original purpose of the arxiv), and breadth of readership. As regards the first, I accept that there are good alternatives (e.g. our university’s dspace archive). But such alternatives fall down badly as regards breadth of readership; far fewer people browse archives such as dspace@cam than routinely view postings to the area of arxiv in which they are interested. Conversely, media such Phys. Rev. Letters have high readership but are subject to publication delays, and in any case have space only for a fraction of those papers submitted to it that are of adequate standard, so cannot be seen as a viable alternative in the majority of cases. Unless the arxiv administrators can suggest a channel that is similar to arxiv in all relevant respects, I shall be forced to regard the statement quoted above as a dishonest one, of the kind one expects from politicians but not from an institution such as Cornell. …”.
Tony
PS – As to Cornell’s use of endorsers to validate arXiv submissions, I know of one paper (in astrophysics) that Hans Bethe endorsed recommending acceptance, but the paper was rejected by the Cornell arXiv on the grounds that Hans Bethe was not qualified to be an endorser.
PPS – No, the paper was not written by me.
PPPS – In case anyone does not know, Hans Bethe was arguably the leading figure in establishing the stature of the Cornell physics department. As the web page at bethe.cornell.edu/about.html says: “… After the war, he brought some of the most outstanding young physicists from Los Alamos to Cornell, in particular, Richard Feynman and Robert Wilson. Under their leadership, Cornell moved into what is now called high energy elementary particle physics, a field in which Cornell remains on the cutting edge. That Hans Bethe has devoted virtually his whole career to Cornell has been of inestimable value to the Department and to the University. …”. Nevertheless, Ginsparg et al had the power at Cornell to disrespect Hans Bethe by rejecting his attempt to be an endorser.
viXra.org is an example of an archive that is similar to arXiv.org. Scientists like Josephson could post here but choose not to because they want unrestricted access to arXiv. It is fine to want such access but he should not be saying that alternatives do not exist when they do.
The stats show that a paper posted here gets just as many downloads on average as one posted on arXiv. Papers can easily be found once they are on any open web service and will be found by people interested in the areas they are relevant to.
I think it is really just a case of: “I don’t care to belong to a club that accepts people like me as members” (Groucho Marx of course)
Brian speaks of “breadth of readership.” I am not sure how to take on this. An argument could be about simplicity, you do not want to keep scanning a lot of different preprint sources from all the public archives. But this could be solved by offering an RSS agregator service.
Another question that Brian doesn’t raise is about long term survival of the preprint database. (I wonder if people was similarly worried about KEK scans). And other point could be added services (citation analisis, etc), but here the king is SPIRES and they allow for different preprint sources,
viXra and arXiv both supply RSS. Ginsparg had grants to develop methods for helping people find papers of relevance to them, but nothing useful has come from that. The reality is that people find papers by lots of means including Google searches, citations, seminars etc. Browsing archives is probably a small part of it.
SPIRES only covers HEP. Both arXiv and viXra have a wider scope than that. SPIRES was indexing HEP papers in viXra for a while when it was run by Stanford, but SPIRES has now moved to CERN who have a much less tolerant attitude to independents and we are no longer added.
There are good professional indexes such as citebase but they are run by the journals and are not supportive of papers outside their system. For now the best place to go for open citation indexing is Google scholar which does include many papers from viXra, but not all. Mostly it includes them if they have been cited.
viXra is committed to remaining around longterm and does not depend on just one person to exist. There are independent mirrors to make it robust.
As to “Scientists like Josephson could post here but choose not to because they want unrestricted access to arXiv.” that is not true because Brian Josephson does NOT have “unrestricted access to arXiv”. His access was restricted (euphemism for being blacklisted) well before viXra existed, (perhaps due to his views on parapsychology and/or cold fusion), and it may be that he does not understand how widely viXra is read.
Maybe you could do a post showing in detail a comparison of arXiv and viXra in that respect (particularly with respect to readership of people who are at major centers of influence) because there are may be a lot of people at institutions like Cambridge who think of viXra as a small newcomer that is not widely read and a detailed showing to the contrary might influence them.
Tony
PS – As to my personal opinion, I am very happy to post in viXra nowadays (unlike my very unhappy earlier days of being blacklisted with no viXra) and my guess is that my papers on viXra get at least as much readership as they would had they been posted on arXiv (maybe even more).
Tony, I think you misunderstood what I wrote about Josephson because your objection does not relate in any way to what I meant. Sometimes the way I write things does not seem to communicate my thoughts correctly to other people. I am too confused about how you could have interpreted it to give any clarification, so let’s not worry about it.
I already posted about some comparison of readership at http://blog.vixra.org/2011/06/...papers-at-vixra-org/
I’m glad you are happy to post at viXra.org.
Yes, academic disciplines are priesthoods. Academic physics is a priesthood in a philosophical sense, a cultural sense, and especially from a psychoanalytic viewpoint.
Just think of the authoritarian and patriarchal milieu in the classroom, or the alternative mother complex mysticism sometimes presented. The reasons for human anti-intellectualism are psychoanalytic, I think.
Try to notice how the initiations and taboos emerging in the discipline are not so different from aboriginal practice.
There is a devotion in academic physics to mathematics that is arbitrary in its foundation. This devotion is priestly because such arbitrary mathematics qualifies by philosophical standards as a deity.
Mostly true. I’ve spend many an hour pondering what sort of science some alien race or alternative reality might have developed had the path dependent process of science played out a little diifferently.
But the definitive predictions/empirical testing part of the scientific method separates the wheat from the chaff, given sufficient time and patience
In spite of those who promote dubious science because of selfish concerns, and in spite of those who feel it is their appointed duty to destroy any new idea before it can upset the status quo,and in spite of the thundering herd who follow along like good sheep, scientists and nature will reveal the “truth” through observation.
That is the beauty and power of science.
Best, Albert Z
Human progress is not inevitable, as history and evolution teaches us.
One could be asking the question how evolved is mankind. On some fronts we’ve evolved a lot. But on a lot of fronts we still tend to act like spoiled children. Human progress is more one step forward and one step back intermixed with times of no real progress, a little bet of sheer terror, some acts of kindness, and a lot of searching around in the dark.
Extinction is not something I would describe as One Step backwards.
Personally, I am not a subscriber to extenction unless it is by natural means.
On a side note, dual time frame ideas have surfaced a lot before in quantum field theory. In fact, one lady starting with Bell’s ideas and others with Bohms have each derived a dual frame. One is in published format several years back. The basis of your ideas is not that unmainstream even though it is often ignored.
In fact, and Fernando Loup could back a bit of this up, at one time while some of us worked on the old Alcubierre idea we had looked at twistor theory and came to some of the same conclusions.
Before you cry a river, think of the season and be thankful. You could be a lawyer.
American law professors have several hundred journals, and all but a handful, and all but one or two of the most prestigious, are run by law students who know almost nothing about the material they are editing except for grammar and footnote formatting.
Law review articles pretty much have to be 70 pages long, no matter how much or how little you have to say (and it is usually the latter), and your career hangs in the balance of somebody else’s students.
From the perspective of the academic discipline as a whole, it is even worse. Almost half of all law professors have never practices law for a single day. The mean level of on the job experience is one year. They’ve never tried a case to a jury, never handled a corporate merger, never gotten someone divorced, never written a will for a dotty 80 year old with a 30 year old lover. Imagine what life would be like as a physics scholar if your papers were reviewed only by bottle washing undergraduates who’d never seen the insider of a particle collider and didn’t know a Lagrangian from a Hamiltonian.
On the plus side, it isn’t corrupt. I have yet to even hear of an article being published for sex or money, although political agendas are rather more promiscuous. On the minus side, if it isn’t sexy, it doesn’t sell. Journals are clogged with constitutional law and feminism and have little to say that is relevant to practicing lawyers or is even relevant to policymakers.
Imagine what physics would be like if the only way to get published in a good journal was to write something that could sell in the trade press – Carl Sagan style. Not that popularizers aren’t great important folks that inspire generations to come (and even good physicists frequently have prose styles that make lawyers and English professors cry), but anomalous azimuthal muon angles aren’t amusing to the average half-educated college student.
Honestly, I think the discipline would be best with something modeled on the ranking system for chess masters. A community open to all, but where the most elite hold the keys to advancement (which actually, is not so far from how the PhD advisor and grant/postdoc system works already), and the highest reward would be praise from those who have already proven their own expertise concretely.
Imagine what life would be like as a physics scholar if your papers were reviewed only by bottle washing undergraduates …
They might get published? And the idea that feminism is not highly relevant is grossly insulting and misogynist. You have no idea at all what I am talking about.
The idea of an untouchable Boys Club elite is a big part of the problem, in many disciplines.
Both constitutional law and feminism have a place in the academy. But, the fixation on accessible topics dampens the possibility of publishing articles on myriad other technical subjects from banking law, to real estate easements, to business valuation, to evidence questions, etc. that are less accessible.
Peer review is supposed to improve quality control and provide ways to improve quality prior to publication. I’m well aware that you feel shut out of the academy and marginalized in that academic world, and that you attribute a big share of that difficulty with gender. Having been in and around academia most of my life, I disagree that gender has any meaningful effect on peer review or the publication process of its own force in the sciences which also, for example, are very even handed with a population that ranges greatly in nationality, ethnicity, age, and even ability to express oneself grammatically in written English.
Academia is very hard on people who try to publish without an affiliation to an academic institution, and that does disproportionately impact women who are more likely to spend some time out of the work force and are more likely to have less flexibility to uproot one’s life to chase positions, although I know of female scientists in industry in Boulder (near my Denver home) who have managed multiple journal publications. But, most of the sorting by gender happens in the pipeline to earn the PhD, not afterwards.
Conversely, academia is probably too lax in allow publication of dubious work from people with well established academic credentials at an institution. Almost every academic journal in almost every discipline is stuffed with unimpressive rot that is absurd, irrelevant, misguided, or trivial.
In theoretical physics, there is lots of really bad stuff floating around the Internet written by self-proclaimed experts in non-peer reviewed settings. Then again, there is also enough off the wall absurd speculation with only the thinnest of empirical hooks in the peer reviewed published material to suggest that perhaps we ought not judge the work of those outside academia too harshly.
The gender effect on peer review is a proven fact, moron.
I was the initiator and most frequent commenter on a thread about Kea’s work at a major Internet physics forum, and I can confirm not only that the thread was summarily deleted after eight months (and several appeals for its reinstatement, even in a “locked” mode, were rejected), but that the moderators explicitly do not want material from vixra discussed on their forum. Their official policy is that only published work should be discussed; they will allow papers from arxiv; but vixra is too far. They acknowledge that this policy may prevent the mention of some genuine discoveries, but consider this a price worth paying in order to maintain overall quality of discussion.
http://sheppeardnotes.wordpress.com/the-lost-thread/
Again the vixra forum? Where is it?
“vixrapedia” would be interesting. Someone could create vixra.wikia.com right now, but wikia sites are full of advertising.
And we don’t like advertising, so no wikia? Just a forum where we can discuss? Like the physics forum?
Anonymity of referees is part of the wider problem with review. Referees can readily impose their own agendas, either to advance their own work and that of their friends, or restrict that of others. Anonymous review does not achieve what it was intended to, because it naively assumes upright and ethical behaviour of all participants.
But the reality is that the whole journal industry has grown so large, that publishers can no longer vouch for the personal integrity and freedom from selfishness of their reviewers. So it really is a quality control problem that the industry has brought on itself. Clever and ambitious people use the flaws in the system for their own selfish purposes, and anonymity has become a useful cloak to hide unethical behaviour. The publishers and editors are unable to detect or treat the problem.
The current system has become subverted by selfishness.
Given the size of the publishing system now, it would be better if all referees’ proper names were revealed.
Say, a situation where those anonymous referees are bought by the industry (usually energy) or government/military, how ‘ethical’ and good system is that? Why can the reasonable minds of the scientists themselves not be trusted?
And today the delay from the review means a lot. It can be used to delay someones work, just as in old days when Poincare’s work was delayed a year, when Einstein published his 1905 paper. There a Nobel Prize was lost to the review system?
The fact that viXra get reactions only tells about its success. Be proud.
Referees are anonymous but editors are not. And ultimate responsibility for decision is in the editor. The referees are just extensions of the editor abilities.
The issue here is that most editors blindly follow referee’s recommendation on manuscript acceptance or rejection. Only a handful of editors bother to dig deeper and bypass referee’s objections. The bottom line is that, most of the time, referees end up having the power to decide.
It doesn’t matter at all if they are anon or not in this case. It should maybe be better they have a name?
A side not on arxiv in general, tried to run a check on the article New varying speed of light theories, the one written by a man who started the whole VSL comology approach. Interesting enough, at least from my end the article showed up as locked and forbidden on the server. I have been viewing some of the conversation going on about the whole arxiv requirment to be sponsored(sort of a mild form of Peer Review)they employed and I am beginning to suspect it is degenerating into a sort of political motivated censorship. Funny thing in all that there is a lot of garbage being written by those sponsored in. So the whole attempt at setting up a fence against crap theories and papers does not work unless you define crap as anything from those outside the main institutes.
If this is the case at one organization I suspect it occurs at a lot of the rest of them. Given that, not only are places like this one vital, I would suggest that perhaps it is time someone established some decent alternatives to the major journals run by these establishments.
Now you mention locks and firewalls, people perhaps does not remember that the name was not arxiv, but xxx (honoring the XYZ and XXX models). Populat legend says that one reason to change the name was that some campuses and companies started to block access to URLs containing the string xxx.
Thinking about, all the problems of censorship that we have got in the “Peer Review 1.5″ phase seem to boil down to editors, and particularly to lack of political abilities, or more precisely the failure of such abilities when they must to cope with internet amplification.
Tony comments, above, and the recent coincidence of meeting Castro in the cafeteria of my campus (he was having lunch with my former thesis advisor) have made me to review the incidents in the year 2000 and I am very surprised of how, accidentally, all us the arxiv crackpots were interlinked. Yes, it was also at that year, and only for a time, that my more extreme papers started to be moved out of hep-th to physics-gen. Surely the arxiv editor felt that he needed to prove that he was acting with equal rules, and if they were aware of my indirect connections (someone on utexas could had been, for instance), they have even more arguments to try to show a consistent approach to rejection. And even it seems that the start of the history is editorial issues in two Elsevier journals, Chaos etc, with a editor inflicting self-publishing blame, and NPB, with other editor mismanaging the work of Castro, and then some of all this interfering with the arxiv.
Hmm, clarification: “my campus” is Zaragoza, not utexas. Carlos was here, paying a visit.
Castro’s story reinforces the fact that there is excessive “behind the scenes” political maneuvering in peer review and science publication. The public at large is generally unaware of this harsh reality.
Yes and not. Perhaps the maneuvering was just initially Elsevier internal, aiming to fight an editor who was actually self-publishing . Nobody doubts that something some actions was needed about Chaos &c The colateral victims, ie near collaborators and nearby research topics, had had a minor damage in a “1.0 world”.
Alejandro, I fail to see how the need to deal with one or two unethical editors justifies ruining the careers of dozens of other people.
More of an over kill situation. But, as with anything else follow the money trail.
Kea, my point is that the situation was managed incompetently, and damage happened because the internet, with its mix of public resonance and lack of personal contact, amplified the effects. An overkill, as Paul says.
To be clear: it is an explanation, not a justification. I know, explanation and justification as a pair have an status very like correlation and causation. But be sure that I am not justifying the damage. I even got a collateral hit of the shockwave.
To me, most editors are like two entangled particles. Change in editor A’s polarization imidiate cause an opposite change in editor B’s polarization. Even though they sit separated, one in Canada and one in Florida, they got entangled in one and the same wave when they physically interacted in the same career/power coupler in the past and then kicked away in opposite directions. They now share the same wave. The wave moves with light speed. But a random change in the polarization of editor A immidiate change the polarization of editor B in the opposite way. That is just the way the universe work for waves, they moves with light speed, but when randomly altered the hole wave is immidiately altered, this is not spooky at all, pure logic. But of course the editors must do anything to preserve their one and the same entangled wave. If unpredictable waves from Vixra is able to make a touch to observe whats going on with editor A then the whole wave will imidiately be destroyd and we will imidiately know whats going on with editor B.
Or ???
There is always corruption with power (Lord Action: all power corrupts&hellip
, but it’s also a more subtle “fashion” problem. Niels Bohr to Wolfgang Pauli (regarding Pauli’s nonlinear field theory of elementary particles), Columbia University, 1958:
“We are all agreed that your theory is crazy. The question that divides us is whether it is crazy enough to have a chance of being correct.”
This is the key problem: all the “sensible looking” ideas have been checked many times over and failed, so you have a situation where the anthropic principle tells you that – if there is any explanation – it may well look crazy, simply because all the “sensible” looking ideas have been long since exhausted. How do you then sort out good-crazy from bad-crazy? It’s quite a job for peer-reviewers. This was the problem when Aristarchus of Samos’s solar system (250 BC) was so unpopular it disappeared with the Library of Alexandria (however, Archimedes objectively summarizes Aristarchus’s main points in his “Sand Reckoner&rdquo
. However, if they want to be peer-reviewers, they should be honest.
One problem with the crazy ideas is that usually they have already been tried and filtered out. So, while a junior peer rejects a crazy idea because it does not fit with the current trends -and so it is unproductive for PhD, promotion, etc- a senior peer will reject the idea because he has a faint remember of having been tried. In fact, it is easier to address this second problem than the former one, as one can use the introduction to prove that the idea was really not fully baked when it was first proposed, or that it was not really analysed in depth, or that new evidence can now be argued supporting the idea.
A problem of the secret referral is that usually this senior peer does not remember -or is even ashamed of remembering it- where and how the idea was first proposed, so the rejection is even more obscure that in the case of the junior peer.
My two recent works suffer of this kind of problem. Koide formula is deeply rooted in the ideas, circa 1977, to produce Cabibbo angle from discrete symmetries. And the sBootstrap is really the first approach that string theoretists did, in 1971, to the fermion sector of Ramond, suggesting that the string was still the mesonic string, but it could be that Ramond fermions were quarks. On other people research, it is remarkable that Nottale prediction of electron mass is also published in the book of Polchinski, presented as a simple argument for the correction of the mass of the electron when gravity does the cutoff. And it could even happen that all the recent rage on reviewing the properties of scattering amplitudes and looking for general properties of Feynman diagrams could be remotely inspired in Cvitanovic flirt with Madness. And Cvitanovic himself has an intense history of the relationships between magical triangles and magical squares of group theory, a idea, of set of ideas that keep surfacing from time to time, usually with some hat tip to octionions
.
And well, there is Knot Theory, but whole books have been written about its flirting with physics. The moral is that a 2.0 referral system should be able to cope with all of this, perhaps stressing the continuity of the research history, perhaps classifying the dead alleys so that any peer can follow the whole history.
Hi Alejandro,
Please see page 34 of my paper http://vixra.org/pdf/1111.0111v1.pdf (also pages 48-49) for an effort to quantize mass using your hep-th/0503104.
I’m going to have to re-do it, removing the non-mathematical material about previous censorship problems and shortening it for journal publication. However, the mechanism is factual, not speculative. It’s ironic that 10/11-d M-theory is now viewed as standard physics, and anything in fewer dimensions is viewed as non-standard!
Hi Nige! yep, probably you should even try to divide it in small, independent letters.
Hans found these relationships semi-empirically (he was developing some quantum relativistic approach to composite W and Z) and I proposed the serial expansion. As I said, Cvitanovic was almost driven to pure, Bedlam-like, madness after Kino****a convinced him to calculate g-2, so I have not risked to deep, alone, on these relationships.
I like your powers of alpha / powers of pi classification, but perhaps this is not the thread to discuss merits or objections. I wonder if we should find some place to reopen the physics forums long thread on low energy mass relationships. Perhaps GUTs only predict m(top)=1 and m(everything else)=0.
Like all things in this new age – people are required to do more thinking for themselves. If you need an editor and reviewers to scout your reading material, you may be in the wrong field (science). Everybody should do their own critical thinking without the unnecessary censorship – read the abstract, skim for red signs and move on to the next paper if you don’t like the one you are reading. There is no need for intermediates (editors, reviewers) – they are strictly there for arguments sake (no pun intended).
We have not commented yet on the idea (in the n-lab) of applying peer review to frozen versions of wiki pages, The wiki can keep evolving, but you can refer to the peer reviewed version.
If somebody seriously would like an internet peer review system one could have an open call for peer reviewers: people give their bio, list of publications, and the blog editors pick the appropriate ones randomly from the correct sample, stopping when three have accepted. Then those reviews could be appended to the blog publication, open for comments and revisions by the rest of the interested community. So comments to the paper should be open after the reviews are in.
There will be people who would attack in the comments of the reviews what they consider crackpot papers, but so what?
Yep, the open call is a better idea, but the scale is very big. Usual journals randomly consider that anyone who has published in the journal is a potential referee. Plus, they ask the authors -and the referees- to suggest aditional ones. But I already made the mistake of thinking that a starting web site had the same right to ask.
Perhaps an intermediate between an unsolicited refereeing request and an open call could be to select the potential referees from people who is commited to publish in open access journals, or in web-only ones (JHEP, etc).
By the book we actually could use more open journals that are willing within reason to look at alternative ideas. Look I agree there are crackpot ideas and junk science out there. Some of it floats around in the so-called establishment journals, especially some of those who charge for publication or reading the articles.
Yes, peer review is important. But as some in other posts have pointed out there where crackpots who turned out to be true. In some ways some of the ideas of Relativity where looked down upon when first proposed and now are the mainline standard.
A few decent open access Journals have existed. But, they usually only reach out to a narrow audiance group.
Who defines what is “mainstream” and what is not?
14 years ago, when I was affiliated to the re****ble scientific institution, my last research work was called something like “Geometric structure of N=(4,4) SU(2)xSU(2) sigma models in harmonic superspace”.
Definitely it was a “mainstream” work, although even after 14 years I do not know if there is any relationship between what was written there and the real world.
On the other hand, I didn’t see any paper on the number of issues that I find really physical (taking account that physics is a science that describes nature). For instance, can the following question be regarded as a “mainstream physics” question:
“Why do the particles have the precise masses they do?”
If yes, why there is so little activity?
I don’t mind if people like to play with models that are not related to the real world (or such relationship can hardly be established), but my impression is that its 95% of all activities in the theoretical physics industry.
Does peer-review process help to focus physicists’ efforts on the real physical problems?
I think the main goal of PR process is to help people distinguish between what is correct and what is wrong. But it is obvious to me that most papers that are “not even wrong” easily pass through the barrier. And I don’t think that its because those papers are useful for progress in science (although I admit that they might be useful). It’s mostly because authors mainly continue to work on what “opinion leaders” worked on in the past (with little modifications).
In my opinion, one of the advantages of VIXRA is that it contains more articles related to the real physics.
Well, Murod, the particular answer to the masses issue is easy: the research was done very intensively in the late seventies, assuming that the GUT masses are zero or irrelevant and then a low energy texture was possible. You can see Wilczek, Zee, Weinberg, Fritschs,… a lot of bigshots involved The results were unconclusive or plainly unsucessfully, and then new students were grown in the doctrine that the GUT masses were the relevant ones.
But you also point to a curious thing: the difference between “mainstream”, meaning the community, and “mainstream”, meaning the public. Out of health science, most of the themes that attract public attention do not have the expected proportional quantity of researchers. Perhaps the most popularly quoted bio research is Stanley Miller “origin of life”. Look how many researchers are actively involved on the topic. Parhaps a “2.0 peer review system” could also incorporate the opportunity of “peerless input” to correct this lack or proportionality.
Sorry, Alejandro, I was probably too angry yesterday after spending 6 hours at police dept just for registering my father’s new car.
So, only GUT masses are relevant, and observable masses are not. Thanks for explaining the mainstream doctrine. That’s a good example of how effective is the peer review.
As Andre Weil knew, jails increase productivity of theoretists, so six hour in a police dept can help too
Anyway, it should be mentioned that while the GUT doctrine is used to excuse working in any low energy prediction, it is also used to avoid working in high energy predictions… I guess that the deep argument here is that we need in any case know how the running of constants work, and then there is not point on doing additional research until more info from the TeV scale. It seems to me a poor ad-hoc excuse, again similar to post-Miller arguments of avoiding further research on abiotical synthesis without more data on the early composition of Earth.
Generally, mainstream is what the majority of the community of science tends to establish as such. For example, aether theory of the Newtonian type when it comes to the forces in nature is not considered mainline. But String theory, even though I like it and consider it a viable path, and has never been proved beyond the math,and remains with little evidence it is real, is considered mainline. Most of the ideas, not all, I find on the archieve here have at least some other scientists who support some aspects of them.
In general things can be popular and be mainline, things can be exotic and still be mainline as long as some of the COS supports simular ideas.