Last update: 2014 07 15
statement to include in referee reports:
"I request that the authors add a statement to the paper confirming
whether, for all experiments, they have reported all measures,
conditions, data exclusions, and how they determined their sample
sizes. The authors should, of course, add any additional text to ensure
the statement is accurate. This is the standard reviewer disclosure
request endorsed by the Center for Open Science [see http://osf.io/hadz3]. I
include it in every review."
3) Treadmill or
standing desk? Some info
26) Simonsohn, Simmons, Nelson "P-Curve and Effect Size: Correcting for Publication Bias Using Only Significant Results (SSRN
How to estimate effect size, if you must.
Selective Reporting Invalidates Bayesian Results Also" (SSRN)
p-hacking misleads Bayesians as much as it misleads other mortals.
24) Simonsohn, Simmons, Nelson "Anchoring
is Not a False-Positive: Maniadis, Tufano and List (2014)
'Failure-to-Replicate' is Actually Entirely Consistent with the
The Anchoring replication is not significantly
different from 0,
but also not significantly different from large.
Simonsohn "Small Telescopes: Detectability and the Evaluation of
Replication Results" (SSRN)
Don't just ask "Is the
replication significant," ask also "is it the replication significantly
smaller than small"
"Evaluating Replication Results")
Silberzahn, Simonsohn, Uhlmann (2014) "Matched-Names Analysis Reveals
No Evidence of Name-Meaning Effects: A Collaborative Commentary on
Silberzahn and Uhlmann (2013), Psychological Science V25(7), p.1504-1505 (.pdf) (data)
Collaborating with original authors we show Herr Kayser is not disproportionately likely to work as a manager after all.
21) Simonsohn, Nelson, Simmons, (2014) "P-curve: A Key to
the File Drawer," Journal
of Experimental Psychology: General, V143(2), p.534-547 (SSRN; p-curve.com)
How to analyze the
distribution of significant p-values for set of findings to undo impact
of selective reporting, of both studies and analyses, on hypothesis
In other words: p-curve
helps tell true findings from the rest.
20) Simonsohn (2013)
"It Really Just Does Not Follow, Comments on Francis (2013)", invited
commentary for the Journal
of Mathematical Psychology, V57(5) p.174-176 (.pdf)
Francis misuses and
misinterprets the publication-bias test. Lesson: tool developers should
anticipate misuse and take safeguards to prevent it.
19) Simonsohn (2013) "Just
Post it: The Lesson from Two Cases of Fabricated Data Detected by
Statistics Alone," Psychological Science, V24(10), p.1875-1888, (Data
& code) (SSRN)
The analysis of Sanna's
and Smeesters' raw data show they are fake. On top of many other
advantages, posting raw data will reduce academic fraud.
18) Simonsohn, Gino
(2013) "Daily Horizons: Evidence of Narrow Bracketing in Judgment from
10 years of MBA-admission Interviews", Psychological
Science, V24(2), 219-224 (.pdf)
(earlier draft: SSRN)
Interviewers avoid giving
too many high/low scores on the same day.
17) Nelson, Simmons, Simonsohn (2012) "Let's
Publish Fewer Papers," Psychological
Inquiry, V23(3), 291-293 (.pdf)
16) Simonsohn (2012) "It Does Not Follow: Evaluating the One-Off
Publication Bias Critiques by Francis (2012a,b,c,d,e,f), Perspectives on Psychological
Science, V7(6), 597-599
The critiques are cherry
picked, and ignoring evidence is not a justified conclusion from the
presence of publication bias.
15) Simmons, Nelson, Simonsohn (2011) "False-Positive Psychology: Undisclosed Flexibility
in Data Collection and Analysis Allow Presenting Anything as
Science, V22(11), 1359-1366 (SSRN)
experimental demonstration, and simulations showing that if a set of
disclosure requirements we propose are not followed, results in
experiments are uninterpretable.
14) Saiz & Simonsohn (2013) "Proxying
for Unobservable Variables with Internet Document Frequency", Journal of the European Economic
Association, V11(1), 137-165 [DATA]
of Internet documents about X proxies for frequency of X; using insight
we replicate published studies predicting corruption.d
Simonsohn (2011) "Spurious Also? Name Similarity Effects (Implicit
Egotism) in Employer Decisions," Psychological
work for companies with which they share an initial. Probably a
12) Simonsohn (2011) "Spurious? Name Similarity Effects (Implicit Egotism) in
Marriage, Job, and Moving Decisions", Journal of Personality and Social
Psychology, V101(1) 1-24 (SSRN)
Three JPSP papers find
that people disproportionately choose spouses, places to live and
occupations with names similar to their own. Analyzing the same and
additional data I find that the existing evidence is spurious.
note: Pelham and Carvallo
wrote a rebuttal to this paper. Here is my 5-page rejoinder titled "In
Defense of Diligence".
11) Pope, Simonsohn (2011) "Round Numbers as Goals: Evidence from
Baseball, SAT Takers, and the Lab", Psychological
Science, January, V22(1), 71-79 (.pdf)
When performance is
measured numerically, round numbers become implicit goals that strongly
influence behavior around them.
10) Simonsohn (2011) "Lessons from an Oops at Consumer Reports:
Consumer Follow Experts; Ignore Invalid Information", Journal of Marketing Research,
February V48(1) 1-12 (.pdf)
Consumer Reports released
& then retracted info on carseat safety. Surprisingly, people
successfully ignored the retracted information.
9) Simonsohn, (2010) "eBay's Crowded Evenings: Competition Neglect
in Market Entry Decisions", Management
Science, V56(7), 1060-1073 (.pdf)
Too many sellers end
their auctions at peak time, so they lose money.
8) Simonsohn, (2010) "Weather to Go to College", Economic Journal (.pdf)
More prospective college
students enroll after visiting campus on cloudy day.
7) Simonsohn, (2009) "Direct-Risk-Aversion: Evidence from Risky
Prospects Valued Below Their Worst Outcome" Psychological Science,
V20(6) 686-692 (.pdf)
People value lotteries
less than their worst outcome due to uncertainty; not confusion or
6) Small & Simonsohn (2008) "Friends of
Victims: Personal Experience and Prosocial Behavior." Journal of Consumer
Research, V35 532-542 (.pdf)
Donors give more to
charities helping the misfortune of someone they know.
5) Simonsohn, & Ariely (2008) "When Rational Sellers Face
Non-Rational Consumers: Evidence from Herding on eBay," Management Science V54(9)
eBay bidders choose
auctions with more bids, so sellers start them cheap.
4) Simonsohn, Karlsson, Loewenstein, and Ariely (2008)
"The Tree of Experience in the Forest of Information: Overweighing
Experienced Relative to Observed Information" Games and Economic Behavior,
V62, 263-286 (.pdf)
People respond more to
information that affected them directly.
3) Simonsohn, (2007) "Clouds Make Nerds Look Good: Field Evidence of
the Influence of Incidental Factors on Decision Making", Journal of Behavioral Decision
Making, V20(2) 143-152 (.pdf)
academic attributes are weighted more if evaluated on cloudy days.
2) Simonsohn & Loewenstein (2006) "Mistake #37: The
Impact of Previously Faced Prices on Housing Demand," Economic Journal,
V116(1) 175-199 (.pdf)
Movers from more
expensive cities rent more expensive apartments, at first.
1) Simonsohn (2006) "New-Yorkers Commute More Everywhere: Contrast
Effects in the Field," Review
of Economics and Statistics, V88(1) 1-9 (.pdf)
Movers from cities with
longer commutes live further from work, at first.
Power point slides
Numbers as Goals: Evidence from the SAT, Baseball and the Lab
Lessons from an Oops
at Consumer Reports: Consumer Follow Experts; Ignore Invalid Information