By Eduard Hovy
USC Information Sciences Institute
Is there a claim?
Is the claim clear?
Is the idea large or small?
Is the idea as large as it can be? Can you generalize it, or apply it elsewhere?
Motivation, Use, or Application
Why should we care?
How can the claim be used? Are there other applications?
Details of the Idea
What are the basic items/elements /representation units of the idea?
What are the rules or types of interrelationships between them?
How elaborated are these items and rules/relationships?
How much of the phenomena do they cover?
Is there enough data in the study?
Is it representative? trustworthy? applicable?
Is the method of investigation clear?
Is it appropriate? Does it ignore phenomena that look relevant?
Is it well-reasoned? no biases or mistakes?
Is prior work recognized? used?
Is there an evaluation?
If so, is it adequate? Complete enough?
Does it speak to the claim?
Does it actually prove the claim?
If not, why not?
Is there a discussion of how one might try to test or prove the claim?
Can one make predictions and (easily) test them?