How to write paper

Writing, especially top-tier conference, is crucial for acceptance.  You should show excellent logical reasoning skill, show the experts that you know the area well and what is the current frontier of the area, that you possess enough scientific training to execute the research, and that you have already thought deeply and carefully about your topic.

You can find the PDF version here:  How to write a paper (PDF)

Reviewers do not see when you are working, thus, they can only judge all of these qualities through reading your paper, yes only your few pages of writing, so writing is crucially important.  Even you work so hard on the topic for 1 or 2 whole years or that you have done everything right, but if you cannot explain clearly, reviewers do not care whether you work hard or not.  After all, they can only judge your entire work of one or two years based on your 10 pages writing.  Here I summarize some key points you need to talk about in your typical paper.  It follows an artifact-contribution-type paper but can be adapted for any papers.  I also summarize the typical reasons being rejected based on section.

Abstract

The primary function of this section is to describe your work, what did you do and your key finding. For beginners, I recommend this five-part structure with fewer than 200 words.  Using this template can help train your logic in arranging arguments. The first part is about speaking broadly about the importance of your area.  The second part is for describing the problem.  The third part describes what you did.  The fourth part describes your key result.  The last part describes the significance, why I should care.

Example:

Given the prevalence of smartphones, mindfulness-based mobile applications (MBMAs) have received much attention (importance). Current MBMAs mainly use the guided meditation method which may not be always effective, e.g., cannot follow the pace of instructions and needs a private environment (problem). This paper presents a framework for interactive MBMAs which allows users to self-regulate their attention according to their abilities and conditions (what you did). The framework is described by an Attention-Regulation Loop and has two components: (1) Relaxation Response and (2) Attention Restoration Theory (what you did II). This framework is validated by being applied through an MBMA called Pause (what you did III). Pause is shown to improve user attention and allows users to quickly feel relaxed even when used in busy environments (results). This framework informs future development for interactive meditation and has broad implications for designing mindfulness and well-being (significance).

Introduction

This is the most important section of all.  It's the face. The primary function of this section is to argue your motivation and why did you do this work.  This section is the first impression, if it sucks, reviewers will likely just skip the rest of the paper and gives you a rejection.  This section usually contains 5 points and 1 key figure  (same structure as abstract but in expanded version).  In writing, the Introduction is just an expansion of the abstract. In the first paragraph, you should explain the background and current state of your area, as well as the core-related work.   In the second paragraph, you should explain the specific problem and gap of past work.  This is perhaps the most important paragraph and you should be very precise what is the problem you want to solve.   In the third paragraph, you should explain what you did.  In the fourth paragraph, you should explain what did you achieve. The last paragraph, it is for explaining the significance and contribution of this work, why we should care, and why it is important for the research community.  For new (especially non-native) students, DON’T DO OTHER FORMAT ASIDE FROM THIS. PLEASE FOLLOW THIS.  It trains you to think logically.  You can try other formats when you get experienced.

In the Introduction, you should complement with the key figure of your paper.  This figure is the most important figure that describes your whole work, so please give effort to it, should be self-explanatory (roughly 80-90% of all accepted paper has excellent Figure 1). 

Reasons for rejection

  • Not logical, i.e., not supported with evidence.  Each paragraph in the Introduction should be supported with strong evidence.  For example, menus are useful…(should be followed by evidence of past work).  The second paragraph…the current linear menu is slow (followed by evidence and sound argument)… the third paragraph…pie menu can be faster (followed by good reasons why you think so)… the last paragraph, the pie menu is important because (followed by strong argument).  The worst argument is “Pie menu is important” or “Linear menu is slow” without any clear supporting evidence.
  • The specified problem is unclear.  Usually, if the problem specified by the authors is unclear, it is usually 100% the paper will be rejected.  The worst problem is about “there is no study who did this, so I want to do it”.  No one did it does not mean it is a good problem, perhaps it is not important thus no one wanna do it.  You must specify really concrete problem before saying no one did it.
  • The problem has already been solved or your solution has already been partially suggested by someone.
  • Too common sense or obvious - no impact to the research community
  • The problem is not scoped clearly thus you are over-promising, e.g., "I will create a software that can recognize human behavior" This is a bad problem because it contains no scope, you are overclaiming, you should BE SPECIFIC what kind of behavior, what people you test with, and to what extent of success you plan to achieve.  Usually, any over claiming is a sure rejection because you make reviewers feel hyped and then disappoint them.
     

Related Work

The primary function of this section is 1) to reveal the gap of past work, 2) identify potential hypotheses, and 3) identify experimental design. One good tip is to read some of the papers close to your work and see what they cite commonly; this commonly cited work is usually important to work.  Another good tip is to categorize your related work into different categories, so reviewers can understand it easily.  It is important to note two points:  first, you should show the difference and why this difference matters; second, good related work is a synthesis of related work, not a bullet list of “An et al. do this, B et al. do this”; instead it should be like a summary of insights and conclusions:  “Researchers have worked on X (A et al, B et al.)  However, C et al. critiqued it as economically inefficient.  D et al. share the same view.   This conflictory view demands further analysis”.  It should give an easy-to-read insight.  You synthesize, not summarize or list.

Reasons for rejection

  • You miss citing a very important work in the field
  • You did not logically reveal the gap or the problem
  • You just list the related work, without giving insights, leaving a lot of "so what" question marks
  • You touch some areas which are not related to your work, thus adding confusion, e.g., for example, you work on menus, but if you cite some work on cognitive science, although it seems related, they can confuse your reviewers.  Choose a model paper close to your topic and learn how they write their related work.


Devices / Interaction technique / Systems (skip this if your paper is primarily experimental)

The primary function of this section is to describe what you have developed, a software, hardware, an algorithm, etc.  It MUST allow people to replicate your technique or device.  Usually, you should explain 1) the design goal (where it will be used, and how you want it to be used), 2) the design rationale (e.g., why you decide to choose A algorithm rather than B algorithm), 3) any tradeoffs you made (e.g., accuracy vs. speed), 4) any technical challenges you face and how did you solve it, how you can be sure your method does not affect the validity of the results., 5) the engineering process (e.g., how did you fabricate the materials, did you use 3D printer, or off the shelf)

Common reasons for rejection

  • You focus too much on implementations but not on the ideas behind the innovation.  Implementation is important but they are of second priority.  The idea is important.
  • You describe inadequately, not allowing me to learn or replicate your work, thus I cannot judge your work
  • Your solution has little technical novelty nor any special challenges or creativity in it
  • Your solution is not grounded in any principles or rationales.  You did not adequately explain why you think your solution will work, and on what basis you are making that claim.


Method / User Study / Evaluation (many titles you can use)

This study section describes the evaluation process.  This section contains typically seven points (This section has a fixed model, so please do not deviate from this style):

  • Experiment objective - it describes the general purpose of your study
  • Hypothesis or research questions - describes the questions or hypothesis you wanna test (good science always has very clear, succinct hypotheses!)
  • Experimental Design - describes the design - within or between subject, what is independent and dependent variables, how did you control other confounding and random variables, how did you ensure construct validity, internal validity, and external validity.
  • Participants - how many participants, how many females and males, their mean age and SD, report if you exclude some people and why you do that, report any demographic information that may affect your study (e.g., in menu study, you would want to report how often did the participants use PC in their daily life), did you pay any compensation for them (this is important to know whether participants have any motivational issue)
  • Apparatus - what instrument you use, e.g., PC, questionnaires, game systems, etc.
  • Procedure - tell the exact procedure during your experiment - what task is done first and later, what constitutes one trial, what constitutes a block, how long the entire experiment, how did you counterbalanced, reviewers should be able to imagine clearly what did you exactly do during the experiment, and able to rerun your experiment.
  • Metrics - describes what metrics you collect, and why you choose this metric
  • ***DON’T DEVIATE FROM THIS FORMAT****.  Researchers including me are dumb and super busy and thus can only understand this format.
     

Common reasons for rejection

  • Poor experimental design - did not carefully control all variables thus the construct, internal, external validity of the results are poor.
  • Choice of methodology - did not carefully explain why you choose certain methodology over the others, for example, did not justify why you choose to do the experiment in a lab based environment, instead of a field experiment, or why you develop custom app instead of a commercial app.
  • Unclear objective - after I read the whole section and I still do not know clearly what the authors are trying to achieve, this paper is usually bye-bye.
  • Participants - small sample size, or too many males/females, unrepresentative group (choosing only university students).  Learn to decide the number of participants using power analysis.
  • Metrics - Relating to construct validity, if I feel that you choose certain metric that cannot justify your objective, for example, doing only interview to justify that your system is faster is usually a no-no.   In addition, using a non-standard questionnaire (students often come up with their own questionnaire!), this paper is also likely to be gone.  The best approach is usually to combine quantitative (physiological tools or performance) and qualitative (interview, questionnaire) to justify your results.  Quantitative data tells you what happen.  Qualitative data tells you why it happens.

Results

The primary function of this is to report your results.  You should not try to discuss the results but only describe the result.  You should not yet explain why you think the result is like this, but just the result.  After reading this section, it should feel boring but informative (there are some cases where you can include some discussion of Results for better flow).  Another point is that you should describe clearly the statistical test you use.  For subsection titling, it is recommended to categorize the results  based on the metrics you defined in STUDY section.

Common reasons for rejection:

  • Inappropriate statistical tests - if you analyze your data using the wrong statistical tests, usually it lowers my score depending on the robustness of the test you choose
  • Bad figures or bad data reporting usually lowers my impression but will not lead to straight rejection.
     

Discussion

The primary function is to discuss why you think the result is like this, did your results match your hypothesis, why sometimes it does not work, what is an important result that I (reviewer) should care and why, do you find anything very surprising? what are the implications of your result?  What are the implications for expert and novices? What is the pros and cons of your device?  How far is it from commercialisation?  The hidden function of this section is to show how smart and how insightful are the authors to the reviewers, and whether you have thought deeply on the topic.

Common reasons for rejection:

  • Just simply rephrasing the results without any deep insights
  • You propose some guidelines or implications that are too common sense (this is one of the most common mistakes!), i.e., you did not offer any actionable design guidelines or implications
  • You discuss some very big claims without any data support or something that is far away from your data
  • The mismatch between what you promise in the Introduction, and what you discuss.

Some tips for discussion

  • Do not repeat your results here, it is redundant, you can though summarize briefly your whole work and your contribution within one paragraph
  • Possible discussion
    • Did you find what you expect?  Discuss in depth anything you did not expect.   The key point here is you need to have a clear research question or hypotheses at the beginning.  Then based on this question in your mind, you discuss your results.
    • Did you achieve what you claim?  and what you did not yet achieve? what is your ultimate goal, and how far are you from it?
    • How did your work compare with related work? consistent, anything surprising?
    • Why did you choose certain experimental/device design decisions, and how did it affect your results?
    • Design implications or guidelines
    • Significance of your work
    • Limitations: for limitations, please only describe the core limitation that you think might affect the validity of your result. Limitation such as “I conduct this study in the US but I have not tested in the UK” is not a limitation to be discussed as it is almost a common limitation in any study.  You should only discuss the specific limitation in your work and why it does not matter too much.  Do not also discuss limitation that makes you look good (very common mistake).  Discuss CONCRETE, REAL limitation.
    • Future work, you only have to discuss one or two really concrete one.

Conclusion

The primary function is to 1) confirm that you achieve what you claim in the Introduction, 2) repeat significance of the work, 3) give a strong takeaway message, and 4) a happy ending sentence.

  • Key point is not to discuss any new things; all is simply a summary, please do not add any new terms or ideas in this section
  • The second point is not to report any detailed results, e.g., my technique is 80% faster than XXX.  You should only say broadly the contribution “my technique will be useful for XXX"
  • Not much impact for rejection or acceptance.
     

The best way to use this knowledge is to use a checklist which can install a habit in your writing. After you use the checklist several times, and once it became habits, you can write well even without the checklist.  I often found students (almost all students) saying to me “sorry, I FORGOT”…this Forgot can be easily avoided by using a checklist.

Key to success

  • FIND A MODEL PAPER - if this is the first time you write your paper, there is almost no way you can write well.  The easiest way is to find a model paper that is closest to your paper topic and use it as your model.  Learn the style of writing, the contents, and the flow of contents.
  • NICE FIGURES - all good papers have nice-looking, self-explanatory, informative figures.  I cannot stress enough, but figures are super effective way to convey your ideas to readers
  • GET FEEDBACK - the only way you can dramatically improve is to get high-level comments from experts.  Don’t be afraid that experts will not reply to you; they like people attending to them.  The more you do, the higher chance you can get.  This means it depends on how fast and well you can understand these high-level comments, and fixed accordingly.
  • ITERATE - iteration is the key.  Keep in mind that real writing starts when you revise.  The typical number for good paper is around 10 to 15 revisions.  If you have around 5 revisions, your paper is likely to get rejected.
  • WRITE EARLY - the earlier you write, the higher chance you get your work to be accepted.  It is a good practice to write clearly even before you do your experiment.  In this way, when you write, you will often find that “oh, actually this part of my experiment sucks because there is no reason why I should do this and that”.  Take this advice and believe me, you will thank me later.
  • ONLY ONE CONTRIBUTION - remember, a great paper has only ONE contribution.  Having a lot of contributions is usually bad paper, as it has no focus or depth.  Also, putting a lot of contributions in limited space can frustrate readers. A good test is trying to talk your contribution in one sentence such that anyone can understand the value of your work.  I was told by some top researchers that the ability to convey your contribution in one sentence is often a heuristic to test for high-impact factor journals or conferences, such as Nature.

Some key writing tips (compiled from common mistakes):

  • One paragraph only one idea
  • Choose each word carefully, such that it has no two meanings
  • Each paragraph starts with an opening sentence (“This paper proposes…”) and a signal (e.g., "However") describing the tone and whole idea of the paragraph
  • Each paragraph ends with an ending sentence - this sentence describes the conclusion of this paragraph
  • Avoid non-informative sentence, e.g., "Our work is new."
  • Each sentence should ALWAYS BE BACKED by some evidence, i.e., DON’T MAKE any claims without evidence
  • Each sentence should have only one interpretation, for example, “my technique is good” is a bad example because good can means so many things, speed, accuracy, etc.  Remove ambiguity.
  • Ask yourself, what is your contribution?  You SHOULD able to explain in one sentence.  And this contribution should be coherent across abstract, introduction, your experiment, discussion, and conclusion.
  • Make clear what results are surprising, what are expected, do not mix them as it hides away why readers should care about your work.  What surprising is usually that one that greatly advances the field.  Too obvious result IS USUALLY not valuable.  Saying “interaction must be faster” is usually a bye-bye paper.  We do not learn anything from these obvious things.
  • Avoid writing too much general knowledge. Assume readers are experts. Imagine people who read has published many CHI papers

 

Make sure this is clear

  • Are the significance and contribution super clear?
  • Did the work introduce new ideas or approaches?
  • Did you make sure that the results are valid?
  • Did you write very clearly and precisely?
  • Did you read relevant previous work and cite them?

 

Why LateX

Well, the answer is simple, LaTeX is simply better!  LaTeX is neat, clean and supports many useful libraries.  It automatically figures out the positions of figures/tables, the formatting (e.g., mathematical equation), the inclusion of bib, etc.  LaTeX allows me to focus on what to write, and not how to make it looks good. LaTeX is not even hard to learn, it will take you only a few minutes to settle. On the other hand, Microsoft Word, especially for research papers, can be a nightmare.  I strongly recommend everyone to use LaTeX for writing research papers - the benefits just outweigh the cost of learning by far.  For Windows, download MikTeX (the distribution), and then install texStudio, and you are ready to go.  For Mac, use MacTeX for distribution, texStudio is also available for Mac.

 

For more readings:

Books I used to learn how to write: