Research process

The research process enables efficient research.  It is in some way forcing you into doing systematic research. It normalizes the chance for experienced and less experienced students to do equally well. There are many good processes out there depending on the nature of the topics.  This is one of the typical research processes.

PHASE 0: BEFORE ANYTHING ELSE - SET YOUR GOAL

Before using this process set the target which conference you will submit and note the publication deadline.  Now schedule your time from now to your publication deadline, by estimating how much time it takes for each step below.  For a new Ph.D., before you go to the process below, first browse the literature.  Get acquainted with the research in the past 3 years, in the conference you want to submit. Read abstracts, look at figures, how they experiment, etc, so you can get a sense of what is the state-of-the-art and what are the open problems. Then decide one good problem to work on.  You will likely stick to this problem for your entire period of Master/Ph.D. life.  Before you start, this short blog may help you:

As an overview, the research process can be separated into three phases: (1) identify the problem and solution (40%), (2) develop solutions and conduct experiments (30%), and (3) write, rewrite, refine (30%).

PHASE 1: PROBLEM AND SOLUTION DEFINITION

  1. Define a problem/research question.  Define a clear, concrete, non-ambiguous problem/research question.  Asking a good question is one thousand times more important than finding a solution.  Trust me, it is really one thousand times or even more, or else you simply waste your time.  It is just like sailing a ship in the wrong direction.  Even the ship is running very fast and efficiently, it does not really matter.  If you feel difficult to define a problem, go to Step 0 and read more papers.
     
  2. Now check the literature carefully.  You should read only from the top tier conferences / venues in your area, to avoid sketchy papers.  Check very carefully, what has been done already in your area relating to this problem.   Find out what are the existing solutions.  What are their pros and cons?  Are there any remaining challenges you can address?  How does your work position in the literature?   This must be done with care, do not be lazy, please, as we need to know clearly how much you will be advancing the field, and also you do not want to find out later that someone has already done your research.  More how to do related work efficiently.
     
  3. Refine your problem.  Now refine your problem statement again to be more precise and narrow.  Specifically, it should clearly show the gap of past work.  Repeat 2 and 3 until you got a concrete, clear, significant, narrow problem.
     
  4. Propose a solution to the problem.  Come up with 3 to 4 solutions, try to do some coding and get some intuition whether they will work.  You will eventually naturally form a set of hypotheses in your head.
     
  5. Double-check the criteria for your solution.
    • Feasible:  Do you have at least a rough idea of how to implement it?  How long will it take?  What is the risk that it will not work as you think?   Will you be able to finish this within your given time?
    • Novel:  Are you sure your solution is novel? (check again possible contributions) . All reviewers are SUPER knowledgeable.  Please do not use luck. Is there really no one who does this or similar ways before? 
    • Significant:  double-check, are you sure your problem statement and the solution is something the research community cares about?  Why is this problem important to solve now? What is so hard about this?  Maybe it's new but there is no significance!  or Maybe you are solving something that is too easy or too obvious!  What is the "social" or "technical" significance?
    • Scientifically verifiable: Can you really show that it is a good and valid solution?  Can you prove in a solid way that the problem is solved? (there are something science cannot answer).

      If all/some of your answer is unsure, please rethink carefully, try an alternative solution, or rescope your problem, such that your work is manageable, novel, and at the same time, significant.  You should feel quite clear and confident.

PHASE 2:  EXPERIMENTAL DESIGN

  1. Think about your experimental design.  Now think about how you can **scientifically** prove what you claim or intend to achieve, in which metric, in which context.  This step is necessary because it makes sure you create some solution that can be scientifically proven to solve a problem.   Draw a mockup table showing how your final comparisons (i.e., how many factors you compare) will look like.   Your professor will love it.
     
  2. Write everything down on the paper using the format of the venues you wanted to submit (highly recommend the latex template).  Now, I know you already feel itchy to start execution.  But wait, first, please write the Abstract, Introduction, Related Work, Methodology, Expected Result, and Expected Discussion section for your paper.  This is perhaps the key step that makes a paramount difference that many overlook. This is necessary to clear your mind and force your mind to think logically and deeply.    Although it takes time, you will often (about 99.99%) to find that you found flaws in your work that you have not thought about before, without writing.   For this step, starts with an outline, answering how the work is novel, significant, and how to logically show that your solution is good and validated.  As for beginners, please choose one model paper from good conferences where you can learn directly from.  Detailed guide here
     
  3. Implement your solution - you can do this more efficiently by learning from GitHub sources.  Don't code from scratch - it takes a lot of time. 
     
  4. Run a pilot study - It is often risky and costly to right away carry out the whole experiment.  Running a cheap pilot study is often useful to see what happens and see whether anything about experimental design or solution needs to be changed.  You will be often surprised that there are many unexpected things happening.  Repeat this step until everything seems to be expected, where you found a trend that you will find a practical and statistically significant effect.
     
  5. Run the experiment and analyze data. - experiment often comes in form of empirical comparison, while analysis often comes in form of analyzing why's of the experiment results.

PHASE 3: PAPER WRITING

  1. Write and polish. Finish the rest of your paper and polish.  Remember that real writing starts in revision.  A good paper usually takes at least 1 month of rewriting (at least 5 - 10 revisions, except for some geniuses, screw them!). Make good figures, good video, write concisely and clearly, it takes a lot of time and effort. Double-check this criterion.
    • Novelty:  What is new here?  What can be learned from here?  And why has it not been done before?
    • Significance:  What is the social or technical significance? Why is it hard?
    • Validity:  Did you really prove what you have claimed?  
    • Clarity:  Last, did you clearly convey 1, 2, and 3?
       
  2. Prepare for rebuttal (conference) or revision (journal).  After several months, the verdicts are out.  Prepare yourself to do a good rebuttal and/or revision.  The key point for rebuttal is "Be polite. Prioritize the most important issues.  SHOW how you change, not simply what you will change."