I have been hearing from many experienced researchers that research is not like coursework and is by its very nature open-ended. There is no fixed path, and even your supervisor may not have concrete answers. In fact most likely, nobody in the entire world may have the answers to your questions. If you’re doing “cutting-edge” research, then this will happen at some point or the other - but it will happen. I have listened to all these insights and have wondered how exciting research must be - come on, we are talking of discovering new things here!
But my oh my, actually doing these things is a whole different thing. For the first time in my experience with research, I have found actual gaps in the literature. And it turns out that while research is hard - sometimes very hard - researchers have the liberty to sometimes take the easy way out. And I think this incremental nature of research is one of the hardest concepts to fully absorb for people who are new to research. Of course we build upon the work of others and leave our work for others to build upon - but what is not clear as a beginner is how much to build upon, how much to actually build, and how much to leave for others.
Research work is usually of a well-defined size. To me, this just seems like an “industry-standard” - possibly regulated by conference deadlines. So called “full papers” (generally in the research track of a conference) are long and detailed, but of course they are not all of knowledge on the topic. Deciding how much to include in a paper seems to me like an art. Trying to go for the low-hanging fruits may be one strategy, but then your work will not be deep enough and your paper will likely be rejected (or you might submit to a smaller track, which is not necessarily a bad thing - but its also not the same thing and as a graduate student or a professor, it turns out that you cannot live off short papers). But at the same time, you can easily hop on to flashy new ideas and there is not much to lose if you fail because you only put so much effort into it. On the other hand, supposedly real research means “deep” results - but how deep is deep enough? To me this sometimes looks like a rabbit hole. I can keep digging and digging and at every step it will seem as if just a little more to complete the picture. When do I exit and call the rest “future work”? I don’t want to stop too early and be called shallow, particularly if I know what the next step is. But I also don’t want to keep digging and digging and never get to the end of the tunnel. Of course there will be an end - but I don’t think it will be aligned to “deadlines”. There might also be challenges that may involve me developing deeper engineering skills or other auxiliary knowledge - but how much before I become a jack of all trades, master of none?
I don’t think there are clear answers to these questions (in the spirit of this post, I should say that these questions are just as open-ended) and I suspect that these tread into the “meta-level” of research which only the experienced researchers have a taste of. In fact I have come to realise that a research supervisor is not just a great source of technical knowledge, but also of this meta-level wisdom.
Note that my research is from software engineering and so my perspective is limited to this field.