We’ve previously addressed many aspects of “how to write your research paper.” (In fact, there is a full YouTube playlist on this; see the link at the end of this post.)
Now, we’re shifting focus to how you wrap things up at the end. This is a “little things that count” move.
We want to consider three things:
- Peer reviews, preferably over multiple drafts,
- Acknowledging useful input or material from others – whether from a peer review partner or some other source, and
- End-of-paper materials, such as Code Availability, etc.
At the end of this post, we address the most extreme (and potentially useful) form of peer review: partnering., or working in pairs. Instead of you doing the work solo, and then getting someone else to look at it, you work side-by-side with someone. In recent times, this has led to the megalithic growth of Google, as two different pair-partners have significantly influenced Google’s emergence and growth.
Peer Reviews and Acknowledging the Work of Others
Peer reviews are essential to improving your work.
A Classic Peer Review Example: Kullback and Leibler’s “On Information and Sufficiency” Paper
Styles have changed over the decades. There is a different tone in papers written several decades ago, compared to more recent ones. In papers before 2000, peer reviews – whether formal or informal – were hugely influential as authors sculpted their works.
Here’s one example of peer review, from Kullback and Leibler’s 1951 paper, “On Information and Sufficiency.” We can see that there really is scientific interchange during this review process. While there is typically a general thank you (to specific reviewers or to “unknown reviewers”), it was common practice several decades ago to identify specific comments / contributions from reviewers.
The kinds of acknowledging specific comments and suggestions from reviewers that we see in this article reflects a time when the internet did not yet exist – much less open-access and online journal article reviews, coupled with the author(s)’ responses. This was an era in which all journals were printed – with no digital versions available at all – and authors had to be very careful of how much “real estate” they wanted to occupy in a journal. The scientific style was very terse.
A More Recent Example: Structured and Sparse
Here’s a second example, from a paper by Thomas Parr and Karl Friston, published in 2019 in Biological Cybernetics.
Acknowledgments: How to Appropriately Thank Others
Final Words: How to Use a Reference, Comments, or Code That a Colleague Has Posted
It’s inevitable. At some point, you’re going to see a reference posted by one of your colleagues (possibly a classmate in a course) that is just perfectly suited for your work. BUT … you were not the one to find this reference.
How do you use it without being cheesy, in a professional sense?
Ahhh! There’s an answer to this little conundrum.
It’s called Acknowledgments.
Acknowledgments come at the end, before your References, and before Appendices.
An Example
Martin J. Wainwright and Michael I. Jordan wrote a major paper in 2008; it has significantly influenced subsequent efforts in machine learning. (That doesn’t mean that those who cite this work, of over 300 pages, have actually read the whole thing.)
At the end, their Acknowledgments shows the influence of MANY thoughtful peers, who have all helped shape that work.
- Wainwright, Martin J. and Michael I. Jordan. “Graphical Models, Exponential Families, and Variational Inference.” Foundations and Trends in Machine Learning 1, nos. 1-2 (2008): 1-305. doi:10.1561/2200000001. (Accessed 04/07/2021). https://people.eecs.berkeley.edu/~wainwrig/Papers/WaiJor08_FTML.pdf
What to Do in Your Paper
YOU CAN DO SOMETHING SIMILAR. It’s elegant, it acknowledges valuable inputs from others, and it is the professional way to do something.
So if you see a reference that you’d like, go ahead and incorporate it, and then acknowledge the person who nominated that reference.
If one of your colleagues has made some valuable and insightful comments, by all means, put that person in your Acknowledgments.
This is professional good manners, and makes you look like a class act!
Acknowledging Code and Data
Using someone’s code is a little more delicate. First, you absolutely want to acknowledge who wrote the code and where you got it from – e.g., as provided by the professor or teaching assistant in a class, or someone’s GitHub, or wherever you got it.
Second, it is poor manners to post someone else’s code in your GitHub, UNLESS YOU’VE MADE SIGNIFICANT MODIFICATIONS. Then, you want to have very clear documentation indicating who wrote what, showing where you picked up on someone else’s code and where you made additions / modifications.
If you’re doing this in a professional context, OF COURSE you would acknowledge the code-writer, or data source gatherer, etc., in your paper. It might be nice to drop those individuals an email, thanking them and letting them know where your variants, etc., can be found.
End-of-Paper Material
At the end of your paper, it is standard to include several short sections. Your journal’s “Instructions to Authors” page will be clear about what is needed for that journal. If you’re doing a class project or are doing a self-initiated project that you will publish online as a Technical Report or White Paper, some of these will also apply – just think through what makes sense.
The standard list is:
- Acknowledgments – previously discussed.
- Code and Data Availability – this is where you’d describe access to your GitHub repository and any other important data sources. Some journals allow for separate “Supplemental Information,” which often includes code and data.
- Funding – if your work was funded by your company or by a government organization, though a contract or grant, you want to acknowledge that funding. This can be included with the Acknowledgments section, and it can be separate if you are also identifying copyright and ownership.
- Conflict of Interest – you generally want to be able to say that you have no conflicts of interest.
- Human and/or animal testing – this involves a whole slew of government regulations. I never work with human or animal subjects, so cannot advise you here – just note that can be a factor for some of you.
- Copyright – very aligned with funding. Sometimes you need a separate and clear statement, especially if you are granting non-exclusive Open Access to the journal that is publishing your work.
- Data Privacy/Confidentiality – again, this is the kind of thing that shows up when working with – not so much human subjects, but – data PRODUCED by humans. There are some cases where this gets very tricky, as in work with medical records. Get corporate and, if necessary, legal guidance here. Be very clear that your data was appropriately anonymized, or whatever steps had to be taken, so that privacy was never breached.
You likely will not need to address all of these issues. What I’ve done, and what I encourage my students to do, is to amass a small stash of downloaded papers that are good for not only technical content, but for style. In other words, they may have a stellar Abstract, or position their Problem Statement just beautifully. Or, they can do an exemplary job of addressing any of the above points.
It’s good to have a stash of documents whose style you can emulate, rather than having to figure out how you’ll address these issues on your own.
Pay attention to papers published in the journal in which you’d like to publish; look for consistencies in style, and once again – consult the journal’s “Instructions to Authors.”
Partnerships – Can Be Better than Solo
Sometimes, the best work comes not just from peer review, but by working collaboratively – over time – with a partner.
Google’s success is at least part due to the creative partnership between its two co-founders, Larry Page and Sergey Brin.
Here’s a good article on the history of Google. https://www.theverge.com/2019/12/4/20994361/google-alphabet-larry-page-sergey-brin-sundar-pichai-co-founders-ceo-timeline
Within Google, though, another great pairing emerged: that between Jeff Dean and Sanjay Ghemawat.
Great article:
The Friendship That Made Google Huge | The New Yorker
To your success! – AJM
References
Kullback, S. and R.A. Leibler. 1951. “On Information and Sufficiency.” The Annals of Mathematical Statistics 22, No. 1 (Mar., 1951): 79-86. doi: 10.1214/aoms/1177729694. (Accessed April 25, 2022.) URL: https://www.jstor.org/stable/2236703.
Parr, Thomas and Karl J. Friston. 2019. “Generalized Free Energy and Active Inference.” Biological Cybernetics 113: 495-513. doi:10.1007/s00422-019-00805-w (Accessed April 26, 2022.) https://discovery.ucl.ac.uk/id/eprint/10082773/1/Parr-Friston2019_Article_GeneralisedFreeEnergyAndActive.pdf
Wainwright, Martin J. and Michael I. Jordan. 2008. “Graphical Models, Exponential Families, and Variational Inference.” Foundations and Trends in Machine Learning 1, nos. 1-2 (2008): 1-305. doi:10.1561/2200000001. (Accessed 04/07/2021.) https://people.eecs.berkeley.edu/~wainwrig/Papers/WaiJor08_FTML.pdf
Previous Related Blogposts
Maren, Alianna J. 2022. “Strategizing Your Research Project: Developing Your Portfolio Elements.” Themesis Blogpost Series (April 5, 2022). (Accessed April 25, 2022.) https://themesis.com/2022/04/05/strategizing-your-research-project-developing-your-portfolio-elements/
Related YouTubes
There is an entire YouTube playlist on “Writing Your Research Paper.”
We’ve put a lot of attention on how to frame up your work. In fact, there is an entire YouTube playlist on “Writing Your Research Paper.” (Playlist link)
Famous Salonnières, High Priestesses, and Professors
Nadia Boulanger – the mentor to some of the 20th century’s greatest composers. See this YouTube: