“It’s the questions we can’t answer that teach us the most. They teach us how to think. If you give a man an answer, all he gains is a little fact. But give him a question, and he’ll look for his own answers. That way, when he finds the answer it will be precious to him. The harder the question, the harder we hunt. The harder we hunt, the more we learn.”
Patrick Rothfus, A Wise Man’s Fear
The most important rule of working in my lab is to ask questions! Don’t understand a paper I’ve sent you? Ask me about the concepts you don’t yet understand. Were my directions unclear? Ask me to explain another way. Don’t remember how to do something? Ask! I know that it’s not always comfortable, but I promise that I will never make you feel bad for asking.
Your thesis is a time where you can develop tools that will serve you in your professional life. Managing a year-long project is certainly one, but depending on your goals, we can also create other types of experiences for you:
My preference is to avoid email as much as possible. I respond best in Slack for most things that would typically be in an email conversation (asking questions, requesting a meeting, etc). If you are not familiar, Slack is like the best parts of email and chat all in one. Whereas I typically respond to emails within a day or so, I often respond to Slack messages within minutes. You should have already received an invitation from me to join our lab’s Slack account. By default, everyone is enrolled in the “general” and “random” channels, and anyone can send private messages to anyone else in the lab. As our lab grows, we can add more fun channels (music recommendations, cute animal pictures, whatever)!
Asana is the project management software I use for my research (and life). Your thesis is a project, and you and I are collaborators on it. If you are doing independent research with me, then your project will be shared with me and your other collaborators. During our weekly meetings, we will examine the goals you have set for your project for the week and also set new goals for the next week. Once you have completed a task, you will check it off, allowing everyone on the project to know one another’s status. If you have not been invited to your project, please let me know ASAP.
Please create a Github account for yourself, and please join our lab organization. Github not only allows you to store and share your code, but to create new versions of old code that has different functionality while keeping everything neat, clean, and organized. Beyond what this tool can do for you, it does a great service to open science: in this lab, we write code that we are have confidence in and that we are happy to release to the world. This keeps us honest and replicable, and also helps all researchers throughout the world push back the “frontier of ignorance” at an even faster rate! Git is a powerful tool. If you are unfamiliar with it, you may find this tutorial helpful!
My reference management of choice is Zotero. In my view, it is better than EndNote in every possible way! First, it is free (as in speech as well as in pizza), which means that you could contribute code to it or change it to fit your needs. It also means that you can continue to use it beyond Bates without having to pay for it! Unlike EndNote, it’s always compatible across different versions. Once installed, you can import a new reference into your library with a single click. There are also plugins for Google Docs, Microsoft Word etc so that you never have to worry about typing in references by hand.
Although you may have been trained to use a tool like SPSS in a statistics course, we will be using R in this lab. There are some reasons for this: first, like Zotero it is free and you will be able to continue using it after your time at Bates, or during your thesis when you are away from campus. Second, R is a powerful engine for plotting data as well as analyzing it. Third, your final thesis document will be in Rmarkdown, giving you one unified location for your analysis, figures, and text. Although there is some overhead to learning R, I think that you will find that it’s much simpler to type in a single line of code to perform a statistical test than to remember which of half a dozen menus to click through in SPSS! I recommend using RStudio as your R environment. It gives you everything you need all in one place. Here are the installation instructions. If you would like to get started in learning R, this is an excellent tutorial.
Within R-Studio, you can use the library ThesisDown to create a gorgeously-formatted thesis. This is not required in my lab, but many students choose to do this because it looks much better than a word document and because you can write code to make your figures and render the figures right in the same document!
If it would be helpful to have a key to the lab, please fill out this form and bring it to myself and Jason Castro to be signed. Please leave the lab locked when you leave. The space used to be a lounge, and I have found random folks using it as such when it is left open.
The user account for all lab computers is BCVL. All lab computers have the same password that Michelle will give you in person.
We will work as a lab at the beginning of each term to determine group meeting time that works for the majority of us. Some lab meetings will be tutorials in which I teach a particular technique (eye tracking, EEG, data analysis, etc.) During other lab meetings, you may be requested to present your current progress on your project. Depending on lab members’ needs, we may take an entire meeting to discuss one member’s project, or we may go “round robin” to get updates on each member’s project. The topics of each lab meeting will be determined by the group in advance, and also announced on Slack. Research can be an isolating enterprise, and we are all smarter together. Lab meeting serves as a vehicle for getting feedback on your project, exposing you to other ideas and problems in the area by hearing about your lab mates’ projects, and allows me to give efficient training on technique.
At the beginning of each semester, please send me a message on Slack with at least three (3) times that would work for an hourly weekly one-on-one meeting. We might not take the whole hour each week, but it’s best to have the time if we need it. I’ll let you know by Slack which time works for me. Meetings will be in my office (Hathorn 106).
Before you can begin collecting data in either eye tracking or EEG, you will need to go through a training protocol. The first step will be to hear an introduction to the technique from me (I will give these periodically during lab meetings, or we can do this in a one-on-one weekly meeting). After you have received this information, we will schedule a time for you to observe me prepare and run a participant in the given technique. After observation, we will do a brief apprenticeship in which you and I collaborate in running the participant. It’s good to volunteer to be this participant as well as observing one. By going through the process of being in an experiment, you know how to advise and motivate your own participants! When you feel ready, the last step is to have me observe you running a participant. When you can do this independently with proper technique, I will allow you to start collecting data for your project.
Proper experimental design is the keystone to experimental success. For our first one-on-one meetings, I will ask you to design your project’s experiment(s), and we will work together to ensure that they have the proper balance and control. This can take time, but trust me, it is much better to spend more time now than to need to re-run the experiment in the future! My aim is to train you to be able to independently design beautiful experiments. In my experience, you’ll do so far more effectively if you can see the flaws in your own design. (Trust me, some of my first ideas still have flaws, this is why we take time here). If I see an issue, I may not raise it with you right away, but may ask you to simulate the experiment in a programming environment so that you can see what would happen if this experiment were run “as is”. Again, patience is critical here and you will be rewarded later with interpretable data! Part of good experimental design is to ensure good statistical power. Remember that power is the ability to find a statistically significant effect given that one should be there. It is my policy to do some kind of power analysis in advance so that we know how many participants we need to run for your project. The reasons for this are three-fold: (1) It is increasingly common for journals to require a discussion of how sample size was determined; (2) It’s scientifically unacceptable to add participants until one gets the result one wants; (3) It is wasteful (and arguably unethical) to just run the maximum number of participants available to us. As the power calculations you learned in statistics make parametric assumptions, we may or may not use them in your particular project. I will work with you to show you some non-parametric ways to get the same information.
If your project involves collecting data from human participants, your project will need to be approved by the Internal Review Board (IRB) before we collect any data. This is a critically important step, and can take some time. The IRB plays a crucial role in independently assessing the research involving human subjects that takes place at Bates College to ensure that all projects meet our community’s ethical standards. As part of this process, you will need to have a valid certificate showing completion of the Protecting Human Research Participants course offered online by the National Institutes of Health (NIH). This training course can take a few hours to complete, and can be done at any point in your project. I suggest that you do it right away when you start working with me because the certificate will remain valid for the remainder of your time at Bates. If you remain in research, you will be re-doing this course every few years for the rest of your career. Most of the remainder of the IRB application will consist of the experimental design that you will have just finished if you have completed a thesis proposal, so it should not take too much time. PLEASE NOTE that I require all students to share their IRB proposals with me BEFORE submitting to the IRB. Again, this can save us precious time if there are any issues. Once your proposal is submitted, it will be a few days before the IRB will read it and give us comments, and we might be required to submit a second application before the protocol is accepted.
Congratulations! The IRB has approved your project! While you are waiting for approval, you can start writing the code that will run your experiments. If you are not an experienced coder, this can be an intimidating process. Remember that learning has some intrinsic discomfort, and that you have many resources at your disposal: Stack Overflow is a great resource for asking and receiving questions about coding syntax (please search for your question ahead of time… I find that 95% of times, my questions have already been answered on the site). You also have me (please feel free to send me a direct message on Slack), and/or other lab members.
You may choose to write your experiment in either Python with the PsychoPy library or in Matlab with the Psychophysics toolbox. These tools allow us to design experiments and present stimuli in a highly controlled manner. For example, we will be able to control the duration of an image on the screen to within a single refresh of the monitor (10 msec). If you are unfamiliar with the Psychophysics toolbox, I recommend working through some of these tutorials. I don’t expect you to work through all of them at once, but for what we do at BCVL, the Accurate Timing Demo and the Change Blindness demo are going to be particularly useful. I will have Matlab templates for some of the major paradigms we use in this lab on our Github account. To start your project, pick the appropriate template and fork the template using the git clone command. These templates are like “Mad Libs” for coding. The major structure of the experiments is in place, but you will need to fill in the details that are particular to your project.
Our goal should be to have tidy data. In the long run, you will save countless hours of hassle by organizing your project well in advance. In the Github repository for your project (and in your Etna folder), you should create the following subdirectories: Experiment, Data, Scripts, Figures, and Manuscript. Any project found not to have this structure will be halted until this structure is created! The experiment folder will likely contain a subdirectory for the stimuli that you use in the experiment, and well as the scripts/functions that are used in the experiment itself. In the data folder, you will save the output of your experiment (be it behavioral responses like reaction time and accuracy, eye movement files, or EEG data). As EEG and eye tracking involve multiple steps of processing, I recommend two (or more) sub-directories, one labeled rawData that will contain a copy of the data before you have touched it, and a processedData directory that contains the finalized data that you are reporting in your thesis. Any analysis scripts that allow you to go from raw data to processed data should be contained in the Scripts directory. Depending on your project, you may have sub-directories within scripts for more specialized analyses. When in doubt, please talk to me. The Figures folder is straightforward. If you are in the preliminary stages of your project, you may want to create a subdirectory called “rawFigures” or “preliminaryFigures”. This is because it takes quite a bit of work to transform a figure from the raw output shown by Matlab or R into something that is publishable, but it’s not always worth the time to make a “camera ready” figure when you are just trying to understand the nature of your data. Making finalized figures is an art unto itself, and we will likely do at least a couple of lab meetings on this topic! In the meantime, I advise naming the figures descriptively rather than something like Figure1.png. The final figure order is not always the same as your order of analysis, and you will be grateful when looking at your file folder to know what each file is. The Manuscript folder you will use last and will contain your draft(s) of your thesis and/or the journal paper that we are writing together for our project.
Counter intuitively, it’s often easier to get a computer to understand your code than another human. In some cases, that human may be you! It is unfortunately common to write some aspect of your code that you are sure you will understand later, only to feel lost a few hours/days/months down the road, losing precious time scratching your head trying to re-create your own past thought process. A similar issue arises when you are trying to share your code with someone else: because there are often several valid ways to code the same thing, it’s not always easy to understand someone else’s train of thought. To ameliorate these issues, we create readable code in this lab by using the following guidelines:
One of the best pieces of advice for writing code is Don’t Repeat Yourself (DRY). If you are doing something a set number of times, use a loop. If you find yourself running the same snippet of code over and over, create a function that can be called in one line. This takes some practice, and I will work with you to help you get the hang of it.
A comment is a cue for a programming language to skip over. By commenting, you can write yourself and your collaborators notes in plain English. In Python and R, the # character creates a comment. In Matlab, you can create a comment by placing a % before a line. There are three situations where comments are particularly useful:
Code is a lot more readable when there is space to let it breathe. This is called white space. Some best practices include:
myVariable=zeros(100,1); Versus:
myVariable = zeros(100, 1);When you create variables, please name them descriptively rather than abstractly (i.e. “presentationTime”, “reactionTime”, “cnnLayer2”, rather than “a”, “b”, “c” etc.) This will help you keep track of your code and will also help others to understand it. To make multi-word variables more readable, I suggest using “camel case”, thatIsCapitalizingTheFirstLetterOfEachNewWord. (Of course, most variables will not be this long!). I find this is far easier to type than placing underscores between each word. If you prefer underscores, go for it!
Just as variables should be descriptively named, so should your scripts and data files. In general, you should avoid “fooVersion1”, “fooVersion2” etc as Future You will curse having to re-discover the differences between the two files. (Note: this is not the case for manuscripts). And as I had to teach myself the hard way, naming something “fooFinal” is just asking the universe for a reason to need yet another version!
Coding is hard work. It forces your brain to think in a way that is not natural. You will get better at it over time, but you will encounter days where you are stuck. While you will grow tremendously by working through some of your problems, if you find yourself stuck for more than an hour, you should seek help. The following resources are available to you:
?meanOr, if you didn’t know that there was a function called mean and wanted to know if there was one, you can search through help functions containing the word “mean” by typing:
??meanIn Matlab, the same two procedures can be done like this:
help mean
lookfor meanFor Python, getting help depends on where you are deploying your Python scripts. I personally like working in the Spyder interactive developer environment (IDE) that has a help panel that is similar in look and feel to RStudio. There are also other methods, and Michelle is happy to help you find one that works best for you.
Debugging is one part rewarding and nine parts frustration. It’s great that the computer tells you exactly what it doesn’t understand, but it’s frustrating to not know how long it might take to squish all of the bugs in your code. This is normal. It’s normal to spend an entire day on something like this. And while it’s easy to think of that day as “wasted”, it really isn’t. You may not have solved your problem, but you learned a great deal. If nothing else (with apologies to T. Edison), you found a few dozen things that don’t work!
Knowing this doesn’t always make the process fun, though. You can always come to me for a pep talk: I can tell you about 15+ years of dumb mistakes that will make you feel better. You can hug one of the stuffed animals in the lab. Sometimes, inspiration will hit by getting out of the lab - go to the gym, have a cup of tea, or take a nice long shower. Sometimes the answer will hit you!
Before we schedule and run naive subjects in any of our experiments, we will run three pilot subjects: (1) you, (2) me, (3) a volunteer colleague in the lab. This will allow us to ensure that the experiment is working as intended, and can allow you to get insights into the experience of the subject. We’ll look at the pilot data together to make sure that we have everything we need to analyze the data, but pilot data will not be included in the main experimental dataset. Please be generous with your time and help your fellow lab members with piloting.
Congrats! You are now ready to schedule participants for your study! This is a two-step process:
You will first need to ensure that the testing room you wish to use is free, and that Michelle is free to help (for EEG and eye tracking). You’ll do this by checking the BCVL experiment calendar. Please see Michelle if you are not a member of this calendar. During some times of the semester, there might be multiple projects vying for a slot. The order of priority is:
Once you have your time slots, you will need to register your experiment with the Psychology department. Once you have done this, students will sign up for slots, and you will receive an email notifation. Important please list the participant on the BCVL calendar once one is signed up. You will be responsible for ensuring that each student volunteer gets their course credit by filling out the appropriate Google form.
Jamie helpfully created a checklist of steps for running EEG participants that can be found here.
General advice:
Some participants will be eager to learn about the purpose of the experiment, while others will only be interested in moving on. Although our experiments do not use deception, if a participant asks about the experiment before or during data collection, tell them that it’s about visual perception and that you’ll be happy to tell them more at the end. At the end of the experiment, feel free to tell them more, and if they have questions that you can’t answer, please feel free to point them to me.
This will depend on the project to a certain extent. Methods that are used frequently (decoding, representational similarity analysis, etc) will have templates on Github.
For first drafts, I’m happy to use either Microsoft Word or Google docs, with a slight preference towards Google docs to keep computer clutter down. There are Zotero plugins for either tool that will allow you to add references with a single click. For final, “camera ready” manuscripts, we’ll work with RMarkdown (see below).
In the third week of September, you will submit a thesis proposal to the Neuroscience Program. This will be read by at least two neuroscience faculty members. The proposal is an abstract for your project as well as a complete methods section, written in the future tense. This is a great opportunity to get mental clarity on all of the steps that will be involved in your thesis. Plan to work with me to craft something we are both happy with before it gets sent to other faculty members. Concretely, this means getting me a draft to look at and comment on at least 48 hours before the deadline. Note: it may be the case that your methods change as the project evolves. This is fine - the proposal that you send to the program is not a binding agreement. I do recommend that you keep track of any and all changes because this will make your life a lot easier later!
There are no Bates-level or neuroscience-level guidelines that we are bound to with regards to your thesis. However, my philosophy is that your thesis is the first document that you are writing as a scientist, and it should be written for other experts. A good “stretch” goal for thesis is that we should be able to submit your final thesis document to a journal like Journal of Cognitive Neuroscience “as is” once you have finished. I encourage you to read journal articles at both a factual level (as you currently do) as well as a meta level in which you note the format, writing style, and types of details that are included or omitted. It’s also helpful to look at the author guidelines for journals. For example, here is one from Journal of Cognitive Neuroscience.
One consequence of this style is that your thesis document is likely to be shorter than those of some of your friends, especially if those friends are writing theses in the humanities. Please do not let this cause you anxiety. I encourage you to think of the thesis as more than the physical document that you turn in at the end of the year - it is the entire process of learning and thinking as a scientist. One thing that some students have liked to do is to include all of the code that they have written as appendices to the document. This can both serve to be open about your scientific process, and also to make a physical manifestation of the work that you have done throughout the year. If these resonate with you, feel free to include your code. But please know that you are under no obligation to do so.
I think that it works best if you write your introduction last. This is because you can look back at your thesis and craft the right story for it once you know the big picture. That said, it’s good to make progress in the fall semester, and I encourage you to send me an annotated bibliography by the end of the fall semester. With this literature review taken care of, I think it works best to start with you sending me an outline of the introduction. I’ll look at it, and we can discuss the global “story” of your thesis. Once we are happy with this, you can fill in the writing.
For analyzing data and making figures, we will be using R. One advantage of this is that you can do the data analysis as you write in the same document, in the same software. This not only makes your life easier, but it also makes your written document more reproducible because it comes directly from the test results rather than from you typing in all numbers by hand from a different program. I like this tutorial to get you started with this process. In our lab Github, I have also cloned an Rmarkdown thesis template used at Reed College on our lab Github. You are under no obligation to use it, but it can help get you started.
While you are reading papers at a meta level, you will notice that most discussions take an “hourglass” shape. By this, I mean that they start and end with very broad things, and narrow down to specific implications of the results in the middle. I find that the following order helps:
I will read and extensively comment on any writing you share with me as long as I have 48 hours of time with it. We will both be happier with your written document if we have worked together to craft each of the sections. Your final thesis document is due to me by 5pm on the last day of finals of the winter semester.
| Criteria | A.level | B.level | C.or.below.level | Percent.of.final.grade |
|---|---|---|---|---|
| Attends one-on-one and group meetings | Always attends both on time and prepared | Small number of missed meetings that were clearly articulated in advance | More than a few missed meetings, or last-minute cancellations for “being busy” | 12 |
| Positively contributes to lab culture | Makes self available in person and on Slack to help fellow lab mates | Helps lab mates, but inconsistently | Does not contribute to helping pilot others’ experiments or other needed tasks | 12 |
| Runs experiments in a timely and organized manner | Is on time to run experiments, has consent and debriefing forms printed and ready, maintains good file structures for projects, files forms promptly after experiment, and does a thorough job cleaning up after the experiment | Either does some (but not all) of the A-level activities, or does each of them in an inconsistent manner | Consistent failure to do one or more of the A-level activities | 12 |
| Treats experimental participants in a professional manner | Thoroughly explains all experimental procedures, attends to participant’s needs, makes efforts to make participant happy and comfortable | Either does some (but not all) of the A-level activities, or does each of them in an inconsistent manner | Consistent failure to do one or more of the A-level activities | 12 |
| Clearly and effectively communicates with thesis advisor | Is proactive about reaching out for help; when asking for help, clearly articulates what the problem is and what steps have been taken to solve it | Is proactive about reaching out for help; working towards clarity in articulating issues | Any of the following: avoids reaching out for help; reaches out for help without attempting to troubleshoot alone; cannot articulate the problem (i.e. “it doesn’t work”) | 12 |
| Abstract clearly and concisely summarizes the study | Clearly states problem and question to be resolved; clearly summarizes method, results, and conclusions | Summarizes problem, method, results, and conclusions, but provides more details than skilled overview | Is vague about the problem; does not provide a summary of the whole project | 5 |
| Introduction clearly defines problem that is being studied | Demonstrates the ability to construct a clear and insightful problem/topic statement with evidence of all relevant contextual factors | Begins to demonstrate the ability to construct a problem/topic statement with evidence of most relevant contextual factors, but problem statement is superficial | Demonstrates a limited ability in identifying a problem/topic statement or related contextual factors | 5 |
| Introduction contains a complete literature review | Provides background research into the topic and summarizes important findings from the review of the literature; describes the problem to be solved; justifies the study; explains the significance of the problem to an audience of non-specialists | Provides some background research into the topic and describes the problem to be solved | Provides an incomplete background; presents background in a superficial manner; insufficient or nonexistent explanation of details to non-specialists | 5 |
| Presentation of empirical findings | Data are analyzed completely and appropriately; statistics are appropriate and reported in correct APA format; figures are clear and contain informative captions | Data are analyzed and reported appropriately, but some secondary analyses may not have been performed; small issues with choices made in data visualization | Data are either incompletely analyzed, or inappropriately analyzed; major errors exist with reporting statistical tests; plots or graphs are incompletely labeled, or have used default parameters that limit their efficacy | 10 |
| Discussion reflects effective synthesis | The writer offers insightful interpretation of the empirical research; writer synthesizes the present results in the context of broader literature; the writer provides specific follow-up ideas for the future and ties this section together with a compelling conclusion | The writer offers interpretation of the research at a somewhat superficial level; attempts are made to synthesize, but somewhat incompletely; follow-up ideas are not fully formed; paper ends rather than is finished | The writer offers little interpretation beyond rephrasing the results; little to no attempt is made at synthesis; the author offers no follow-up ideas, or ideas that are too general to be relevant; conclusion is absent | 10 |
| Mechanics (style, syntax, diction, etc) | Is free or almost free of errors of grammar, spelling, and writing mechanics; appropriately documents sources in APA style; employs scholarly tone and style | Has errors but they don’t represent a major distraction; documents sources; attempts to employ an effective tone and style for the purpose of the paper, but falls short of that goal | Has errors that obscure meaning of content or add confusion; neglects important sources or documents few to no resources; Tone and style undermine the goals of the thesis | 5 |