The Art of Nudging

On a friend's complaint about a PI not getting back to her about the next stop on a project, I was reminded of this neat advice from cognitive psychologist Dennis Pelli, on his great page of concrete writing advice:

Nudging. Collaboration is wonderful. The key ingredient is that you both must need each other. That's what will carry you through the hard patches. However, many manuscripts die sitting on the desk of someone who is planning to get back to it soon. How do you get it moving again? This is often described in moral terms specific to the personal association, but, after many years of sending and receiving reminders, I've come to think that it's a professional skill. Some people are good at it and they collaborate to produce many papers. Others aren't immoral; they just aren't good at it. Watching, from both sides, what works and what doesn't, I note that there is a trick to it. Start very very mildly, lightly reminding. And stay there. Don't escalate. This is counter-intuitive because, as a sender, one is embarrased by the implicit criticism of the reminder, and one feels a need to justify the action by moralizing and describing dire consequences. But all that negative stuff discourages the recipient who probably needs only the reminder and perhaps some encouragement. And, of course, do it. Always very lightly, but frequently enough to keep the paper in your recipient's mind. Mastering this unsung skill and collaborating with good nudgers — nudging and being nudged — may greatly increase the number of papers that you publish.

Some Hard-Earned Principles for Managing Data and Experiment Code

This is where I'm at now, after 10 years in vision science and many painful experiences with messing up data at some unknown time in the past, or having to redo grueling Excel-based analyses. If you can start this way from the very beginning, you'll have a nicer time than I did.

A lot of this has to do with what I would call "personal forensics": can you reconstruct where the data came from? Can you find the code for a particular experiment, even 5 years later? Can you rerun the analysis and get the same result? But it should also help to avoid getting snarled up even in day-to-day work, e.g. moving code and data between development and experiment machines.

These are framed in single-user terms, but of course this stuff would ideally be thought of in terms of lab policy (and usually you should follow the policy of your lab rather than these if there's a conflict). It's also limited by my experience, which is primarily psychophysical and eyetracking experiments - there are probably many special principles that apply for massive data files, that have many processing steps before they turn into delicious chart and statistical output.

Also wanted to say that a number of these were inspired by my friend Tom Wallis's blog, such as this excellent post about how to organize your folders. Here we go:

• On the top level of folders, organize by project (rather than type of file), keeping everything for a particular project together.
• As much as possible, use text files for all the raw data output of your experiment - probably in comma separated format (those open automatically in Excel, if they have the extension .csv, and are easily imported by every analysis package). Then the first row will be descriptive column headings (no spaces). I use separate files for separate sessions, and have each row represent a trial.
• Put lots of identifying info into the data file name (underscores rather than spaces) so that it makes sense on its own - at least the experiment title, the participant ID, and the complete date. Some people put the time in seconds too.
• After each participant, add notes about the run, including anything that went wrong and any observations they made, to the end of a text file in the same folder as the results. (and labelling the entry with the same convention as the data file)
• Don't ever touch the raw data files, except to fix errors in them (and make sure the original has been committed if so.)
• A column with a timestamp for each trial down to the second (such as the 7 digit number returned by the Psychophysics Toolbox's GetSecs) can be helpful both for unambiguously identifying trials, and for evaluating the time course of an experiment.
• Generate intermediate numerical files when necessary, also in text format, and store them in a different folder than the raw data.
• Do all statistics and summarizing in scripts, in something like Python, Matlab or R - something very general, that has a lot of support and longevity. Matlab is probably the worst bet at the moment, since it is the only closed source of the three (and expensive), and many scientists and mathematicians are deciding they don't want to put their time into adding value to a closed program that is not available to everyone (and thereby increasing lock-in).
• Also do all the figure generation in one of those languages, with a single script that can go all the way from the raw data to the final, publication-quality figures with no manual tweaking on your part. Also generate them at the proper scale.
• Output the figures in PDF format (or some other vector art format, but PDF appears to have the most support).
• Keep the analysis scripts in a separate folder from the experiment code.
• Put all the files (except the pdfs or other bulky binary files), including experiment code, analysis code, and data, under some kind of version control, such as Git or Subversion. If you have any folders with any variation of "old version" in the title, you need version control.
• Commit whenever there's a big significant change, or important fix. Commit raw data immediately upon collecting, so that the uncontaminated data is always safe. It may not be to the forensic standard of a court of law, but for your own auditing of your data it is plenty of protection.
• As is the software engineering wisdom, only commit code that you have tested at least a little and feel confident that it runs (because it's harder to retroactively investigate code that doesn't run), and write sensible comments to go with the version.
• When there needs to be multiple copies of code on multiple computers, make sure they're linked up properly with version control (in Git this requires the Clone command), and then resynced when necessary.
• If you can, have all your stuff, from every phase of your career, on one hard drive. Apart from big video files or giant data files, with hard drives getting bigger all the time (and search getting faster and smarter) there's no reason not to keep it together, and it gives you a target for backup.
• The bare minimum backing up of data is automated, daily backup, of everything, to a device you control. Anything less than this is incredibly risky! Ideally it will also be incremental (using something like Time Machine on the Mac, Windows Backup on WIndows, or rsync on Linux), meaning that you can "rewind" to versions on earlier dates. That protects against cases of data getting invisibly corrupted and then backed up.
• It probably helps a bit to have lots of incidental data redundancy, such as DropBox and additional copies of the repository on different computers/USB keys (and any kind of possibly-unreliable university- or company-wide automated backup). However, don't let those redundancies relax your vigilance with regard to the first type of backup. Do not spend any time on the "copy two or three folders manually once in a while" backup strategy, as it is almost worthless.
• On the other hand, having additional off-site backup is a very good idea - earthquakes might not be an issue where you live, but I've known several labs that have been burgled. One method is to rotate complete clones of the hard drive monthly, say, and take the unused drive to some other location. If privacy isn't a concern, then an internet-based backup service like Carbonite may be a good solution for this too.

This was originally an email to a friend who's starting his Ph.D. in Psychology, and Jim Davies said I should post it, so sorry it's just a huge unstructured list on scattered topics.

There are two big potential speedbumps that I see to adopting these: one is learning those programming tools that can automate manipulating text files - for example, how to paste together output files from multiple participants to make one aggregate file, including stripping off the header row and adding a column for the participant. This all goes with Tom Wallis's principle of, don't touch your data "by hand", which I agree with, and is necessary to achieve the ideal of automating all the steps to go from raw data to submission-quality figure and statistical output. The other is learning to use version control software such as Git, which definitely requires conceptual investment.

For some people, learning this way of working might just be too much of a pain, and not worth it relative to just getting on with the science, in whatever tools they are used to. But if you think you can get past those speedbumps, I firmly believe it will pay off bigtime, and keep paying off throughout your career. It should let you spend more of your time doing the important, fun parts of science - and less time on resizing a figure by hand for the 27th time, or late, panicky nights trying to figure out where those spreadsheet numbers came from.

Numbered Life Advice from Some People I Admire

A lot of people I admire have been writing advice in list form recently! Here are some that resonate with me:

John Hodgman:

1. Try to be around people that make you feel genuinely happy

2. Disengage from people who make you feel bad and don’t care about how you feel

3. Do more favors than you ask for

4. Remember it always hurts to ask

5. Let people know when you are genuinely thinking nice things about them and be alert to the more frequent times when you are not thinking about anyone but yourself at all, and remember that we are all like this, and so if someone lets you down, let them off the hook. You are letting plenty of people down all the time in small ways and that’s how it goes being an individual human being.

6. Don’t sit on the same side of a booth in a restaurant even if you are lovers

7. Don’t leave a lot of voicemails

8. Please don’t write lots of long emails

9. Be nice

and that’s all you need to do.

Andrew WK:

To be a real human being, you must try...

1. To care about someone and something more than yourself.

2. To accept help from someone even when you believe you don't need anyone.

3. To cheer people up and bring them simple joy in times when it seems hardest to smile.

4. To bring loving comfort and sincere hugs in the midst of violence, pain, and suffering.

5. To recognize your own shortcomings and failings before lashing out at another's weakness.

6. To have true compassion when someone's in a bad mood, with the understanding that they might be going through a hardship you're not aware of.

7. To constantly remember that life is a fragile and precious miracle which requires all our collective effort to protect.

8. To humbly work to improve our own defects and cut everyone else a little more slack.

9. To remember that being a loving and positive person isn't always easy, but it's always worth it.

10. And lastly, to never give up on the power of humanity and on your own potential to be a caring, loving person.

(something I like is that this was his reaction to a list of "how to be a real man", rejecting all that jazz)

Werner Herzog:

1. Always take the initiative.
2. There is nothing wrong with spending a night in jail if it means getting the shot you need.
3. Send out all your dogs and one might return with prey.
4. Never wallow in your troubles; despair must be kept private and brief.
5. Learn to live with your mistakes.
6. Expand your knowledge and understanding of music and literature, old and modern.
7. That roll of unexposed celluloid you have in your hand might be the last in existence, so do something impressive with it.
8. There is never an excuse not to finish a film.
9. Carry bolt cutters everywhere.
10. Thwart institutional cowardice.
11. Ask for forgiveness, not permission.
12. Take your fate into your own hands.
13. Learn to read the inner essence of a landscape.
14. Ignite the fire within and explore unknown territory.
15. Walk straight ahead, never detour.
16. Manoeuvre and mislead, but always deliver.
17. Don't be fearful of rejection.
18. Develop your own voice.
19. Day one is the point of no return.
20. A badge of honor is to fail a film theory class.
21. Chance is the lifeblood of cinema.
22. Guerrilla tactics are best.
23. Take revenge if need be.
24. Get used to the bear behind you.

Justin McElroy:

-You can be funny and kind or funny and cruel. The second one is easier, but the first one is worth it.

-Dip the french fry in the Frosty. Go on, try it.

-Habit is a powerful force we forget about until it’s turned against us. Be careful which ones you create.

-You will remember the most embarrassing crap you do in your life forever and in perfect clarity. Everyone else will remember the kindest things you do. It all comes out in the wash.

-If you’re doing a remote podcast, it’s worth it to record audio locally and mix it together. Trust me on this one.

-You’re the only one who can let go of your grudges. It’s worth it, I promise. They’re not doing you any good.

-Doing the good, brave, kind things can feel silly if you let your internal critic get in the way. Reminder: No one else can hear that guy.

-I can count on one hand the number of times putting out negativity has brought me back something worthwhile and even when it works it feels terrible.

-Want to be better-liked immediately? Today? Right now? Use people’s names. Ask more questions. Make the person you’re talking to feel important without empty flattery.

-Don’t correct people. Unless their wrongness will lead to them getting hurt or hurting someone else. You’ll have a fleeting sense of superiority and they’ll resent you. Nothing worthwhile comes of it. This used to be so hard, but now I cringe when someone else does it.

-Cooking a Hot Pocket in the oven may seem counterintuitive, but man, it can really elevate it.

-Learning to appreciate things you don’t initially enjoy is the power to fill the world with stuff you like.

-Empathy is the final step of maturity. It can take some work, but you’ll be shocked how much easier the world is to navigate when you remember to use it.

-You’re probably not drinking enough water.

-There’s no narrative to your life, no arc, no reward for achieving all the things you want. That kind of thinking is a recipe for a you-centric world view and is a very lonely road. Focus instead on the role you play in the stories of others. When I was young, there was an old man named Brady at our church who always had gum. No matter what, Brady always had gum that he offered up freely. When he died, it devastated our youth group and I still remember him two decades later. Brady might have only played a bit role in all our life stories, but by playing it with generosity and kindness he achieved a sort of immortality. Putting others first with a cheerful heart isn’t easy, but because of that, even the smallest acts can leave an incalculable impact.

-Don’t read the comments. And when you do, only reply to the nice ones.

Checklists Will Save Your Butt

One of the things that makes preparing for travel the most stressful for me is worrying about forgetting to pack something. There's too many things to keep track of, and I have in fact forgotten very important things on numerous trips. Including my underpants. (which is why to this day some of them have tags in swedish)

This time I'm using checklists, and it's made all the difference. Whenever I thought of something that I need to remember to take on my trip, I added it to a "to do" list on my phone. Then in the last frantic minutes before I left all, I had to do was to go through the list.

This is an obvious idea, but I think it's more powerful than I realized. I read a whole book about it, called the Checklist Manifesto. It wasn't very good, but I got the main point, which is that you should use checklists. Super highly trained professionals with thousands of hours of experience, like airline pilots and surgeons, use them, so there's no shame in it.

I keep a checklist for things to pack, and one for leaving hotel rooms, and one for before I run a participant on my experiment and one for after they finish. I'm sure I'll find lots more uses for them in the future.

The most important point is that it be a physical list, and that you physically check things off. A "mental checklist" doesn't count. At all!

The whole point is to stop that feeling of constantly rereminding yourself of important things - what David Allen in Getting Things Done calls "open loops". That's stressful, and that takes away from your focus. Also, your brain sucks at it.

But really keeping that list, and really checking it off, always takes willpower for me! There is always a voice in my head that says, "this time, you've got it - you don't need to actually go through the list". And that's when things go south.

Look at it this way: say you have a list of 8 items. Even if you would only forget one item, one time in five (although for me it's more like one time in one) that one item will still be worth the few seconds it takes to check it. Especially if that item is your passport or visa. I'll just keep telling myself that.

More checklist tips:

• Start making it right away. As soon as packing is on your mind. And throw things on there as soon as you think of them, thus closing the "open loop".
• If you use an electronic checklist, like a "to do" app on your phone, then it is easier to always have on hand. I use a nice basic open source one called Simply Do.
• With an electronic one you can also reuse it: simply uncheck all the items.
• Keep it pruned, complete and current. Try to make sure every single thing you need to remember is on there, and don't have a single thing on there you don't need to remember (at least don't leave it unchecked)

Trusting your checklist is hard, and wariness of false security is a good instinct. But I think that's where you should focus your effort, on your checklists, not on constantly mentally scanning for things you've forgotten: even with a simple trip, there are too many things, and stuff will fall out, like a boarding pass from a loosely-held boarding pass envelope. (something else that has happened to me!)