Syllabus

Pre-requisites

Students are expected to have completed the Psychology Deparment’s graduate statistics course series (522, 523, 524, 525), and have thus gained at least some rudimentary experience in a scripting language such as R (or Python).

Learning objectives:

  • Students will evaluate the reproducibility and replicability of psychological research papers.
  • Students will automate research analysis activities.
  • Students will scale computing to very large datasets.
  • Students will collaborate using version control tools.
  • Students will use non-parametric/resampling statistical analysis methods.
  • Students will use high-performance computing instruments.
  • Students will evaluate different approaches to visualizing and presenting conclusions from large and high-dimensional data.

Number of credits, activities, and hours:

  • Credits: 3

  • Lecture: 1.5 hours (Monday 1:30 - 3pm, Kincaid 108)

  • Lab / Studio: 1.5 hours (Wednesday 1:30 - 3pm, Kincaid 108)

  • Reading and individual reflections: 3 hours

  • Group projects: 3 hours

  • This is an estimated average, and the distribution across the latter two will probably vary from week to week.

Office hours: Tuesday 4:30 - 5:30, Kincaid 501

Repeatable credit: No

Evaluation details:

  • Reading reflections: 30%
  • Studio participation (includes providing feedback!): 30%
  • Final project: 40%

More details

Reading reflections should be approximately 100 words that focus on the attempt to actively connect the reading to your subject matter interests in Psychology. Posted to GitHub discussions forum for the course. Due on lecture days at 1:30PM.

Studio participation includes a pull request of content to be presented as a PR on the course repository:

  1. Project proposal: Approximately 250 words that describe the aims and scope of the project. Due 4/10 at 1:30PM

  2. Project progress report: Approximately 250 words describing progress thus far. Due 4/24 at 1:30PM.

  3. Data visualization: a data visualization (code and visual) from your project. Due 5/15 at 1:30PM.

  4. Project ethics statement: Approximately 250 words describing ethical issues arising from the nature of the project. Due 5/22 at 1:30PM

Each of the above tasks will be scored as 0: not completed, 1: completed, but lacking, 2: completed to full spec. Each one of these assignments will factor in equally for a total of 60% of the final grade.

Final project has two components:

  1. An in-class presentation on 5/29.

  2. A project report: up to 6 pages (including figures), submitted as a pull request on the course repo, due 6/5 at 9am.

Each of these components will be scored between 0 and 20, accounting for a total of 40% of the final course grade.

Extra credit : Pull requests that fix errors (typos, factual errors, etc) on the slides, available at https://uw-psych.github.com/psych532-slides/. Extra credit can be used to complement missed assignments, or to improve the overall grade in the class.

Class schedule:

Lecture: Mon 1:30 - 3pm, Kincaid 108

Lab / studio: Wed 1:30 - 3pm, Kincaid 108

Office hours: Tue 4:30 - 5:30pm, Kincaid 501

Schedule

Date Topic Readings Slides Comments
3/25 Lecture: The intellectual foundations of data science Donoho, “50 years of data science”; Breiman “Statistical modeling: the two cultures”; Halevy et al. “The unreasonable effectiveness of data” slides
3/27 Lab: version control Neuroimaging & data science, chapter 6
4/1 Lecture: Reproducibility Open Science Collaboration “Estimating the reproducibility of psychological science”; Rokem, Staneva & Marwick “Assessing reproducibility”; Yarkoni “The generalizability crisis” (only the main article!) slides
4/3 Lab: version control, cotd. Neuroimaging & data science, chapter 6
4/8 No class
4/10 Studio: project proposals
4/15 Lecture: Best practices for working with Big Data Wickham “Tidy data”; Wilson et al. “Good enough practices in scientific computing”; Bryan, “How to name files” slides
4/17 Lab: continuous integration
4/22 Lecture: Statistics with big data Altman & Krzywinski “The curse(s) of dimensionality”; Efron and Diaconis, “Bootstrapping”; Geoff Loftus “Psychology will be a much better science when we change the way we analyze data” slides
4/24 Studio: project progress
4/29 Lab: High performance computing w/ Altan Orhon
5/1 Lecture: Machine learning Yarkoni and Westfall “Choosing Prediction Over Explanation in Psychology: Lessons From Machine Learning” slides
5/6 Lecture: The FAIR principles Wilkins et al. (2016). The FAIR Principles slides
5/8 Lab: the “hidden curriculum” of software engineering Evans “The pocket guide to debugging” (physical copy!); XXX
5/13 Lecture: Data visualization with large and high-dimensional data Allen et al., “Data Visualization in the Neurosciences: Overcoming the Curse of Dimensionality”; Goldstone, Pestilli, Börner “Self-portraits of the brain: cognitive science, data visualization, and communicating brain structure and function” slides
5/15 Studio: data visualization
5/20 Lecture: Ethical considerations in data-intensive research Alexandra Paxton “The Belmont Report in the Age of Big Data: Ethics at the Intersection of Psychological Science and Data Science” slides
5/22 Studio: project ethics statement
5/27 Memorial day No class
5/29 Project presentations Final project report is due June 5th at 9AM

Course communications

Whenever possible, please use the course GitHub Discussions forum for questions that may be of interest to multiple students in the course. Individual questions can also be addressed to Ariel at arokem@uw.edu. When emailing please include “[PSYCH532]” in the subject line.

Feedback

This is the first time that this course is taught, so feedback on the course content or format would be If you would like to provide feedback during the course of the quarter, you are welcome to send Ariel an email at any point. In addition, we will conduct a mid-quarter.

Diversity and inclusion

The University of Washington’s Department of Psychology is committed to promoting diversity and fostering equity and inclusion in all aspects of its activities and initiatives. This course in particular is is explicitly designed to bring learners from a variety of different backgrounds and from all department research areas, into the practice of reproducible data-driven discovery. In order to foster an inclusive learning environment we encourage the following kinds of behaviors on all platforms and during all events, both in-person and online:

  • Use welcoming and inclusive language
  • Be respectful of different viewpoints and experiences
  • Gracefully provide and accept constructive criticism
  • Show courtesy and respect towards other community members

On the other hand, we will not tolerate discrimination or harassment based on characteristics that include, but are not limited to gender, gender identity and expression, sexual orientation, marital status, disability, physical appearance, body size, race, age, national origin or religion, educational background or research interests. If you find that you or someone else have been the traget of such behaviors by any participant or by the instructor, we ask that you follow the Psychology Department’s process for bias reporting.

Religious Accommodations

Washington state law requires that UW develop a policy for accommodation of student absences or significant hardship due to reasons of faith or conscience, or for organized religious activities. The UW’s policy, including more information about how to request an accommodation, is available at Religious Accommodations Policy https://registrar.washington.edu/staffandfaculty/religious-accommodations-policy/. Accommodations must be requested within the first two weeks of this course using the Religious Accommodations Request form https://registrar.washington.edu/students/religious-accommodations-request/. For this course, accomodations may come in the form of absence from some course sessions or adjustment of the timelines for submission of assignments.

DRS Accommodations:

It is the policy and practice of the University of Washington to create inclusive and accessible learning environments consistent with federal and state law. If you have already established accommodations with Disability Resources for Students (DRS), please activate your accommodations via myDRS so we can discuss how they will be implemented in this course.

If you have not yet established services through DRS, but have a temporary health condition or permanent disability that requires accommodations (conditions include but not limited to; mental health, attention-related, learning, vision, hearing, physical or health impacts), contact DRS directly to set up an Access Plan. DRS facilitates the interactive process that establishes reasonable accommodations. Contact DRS at disability.uw.edu.

Academic support

A list of University of Washington academic and non-academic support programs is available in http://academicsupport.uw.edu/campus-resources/.