STAT 158: Design and Analysis of Experiments

Lectures: Tu, Th 9:30am-11:00 am, 2040 VLSB. Instructor: Andrew Bray

Email: [email protected]
Lab: Monday 2-4pm Evans 342. Preferred pronouns: he/him
Monday 4-6pm, Evans 342. Office Hours: 415 Evans
Wed 3-4 pm
Topics: Thu 11 am – 12 pm
• Randomization
• Potential outcomes GSI: Katherine Kempfert
• Randomization inference Preferred pronouns: she/her
• Blocking and regression adjustment Email: [email protected]
Office Hours: Location TBA
• Factorial designs
Tue 3-5 pm
• Repeated Measures / Split-plot designs Fri 11 am – 12 pm
• Noncompliance
• Interference

Texts:
A First Course in Design and Analysis of Experiments, G. W. Oehlert (2010). 1st Ed. Out of print, freely
available online: http://users.stat.umn.edu/~gary/Book.html

Field Experiments: Design, Analysis, and Interpretation, A.S. Gerber and D.P. Green (2012). Available on
electronic reserves: https://ucbears.lib.berkeley.edu/991046968519706532_C122557164/view

Equipment: Access to a computer with an R installation and Internet access will be necessary. If you do not
have access to a computer, you can borrow one from the University library. See
https://studenttech.berkeley.edu/hardware-lending for more details. The Student Technology Equity
Program is another good resource. Feel free to contact the instructor if you have concerns about your access
to needed technology.

This is a working draft of the syllabus and is subject to change.

Learning goals

By the end of the semester you should be able to:

1. Understand why careful design of experiments can aid us in learning from data.
2. Confidently analyze data from a designed experiment and describe the meaning of the results.
3. Evaluate the pros and cons of different potential experimental designs and recommend a design given
information about the question of interest and study constraints.
4. Design, plan, and execute a small laboratory-style experiment to answer a question of interest.

Prerequisites:
STAT 134 and 135 (or consent of instructor). STAT 135 may be taken concurrently (though not

Bray, Stat 158 Spring 2025 1

recommended). STAT 133 is also recommended; in particular, we assume familiarity with the R
programming language. If you do not have experience with R, in the first week you should seek out a brief
introduction to the programming language to get up to speed.

Lectures:

• Lectures are a mix of board work and slides. Most students find it useful to take handwritten notes
during lecture.
• I’ll be posting to bCourses any slides that I use and any computer code.
• Lectures are not recorded. If you miss a lecture, your best bet is to borrow notes from a neighbor or
carefully read through the corresponding sections in the textbook.

Labs:

• Lab content will focus on data analysis, visualization, working on practice problems, and reviewing
content from lecture.
• Labs are graded credit / no credit and are due 9 am Thursday of the week they’re done in Lab. We
will scope the labs so that you should be able to finish them during lab.
• You’ll be running experiments in lab on Feb 24, April 7, and April 28 so attendance is expected
particularly on these dates. Results from the experiments may be needed to do homework problems.

Assessment:

Homework
We anticipate giving 6-8 homework assignments during the semester. Homework will be posted to
bCourses and will generally be due 1 week later (all assignments are due at 9:00 am Thursday). All
homework is due via Gradescope unless otherwise noted. Homework will be a combination of
mathematical exercises and data analysis using the computer. All data analysis should be turned in as a
pdf made from a Quarto document. Any mathematical calculations can be included in the Quarto
document using LaTeX or handwritten then scanned in as a pdf.

Exams
The midterm is in two parts (equally weighted). Part I will be a data analysis in R and submitted as a
Quarto document. Part II will be a paper and pencil conceptual exam. Part I will take place in lab on
Monday Mar 17. Part II will take place in lecture on Thursday Mar 20.

The final exam will be held Wednesday, May 14 11:30 - 2:30 pm. The location will be announced
later in the semester.

Final project
All students will complete a final project in which they will design, run, and analyze data from their
own experiment on a topic of interest to them. Projects will be done in small groups of 3 students. The
project will have intermediate steps along the way to help you pace yourself and make sure that your
experiment is progressing successfully. You will be graded on the quality of the writeup as well as the
quality of the design and analysis. You will not be graded on the outcome of the experiment.

Each member of the group will also individually evaluate the other members of the group, and the
evaluations will not be shared with other members of the group. If there is a problem that appears
based on the evaluations, different students within the group may receive different grades, though I
expect this to be uncommon. This can be true even if no member of the group “complains” about other
students, but merely that I found that the division was inequitable, or that the group did not jointly

Bray, Stat 158 Spring 2025 2

 contribute to the components enough to deserve equal division of the grade (good or bad).

You will be given the option either to form your own group or to be placed into a group by the
instructors. After the initial proposal is turned in, an opportunity will be provided to switch groups if it
turns out that your group is not working well. After this point, it will be difficult to change group
membership. If after this point you feel that there is a problem in your group, you may discuss it with
me privately to find an equitable solution with respect to finishing the project.

The anticipated deadlines associated with the different portions of the project are as follows:

• Tuesday, Feb 13 – project groups formed.

• Thursday, Feb 20 – initial proposal due.
• Thursday, Apr 10 – design proposal due.
• Thursday, Apr 24 – protocol due.
• Thursday, May 8 – final report and self-evaluation due.

Overall score
Your letter grade for the course will be based on a weighted average of your scores for all work in the
semester as follows.
• Homework (each assignment weighted equally): 14%
• Labs: 1%
• Midterm: 25%
• Group project: 35%
• Final exam: 25%
Grades will not be curved. Students scoring 90% or above overall will receive letter grades in the A-range,
students scoring 75%-90% will receive letter grades in the B-range, and students scoring 60%-75% will
receive letter grades in the C-range.

Online Resources

bCourses
Homework assignments, extra readings, and material from lecture will be posted here. I will also
make course announcements through bCourses.

Ed Discussion
I have created an Ed Discussion site for this course, which you can access through the link in
bCourses. This is an online forum to ask questions to fellow students and course staff.

Gradescope
Homework assignments, take-home exams, and regrade requests (see Policies section below) will be
submitted through Gradescope, which you can also access through the link in bCourses.

Policies

Possibility of revisions to course policies

All course policies, including assessment, are subject to change during the semester in response to
unforeseen events including but not limited to public health directives, natural disasters, and medical
emergencies among members of the course staff.

Late Homeworks

Bray, Stat 158 Spring 2025 3

 All students will have 5 late days that they may use for turning in homework after the due date. This
will take the place of any extensions due to sickness or conflicts, unless there are extenuating
circumstances, so use them wisely. To use a late day, you must submit a Google Form (link will be
posted here soon) requesting a late day before the homework is due, or you risk receiving a large
penalty or a zero. Late day requests by email will not be answered. Late days cannot be used for the
group project, the midterm, or labs, and they cannot be requested once homework solutions are
posted. Late days are counted at the 24-hour-period level after the time the assignment was due. E.g.,
the first assignment is due at 9 am, Feb 13. If you turn it in at 9:05 am, Feb 13, it counts as use of one
late day, and if you turn it in at 9:05 am, Feb 14, it counts as use of two late days.

Regrade requests
Once your assignments have been graded, please log into Gradescope to read through the feedback.
If you notice any errors in applying the rubric, please let us know! Regrade requests can be
submitted through Gradescope are due within one week of the release of the graded assignments. In
writing a regrade request, please be specific about the nature and exact location of the error you feel
the grader has made.

Academic Honesty Policy

The student community at UC Berkeley has adopted the following Honor Code: “As a member of the
UC Berkeley community, I act with honesty, integrity, and respect for others.” My expectation is
that you will adhere to this code. Beyond the importance of respecting your fellow students, acting
with integrity in completing course assignments helps ensure that they achieve their purpose, which
is to help you learn and develop valuable statistical understanding and skills.

• The three primary assessments in this class should represent your own understanding and work either
individually (the Midterm and the Final) or jointly with your group (the Project). The project takes
place over many weeks and you have the opportunity to draw on many resources to improve your
report. Submitting work that is substantively not your own – either copied from elsewhere or
transcribed mostly unchanged from an LLM – is considered academic misconduct.
• The role of homeworks and labs is practice for primary assessments of the exams and project. Again,
submitting work that is substantively not your own – either copied from elsewhere or transcribed
mostly unchanged from an LLM – is considered academic misconduct. More importantly, it is self-
defeating. These only serve as useful practice if you struggle with them. Please stop by office hours
to work through any questions you have.
• LLMs can be helpful as rich replacements for google search or stack overflow, particularly for
coding. Their use is permitted, but ask yourself two questions as you use them: is this saving the me
the (productive!) struggle of working this out myself? And: Is the work that I’ve submitted
substantively my own?
• On all written assignments, including the homework, you should include a section listing all the
sources you drew on in producing your answers. This includes the names of students you worked
with, resources that you’ve found online, or LLMs (in which case please describe precisely what you
used them for – I’m very curious to know!).
• If you have any questions about whether something would be considered academic misconduct,
please ask!

Anyone caught cheating will be receive a grade penalty and will be reported to the University’s Office of
Student Conduct.

Inclusivity and Accommodation

My hope is to establish a learning environment in this course that welcomes diversity of thought,
perspective, and experience, and to be respectful of your individual identity as a student. I am happy
to use your preferred name and/or personal pronoun. If you feel uncomfortable due to anything that is

Bray, Stat 158 Spring 2025 4

 said in class, or if you feel that your performance in the course is being impacted by experiences
outside of class, please do not hesitate to reach out to me about your concerns.

In addition, if you need accommodations for any physical, psychological, or learning disability, please
speak to me after class or during office hours. Please note that you must make arrangements in a
timely manner through DSP so that I can make the appropriate accommodations.

Acknowledgments

Most of the materials used in this course, including this syllabus, are close adaptations from materials
originally created or compiled and generously shared by Prof. Elizabeth Purdom and revised by Prof. Sam
Pimentel. In writing this syllabus I also adapted content from Prof. Chris Paciorek, and from Prof. Monica
Linden of Brown University.

Bray, Stat 158 Spring 2025 5

Calendar
The following is only a guide, and there is likely to be slight variation as the semester progresses.

O=Oehlert, G&G = Gerber & Green, *=reading not from book, available on bCourses.
Week Topics Lab Corresponding Assignments
Reading Due
Jan Course Logistics Anchoring Experiment O: 1-2.3
20-24 Components of an (in lecture) G&G 1
Experiment *A/B testing
article
Jan Hypothesis Testing O: 2.4
27-31 Comparison of two G&G: 2.1-2.2,
conditions 2.4, 2.6
Permutation tests
Feb Randomized Designs O: 3.1-3.8 Th: HW1 Due
3-7 with one factor (CR[1])
Feb Designs with 2 factors O: 8.1-8.6 (can Th: Project
10-14 (CR[2]) skip last part of Groups Due
8.2, p 169-170) Th: HW2 Due
Feb 17- Designs with more than No lab (holiday) O: 8.7, 9.2.1 Th: Initial
21 2 factors (CR[k]) Proposal Due
Feb 24- Comparisons, Contrasts Eye-hand coordination O: 4.1-4.3, 5.1- Th: HW3 Due
28 and Confidence experiment (attendance 5.4.3, 9.1
Intervals required) G&G 3.5
Mar Block designs, Latin O: 13.1-13.3, Th: HW4 Due
3-7 squares G&G 3.6.1
Mar Split Plot O:16.1-16.2
10-14 Repeated Measures
Mar Review (Tu) + Midterm
17-21 (Mon and Thu)
Mar Spring Break Spring Break Spring Break Spring Break
24-28
Mar 31- Random Effects O: 12.1- Th: HW5 Due
Apr 4 12.3,12.15-12.6
Apr Power Paper Bridge O: 7, 10.3 Th: Design
7-11 Experiment (attendance Proposal Due
required)
Apr Multiple regression and G&G 3.6.2, Th: HW6 Due
14-18 experimental design G&G 4.3-4.4
Cluster designs
Apr Noncompliance G&G 5, 8 Th: Protocol Due
21-25 Interference
Apr28– Heterogenous treatment Student Experiments G&G 9 Th: HW7 Due
May 2 effects (attendance required)
May RRR RRR RRR Th: Final Project
5-9 Due
Th: Self-
Evaluation Due
May Final Exam: Wed May 14 11:30-2:30 pm
12-16

Bray, Stat 158 Spring 2025 6