Introduction

Welcome to Notre Dame

Welcome to Notre Dame. Welcome to Research Methods.

This course is about learning how to do research in computing: how we turn curiosity into claims, and claims into evidence that other scientists can scrutinize, reuse, and build on.

Quick introductions

In one sentence:

• Name + research area (or what you think you might work on)
• One thing you hope this course helps you do better this semester

Who am I in your PhD

A PhD is a multi-year apprenticeship: you learn to do research with an advisor, and (ideally) solve one hard problem really well (maybe two).

This course is not here to replace your advisor. In this class, I’ll give you tools, vocabulary, habits, and a process you can carry into any lab and any subfield. But always defer to your advisor on what matters most for your topic, community, and publication norms.

What I am: a crash course in the mechanics of research. How to read papers, form questions, build evidence, reason about validity, and communicate well.
What I’m not: your dissertation committee, your lab manager, or the final authority on your subfield.

Logistics and expectations

Where things live

• Canvas will be our hub for announcements, prompts, and discussion.
• If we ever need remote flexibility, we’ll use whatever format is simplest and most reliable, but the default expectation is: be present, participate, and be ready to discuss.

How each week works

Most lectures have pre-reading and a short typed response.
The goal is not busywork. You need to show up with something to say.

Participation norm (important)

This will be discussion-heavy. If you’re quiet in a given session, that’s fine, but you should have something prepared: a question, a critique, a confusion, a connection.

Why I teach this course

Early in the PhD, many students get stuck in the same place: they are working hard, but don’t yet have a stable mental model of what research is supposed to look like day-to-day.

Students spend a lot of time thinking: “What problem am I going to solve in my PhD?”
Sometimes the advisor assigns a problem from a funded project. Often the advisor has a plan—but also hopes you’ll figure things out fast and contribute original direction.

So, the real early-game questions are practical:

The “early PhD questions” checklist

• What is the problem, exactly?
• How do I find out more about it (what keywords, what communities, what venues)?
• What do we already know? What don’t we know?
• How do I survey the literature without drowning?
• What does “improvement” actually mean here?
• How do I know if my solution is correct or merely plausible?
• How do I present and write this work so it survives peer review?

This course exists to make those questions less mysterious.

What you’ll learn in this course

By the end of the semester, you should be able to:

• Start a research project and follow a plan
• Read research papers critically
• Apply statistics correctly (and recognize when they’re being misused)
• Communicate clearly: argue, write, and present cogently
• Review other people’s work professionally
• Operate like a scientific professional

What is Research?

1. It is systematic, investigative process by which our knowledge about the universe is improved and refined.
2. Two general categories:
   1. Basic research aimed at increasing general scientific knowledge
   2. Applied research aimed at solving problems or developing new processes, products, tools, or techniques.

In-class activity: “What worries you about the next 5 years?”

Take 2 minutes. Write down (privately) what worries you most about the next five years of graduate school.

Common themes:

• “I don’t know what my thesis topic is”
• “I feel behind”
• “I’m not sure how to read papers efficiently”
• “I don’t know what counts as a contribution”
• “Imposter syndrome”
• “Publishing feels opaque / political”
• “I’m worried I’ll waste time building the wrong thing”

What does it take to become a successful scientific professional?

Four traits matter more than raw intelligence:

• Work ethic — research is hard work
• Resiliency — you will fail; successful PhD students persist
• Creativity — the best papers contain something genuinely new
• Communication — clear writing/speaking determines whether ideas spread

A blunt truth: if your work can’t be explained clearly, it effectively doesn’t exist to the community.

These are the objectives of the course:

1. Introduce and discuss concepts in research methodology, empirical analysis, and the scientific enterprise in computing.
2. Prepare students for conducting research by examining how to plan, conduct, and report on empirical investigations.
3. Introduce core steps of a research project, including formulating research questions, theory building, data analysis (using both qualitative and quantitative methods), building evidence, assessing validity, and publishing.
4. Students will have a working understanding if principle research methods used to study human interaction with computer technology including controlled experiments, case studies, surveys, archival analysis, action research and ethnographies.
5. Students will also understand pertinent topics in peer review, ethical obligations involving human subjects research, how to give a scientific presentation, and how to write research papers, survey papers, and funding proposals

---

Eddington’s Two Tables: why research feels “weird”

I need not tell you that modern physics has by delicate test and remorseless logic assured me that my second scientific table is the only one which is really there–wherever “there” may be. On the other hand I need not tell you that modern physics will never succeed in exorcising that first table–strange compound of external nature, mental imagery, and inherited prejudice–which lies visible to my eyes and tangible to my grasp. […]

Yes, no doubt they are ultimately to be identified after some fashion. But the process by which the external world of [science] is transformed into a world of familiar acquaintance in human consciousness is outside the scope of [science]. And so the world studied according to the methods of [science] remains detached from the world familiar to consciousness, until after the [scientist] has fashioned his labours upon it. Provisionally, therefore, we regard the table which is the subject of physical research as altogether separate from the familiar table […]

It is true that the whole scientific inquiry starts from the familiar and in the end it must return to the familiar world but the part of the journey over which the [science] has charge is in foreign territory.

Arthur Eddington

Eddington’s point: there’s the table you experience (solid, colored, touchable) and the table physics describes (mostly empty space, atoms, fields)

1. The table you have in front of you. It has a weight, shape, color, and lots of properties. We can put books upon the table, etc.
2. From a different perspective this same table is actually made up of atoms, which are actually mostly empty space. We can’t actually touch the table, and atoms don’t have any real color.

Familiar table: the real-world thing we care about
Scientific table: the proxy/metric/model we actually measure

Some examples:

• ML fairness — Familiar: “fair treatment.” Scientific: parity/equalized-odds/calibration metrics.
• LLMs/NLP — Familiar: “understands meaning.” Scientific: loss, benchmarks, preference scores.
• Systems — Familiar: “fast + reliable.” Scientific: p95 latency, throughput, SLOs, synthetic workloads.
• Security/Privacy — Familiar: “safe + trustworthy.” Scientific: threat model results, formal guarantees, DP ε, pen tests.
• HCI — Familiar: “good experience.” Scientific: SUS, task time, errors, CTR/retention, Likert scales.
• Software eng — Familiar: “maintainable quality.” Scientific: coverage, complexity, warnings, churn/defects.
• Theory/algorithms — Familiar: “efficient solution.” Scientific: asymptotics, worst-case bounds, approximation ratios.
• Social/data computing — Familiar: “influence/community.” Scientific: centrality, engagement, diffusion, clustering.

Bring it back:

• Science takes us into “foreign territory,” but must return to the familiar world with something usable.
• A lot of research-methods pain comes from confusing the measurement world with the lived world.

This is something that deserves thought.

“Science works” (and why that’s not the whole story)

Science is the most powerful tool humanity has for understanding and controlling the world.
It’s everywhere: buildings don’t collapse; airplanes fly; vaccines work.

But: “science works” is not the same as “science is simple,” or “science is purely objective,” or “science tells us everything that matters.”

So this semester we’ll keep returning to:

• How we know what we know
• How methods create (and limit) claims
• How humans and institutions shape the scientific enterprise

Big questions to keep in your pocket all semester

• What separates science from pseudoscience?
• When do we earn the right to call something a “law”?
• Are all sciences “just physics,” or do higher-level explanations stand on their own?
• Does science describe reality, or does it build useful models?
• Is science objective, or does it have perspective (since scientists are human)?

You don’t need answers today. But you should learn to argue about them carefully.