Lesson 104 — Comparing Practicalities and Implications of Data Collection Techniques
Strand: Statistics | Descriptor: AC9M8ST01 | Duration: 45 minutes
Learning Intentions
- To compare census, sample, experiment and observation in terms of cost, time, accuracy and ethics.
- To recommend and justify a data collection technique for a real scenario.
Success Criteria
I can:
- Compare census and sample in terms of cost, time and accuracy.
- Compare experiment and observation in terms of practicality and ethics.
- Explain at least two ethical considerations in data collection.
- Recommend an appropriate technique for a scenario, justifying the trade-offs involved.
Warmup
(6 minutes — quick calls, pairs, mini whiteboards)
For each, call census or sample, and explain your call in one sentence:
- The ABS wants to know the exact population of Australia on a given night.
- A phone manufacturer wants to know how satisfied customers are with battery life.
- A teacher wants to know the average height of their own class of
students. - A pollster wants to predict the outcome of a federal election three days before voting.
Answers: 1. Census — exactness for the whole country is the entire point of the exercise; 2. Sample — asking every customer worldwide would be far too slow and costly for a routine satisfaction check; 3. Census — the “population” here is just
Today’s question: given a real problem, which technique should you choose — and what do you gain or lose by choosing it?
Activities
Activity 1 — Explicit Instruction: the Trade-offs (14 min)
I do — census versus sample:
| Factor | Census | Sample |
|---|---|---|
| Cost | Very high — the 2021 Australian Census cost over | Much lower — a national opinion poll of |
| Time | Slow — the ABS spends years planning, months collecting, and over a year processing Census data | Fast — a poll can be conducted and reported within days |
| Accuracy for the whole population | In principle exact, but never perfect — some people are missed (homeless people, those overseas) and some data is entered wrongly | Involves sampling error — the sample’s result will not exactly match the true population value, but with a well-chosen sample the error is small and can be estimated |
| Practicality | Only feasible for organisations with major resources (national governments); disruptive to collect from everyone | Practical for almost any organisation; the challenge shifts to choosing a sample that fairly represents the population (Lesson 106) |
I do — experiment versus observation:
| Factor | Experiment | Observation |
|---|---|---|
| What it can establish | Cause and effect — because the researcher controls what changes | Only association/patterns — you cannot be certain what caused a pattern you merely watched |
| Practicality | Often requires careful control of conditions (e.g. matching plots of land, matched groups of volunteers) — harder to arrange | Often easier to set up — no need to control conditions, just to record accurately |
| Ethics | Raises real concerns: is it fair or safe to give one group a treatment and withhold it from another? Human and animal experiments require ethics approval and informed consent | Generally lower risk, since nothing is done to anyone — but privacy still matters (is it acceptable to record people without them knowing?) |
We do — apply the tables. For each pair, use the tables to argue which technique is more appropriate and why:
- A hospital wants to know if a new physiotherapy exercise speeds up recovery after knee surgery.
- A council wants to know how many native birds live in a local wetland.
(Expect: 1. An experiment — comparing patients who do the exercise with those who do not, but only ethically with informed consent, because it directly involves people’s health and recovery; a good design uses a sample of patients, not every patient in Australia. 2. Observation — counting birds without disturbing them is safer for the animals and simpler to arrange than any “experiment” on a wetland.)
Activity 2 — Guided Practice: Recommend and Justify (14 min)
Pairs. For each scenario, recommend a technique (or combination), and justify using cost, time, accuracy and ethics where relevant.
- A government wants to know the exact number of people needing emergency housing this year, to allocate funding fairly.
- A soft drink company wants to compare two new flavours to decide which to launch.
- A university tests whether a new teaching method improves exam results, using two of its own tutorial classes.
- A wildlife researcher wants to estimate the number of sharks along a coastline without catching or tagging them.
- A social media company wants to know how a proposed design change affects how long people use the app, so it trials the new design with
of users before deciding.
Circulating prompts:
| Prompt | Purpose |
|---|---|
| Who is harmed, inconvenienced, or has their privacy affected by this method? | Surfaces the ethical dimension explicitly. |
| Could a smaller sample answer this “well enough”? | Tests whether a census is truly justified by the stakes. |
| Would people behave differently if they knew they were being watched or tested? | Introduces the idea that observation and experiments can change behaviour just by happening (the observer effect). |
| Is consent needed here — and has it been given? | Ethics test for anything involving people directly. |
(Sample discussion: Q1 leans census-like, because funding fairness for vulnerable people justifies the cost and effort of trying to count everyone in need; Q3 is an experiment on a sample (two classes) and raises fairness concerns — is it fair that one class gets a possibly better method and the other does not? Universities require ethics approval for exactly this reason; Q4 is observation, chosen specifically to avoid the stress and risk to sharks that capturing or tagging would cause; Q5 is an experiment on a sample of users, and raises a genuine current ethical debate about whether users should be told they are part of a trial.)
Activity 3 — Inquiry: Should the Census Become a Large Sample? (7 min)
Pairs, then whole-class discussion.
Some countries have replaced their full population census with a very large, carefully chosen sample (sometimes over a million people) combined with existing government records, to save money and time.
- What would Australia gain by doing this?
- What would Australia risk losing?
- Whose needs particularly depend on small, local, or minority-group data being accurate — and how might a sample affect them?
Socratic scaffolding:
| Prompt | Purpose |
|---|---|
| Understand the problem — what does the Census actually get used for? | Electoral boundaries, funding formulas for schools and hospitals, planning services for specific suburbs and small communities. |
| Devise a plan — how would a sample’s accuracy compare at a national level versus a small local level? | A large sample can be very accurate nationally, but small areas or small population groups may have too few sampled people to be reliable. |
| Carry out the plan — name one group most at risk of being under-represented in a sample-based count. | E.g. small remote communities, specific cultural or language groups, people experiencing homelessness. |
| Look back — is “cheaper and faster” always the right trade-off? | Not necessarily — it depends on what the data will be used for and who depends on its accuracy. |
Discussion targets: this is a genuine, ongoing policy debate — there is no single “correct” answer, but the reasoning (cost and speed versus accuracy for small or vulnerable groups) is exactly the reasoning this course wants students to practise.
Checks for Understanding
(4 minutes — exit ticket, collected)
- Give one advantage and one disadvantage of a census compared with a sample.
- Give one advantage and one disadvantage of an experiment compared with an observation.
- Name two ethical considerations relevant to collecting data from people.
- Reasoning. A company wants to test a new medicine. Explain why an experiment is necessary here rather than pure observation, and why ethics approval matters.
Answers: 1. Advantage: complete, exact coverage of the population; disadvantage: expensive, slow, and still imperfect in practice (accept other valid pairs); 2. Advantage: can establish cause and effect; disadvantage: harder to control and can raise fairness or safety concerns (accept other valid pairs); 3. Any two of: informed consent, privacy, fairness between groups, potential harm, honesty about how data will be used; 4. Only an experiment (deliberately giving the medicine to some and not others) can show whether the medicine causes improvement, since observation alone cannot rule out other explanations; ethics approval is needed because people’s health and safety are directly affected, and they must consent to the risks involved.
Common Misconceptions
| Misconception | How to pre-empt it |
|---|---|
| ”Bigger is always better” — a census is always the right choice. | Cost/time table; the Census-as-sample debate in Activity 3. |
| A large sample is automatically as good as a census everywhere. | The small-community discussion in Activity 3 — sample accuracy can vary by group and region. |
| Ethics only matters for medical experiments. | Activity 2’s Q5 (app design trials) shows ethics applies to everyday digital experiments too. |
| Observation has no ethical dimension because “nothing is done” to anyone. | Privacy and consent still apply — recording people without their knowledge is a real ethical question. |
| Confusing sampling error with a mistake. | Sampling error is an expected, describable feature of any sample, not a sign something went wrong (developed further in Lessons 106–110). |
| Believing cost and time are the only practical factors. | Explicit accuracy row in both comparison tables. |
Enrichment — Competition-Style Problems
E1 (Kangaroo style). A national census costs roughly
Answer
Census:
E2 (AMC Junior style). A researcher observes shoppers’ choices without asking permission, arguing “I am not touching anyone, so there is no ethical issue.” Give one reason this reasoning is incomplete.
Answer
Privacy is an ethical consideration independent of physical contact — recording identifiable people’s behaviour without consent can still raise concerns, especially if the data could identify individuals or be used in ways they did not agree to.
E3 (Challenge). Explain why a school cannot ethically run a true “experiment” comparing outcomes for students given extra tutoring versus a matched group deliberately denied it, even though the design would answer the research question cleanly.
Answer
Deliberately withholding a potential educational benefit from some students to satisfy a research design would likely be considered unfair and harmful, so ethics review would not approve it. Researchers instead use methods such as comparing students who happen to already differ (an observational study) or offering the tutoring to everyone but analysing existing variation in take-up — weaker at proving cause, but ethically sound.
E4 (Challenge). A wildlife census of an entire national park is attempted but only
Answer
Not strictly — since only part of the intended population (the whole park) was actually counted, the result is really a large, non-random sample (the accessible
Homework
- List two advantages and two disadvantages of using a census instead of a sample.
- List two advantages and two disadvantages of using an experiment instead of an observation.
- A school nurse wants to know the average resting heart rate of all students. Recommend census or sample, and justify using cost, time and accuracy.
- A researcher wants to know whether a new fertiliser increases vegetable yield. Explain why this should be an experiment, and describe one ethical or practical issue to consider (e.g. fairness between test plots, environmental impact).
- Explain the difference between sampling error and a mistake in data collection.
- A social media platform wants to test whether a new notification style increases app usage. Describe how this could be done as an experiment on a sample of users, and state one ethical concern.
- Reasoning. Explain why observation is generally considered ethically “safer” than experimentation, using an example.
- Challenge. A country has a population of
million. Running a full census costs approximately 18 200,000 $60$ per person. (a) Calculate the total cost of each approach. (b) The sample-based approach is cheaper, but name two situations in which the extra cost of a full census could still be justified.
Answers: Q1 — advantages: complete coverage, no sampling error; disadvantages: high cost, slow, still imperfect due to non-response or errors (accept other reasonable answers). Q2 — advantages: can establish cause and effect, results are more conclusive; disadvantages: harder to control, can raise ethical or fairness concerns (accept other reasonable answers). Q3 — sample is more practical: measuring every student’s heart rate is time-consuming and not necessary for a reliable average, since a well-chosen sample gives a good estimate far more quickly and cheaply. Q5 — sampling error is the expected, normal difference between a sample’s result and the true population value, arising simply because not everyone was measured; a mistake is an avoidable error such as mis-recording data or a broken instrument. Q7 — observation does not involve deliberately changing anyone’s circumstances or withholding a possible benefit, so it carries less risk of harming participants — e.g. watching how students naturally choose extracurricular activities raises fewer concerns than deliberately assigning some students to activities and others to none. Q8 — (a) census: