Sampling, Estimation & Confidence Intervals — Concept Check
Questions on Sampling, Estimation & Confidence Intervals. We almost never see all the data, so we estimate from samples — and confidence intervals tell us how much to trust those estimates, which is how you compare models honestly.
Questions in this quiz
8 questions on Sampling, Estimation & Confidence Intervals — Concept Check. Work through them here, then start the quiz to answer them and see your score.
1.A team is evaluating two machine learning models. Model A has an accuracy of 85% with a confidence interval of ±2%, and Model B has an accuracy of 84% with a confidence interval of ±3%. Which model should they consider more reliable?
- AModel A, because it has a higher accuracy.
- BModel B, because it has a higher potential maximum accuracy.
- CModel A, because it has a narrower confidence interval.
- DModel B, because it has a wider confidence interval.
2.A data scientist claims that increasing the sample size from 100 to 400 will reduce the standard error by half. Is this claim correct?
- ANo, because standard error does not depend on sample size.
- BNo, because increasing sample size has no effect on standard error.
- CYes, because quadrupling the sample size halves the standard error.
- DYes, because standard error decreases with the square root of the sample size.
3.A company reports a click-through rate of 5% with a confidence interval of ±1%. What does this interval imply?
- AThe click-through rate will always be 5% for future samples.
- BThe click-through rate is likely to be above 6%.
- CThe true click-through rate is exactly 5%.
- DThe click-through rate is between 4% and 6% with a certain level of confidence.
4.During an A/B test, the results show overlapping confidence intervals for both versions. What does this indicate about the test results?
- AThe test should be rerun with the same sample size.
- BThe difference in performance is likely due to noise.
- CThe overlapping intervals indicate a significant difference.
- DThe version with the higher mean is definitively better.
5.A researcher uses bootstrapping to estimate the accuracy of a model. What is the primary advantage of this method?
- AIt eliminates the need for a test dataset.
- BIt requires fewer samples than traditional methods.
- CIt makes fewer assumptions about the data distribution.
- DIt always provides a narrower confidence interval.
6.Why is it important to ensure that a sample is representative of the population when estimating a parameter?
- ATo guarantee a narrower confidence interval.
- BTo minimize the standard error.
- CTo reduce the sample size needed.
- DTo ensure that the estimate is unbiased.
7.A model's accuracy is reported as 90% with a confidence interval of ±5%. What does the confidence interval suggest about the model's accuracy?
- AThe model's accuracy is likely between 85% and 95%.
- BThe model's accuracy is exactly 90%.
- CThe model's accuracy will always be above 90%.
- DThe model's accuracy is likely below 85%.
8.If a sample size is increased from 50 to 200, how does this affect the standard error of the mean?
- AThe standard error is reduced to a quarter.
- BThe standard error doubles.
- CThe standard error is halved.
- DThe standard error remains the same.