Confidence Intervals / Tests of Significance & Power

Confidence Intervals / Tests of Significance & Power

• Confidence intervals

• Purpose – to estimate an unknown parameter with an indication of how accurate the estimate is and of how confident we are that the result is correct
• 68-95-99.7 rule says that the probability is about 0.95 that x bar will be within two standard deviations of the population mean score
• Estimate +/- margin of error – form for confidence interval
• Confidence interval for a parameter is an interval computed from sample data by a method that has probability C of producing an interval containing the true value of the parameter
• Any normal distribution has probability about 0.95 within +/- 2 standard deviations of its mean
• To construct a level C confidence interval:

• Find the number z* such that any normal distribution has probability C within +/- z* standard deviations of its mean
• If two tail, you find the z* using table A on the left side and on the right side, the area in the middle = C = our confidence interval
• Probability C that x bar lies between (u – z*(std dev/sqr rt of n)) and (u + z*(std dev/sqr rt of n)):

=

• M = (u +/- z*(std dev/sqr rt of n))

• Choosing Sample Size

• To obtain a desired margin of error m, plug in the value of std. dev. And the value of z* for your desired confidence level and solve for the sample size n

• n =

• Tests of Significance

• Purpose: assess the evidence provided by the data in favor of some claim about the population parameters; compare observed data with a hypothesis whose truth we want to assess
• Example – need to conclude whether or not the true means of the two random samples are different
• Key steps:

• Started with a question about the difference between two sample means and are trying to determine whether or not the data are compatible with no difference
• Compare the means of the data and fine the difference between them
• Turn the results into probabilities

• If the probability is very small – we have observed something that is very unusual or the assumption (that there is no difference in the means) isn’t true
• Null hypothesis – the statement being tested in a test of significance
• Alternative hypothesis – the statement we hope/suspect to the true; is true when we reject the null

• Test Statistic

• Measures compatibility between the null hypothesis and the data
• Purpose: use it for the probability calculation that we need for our test of significance
• Z = (estimate – hypothesized value)/(standard deviation of estimate)

• P-Values

• Probability, assuming null hypothesis is ture, that the test statistic would take a value as extreme or more extreme than that actually observed
• Smaller the P-value, the stronger the evidence that the null hypothesis is not true
• To solve:

• Find Z = (estimate – hypothesized value)/(standard deviation of estimate)
• Check Table A for equivalent z score and find the percent/probability
• Need to times the probability by two since it is on both sides of the curve and then you get the percentage of observing a difference as outside of the main area of the curve

• Significance Level

• If the p-value is as small or smaller than alpha, we say that the data are statistically significant at level alpha

• Steps for Tests of Significance

• State null hypothesis and determine alternative hypothesis
• Calculate the value of the test statistic on which the test will be based – to measure how far the data are from the null
• Find the p value for the observed data – this is the probability that the test statistic will weigh against the null
• State a conclusion – conclude whether you can reject null or not

• Tests for a Population Mean Summary

• zTest for Pop Mean = z = (Xbar – mu0)/(std. dev./sq. rt. n)

• Two-Sided Significance Tests / Confidence Intervals?

• Two-sided tests rejects null hypothesis exactly when the value mu0 falls outside a level 1-mu confidence interval for the mu (mean)

• P-values versus fixed alpha

• P-value is the smallest level of alpha at which the data are significant
• P-value gives us a better sense of how strong the evidence is

• Power & Inference as a Decision

• Power = probability that a fixed alpha significance test will reject the null when a particular alternative value of the parameter is true
• Use when:

• “is N subjects a large enough sample for this project?”

• Calculate power in 3 steps:

• State null and alternative hypotheses, as well as the significance level, alpha
• Find values of Xbar that will lead us to reject the null

• Use z test → z = (Xbar – mu0)/(std. dev./sq. rt. n)
• Plug in the z score for z from Table A for whatever probability of test significance you choose
• Solve for Xbar → as the unknown variable

• Calculate probability of observing these values of Xbar when the alternative is true

• Use same formula as in part 2, but solve for P instead of Xbar

• High power is desirable
• Ways to increase power:

• Increase alpha
• Increase sample size
• Decrease standard deviation
• Consider a new alternative hypothesis further away from mu0

• Types of Error

• Type I – reject null and accept alternative when null is true
• Type II – accept null when alternative is true

Inference for Mean of a Population / Comparing Two Means

• T distribution

• Use when standard deviation is not known
• When standard deviation is not known,         we estimate it with the sample standard deviation (s/sq. rt. n), which is equal to the standard error of the statistic
• One-sample T statistic:

• T =(Xbar – mu)/( s/sq rt n)

• Degrees of freedom?

• One-Sample T Confidence Interval

• C = Xbar +/- t* (s/sq rt n)
• Margin of error = t*(s/sq rt n)

• One-Sample T Test

• Same as with z test
• t = (Xbar – mu0)/(s/sq. rt. n)

• Calculate power of T test

• We assume a fixed level of significance, often alpha = 0.05
• Step 1 – decide on a standard deviation, significance level, whether the test is one-sided or two-sided and an alternative value of mu to detect
• Write the event that the test rejects the null in terms of Xbar
• Find probability of this even when population mean has this alternative value

• Comparing Two Means

• Use when you are comparing responses from two groups and the responses in each group are independent of those in the other group
• Two Sample Z Statistic

• Z =

• Two Sample T Significance Test

• T =

• Two Sample T Confidence Interval

• Questions:

• When do you use t vs. s? Is it just when you don’t know the standard deviation?
• How do you choose S in the power equation?

Comparing Two Proportions & Chi-Squared Tests

• Two-way tables: organize data by two factors (e.g., group by age, then by education)
• Marginal distributions – summarize each variable independently
• Two-way table describes the relationship between variables
• Chi-square test:

• Purpose: to determine if the differences in sample proportions are likely to have occurred by just chance because of the random sampling
• Null hypothesis in this test = no relationship between the variables

• Compare actual counts with expected counts to test this
• Expected count = (row total x column total) / n

• Formula:

[pic 1]

• Large values for x2 → evidence against H0

• When to use?

• All individual expected counts are 1 or more (≥1)
• No more than 20% of expected counts are less than 5 (< 5)