Forum: Author Hangout

That assumes the two polls are of identical size.

What assumes? The 4-point average or something else?

(With, while would normally assumed to be Democratic for the personalities of the staff there, was regarded to be at least a little pro-Trump by the assumptions inherent in their fundamentals. That component though is quickly losing weight at this late phase.)

Not quite sure who you're referring to here. The 538 Staff? The pollster's staff? Somebody else?

Not quite sure who you're referring to here. The 538 Staff? The pollster's staff? Somebody else?

There are reasons for assuming 538 could be "left" leaning. But it could just be the "liberal bias of reality."

However, the organization they are part of do supposedly have "left" bias, and quite few Trump supporters assume that as default. At the same time, 538 got quite a bit of beating from real lefties for even giving those almost 1 in 3 chances to Trump in 2016 they did (before he indeed won, of course), and they are softer criticized this year for giving Trump implied incumbency advantage and expecting tightening of the race in the end (what doesn't seem to be happening, despite RCP possibly artificial attempts to make it look like that it indeed is).

I didn't emphasize the size of the polls as most are usually less or around thousand, so you can assume it was buried inside the "methodology" requirement. All else being exactly equal, you could probably indeed combine the polls like that and the size would matter. However how little I know about statistical theory, the sample size isn't all that critical as long as it is representative. So, with two pollsters claiming adequate representation (and/or weighing) those two polls could indeed be averaged together as equal although I too would tend to think that the larger one might be more accurate and thus receive more weight, but probably not necessarily ten times the weight of the smaller poll.

However how little I know about statistical theory, the sample size isn't all that critical as long as it is representative.

However, the sample is only really representitive of the smallest group it fits. In most cases, this translates to "this is what those people with listed phone numbers who were willing to spend time answering a survey, and live in {specific area] think"

To have a accurate poll for politics, you first need to know what the voter turnout is gonna be. Then you need to poll about 30 individuals of each combination of gender,location,age,net worth,party affiliation,ethnicity,educational background. Then you have enough data to assume normal distributions, so you multiply each group's split by the percentage of the people who'll turn up to vote that each group represents, add those all up, and you should almost certainly know how the popular vote will turn out. Once you figure out how the dead people from Chicago plan to vote, that is.

There are over 300 million people in the US. I fail to see how a sample size of 1,000 could be representative of a nationwide poll.

Some thoughts on a back of the envelope calculation for an adequate sample.

At least one person from each state, that's 50.

Urban people vote different from rural people. Make that 2 from each state. 100

People in DC get to vote for president but DC is all urban 101.

Male/female 202.

Age: here it starts to get harder, how many ways to cut age brackets. We are talking about election polling so cut off the bottom at 18. I'll say 18-30, 31-50, 5-65 and 66+. That's 4 age brackets: 808.

Income. I'll make it simple Pick a percentage for +/- median income and under over and the middle for three income brackets. 2424

Military/Civilian government employee/private sector/retired(or on gov assist) 9,696

Race: How many ways to cut this? I'll pick White/Black/Hispanic/Nat Amer/Asian/Mixed. 58,176

Religion: Protestant|Catholic|Jewish|Islamic|Atheist|Other
349,056

I think people can see where this is going.

It gets worse.

You're only grabbing a single (possible) respondant for each demographic, so you have no idea if that individual is representative of that demographic group. Multiply by 30 and you can assume normal distribution, which means the vote split is actually representative. Although you still have the issue that you've only got the public opinion of those people willing to answer polling questions. So no indication of what the opinion of those who wouldn't answer, plus no way to determine the outcome of an election since you don't know what voter turnout from each demographic is likely to be.

It gets worse.

I know. I stopped having established a basic pattern.

In a non-homogeneous population, 1% of the target population might get you a truly representative sample, if you sample selection methods aren't biased.

Tiny samples work for QC testing for a mass production process because the units produced are supposed to be homogeneous.

They don't work for opinion polling of humans.

Yeah, but my point was that even having a representative sample is useless for predicting election results because a)the population you're measuring is "people who'll answer polling questions" not "the entire population of the country" and b)you need to know voter turnout data a priori to even determine who'll win the popular vote. But you then need to look at the state by state data to determine how the EC breakdown will go.

you can assume normal distribution,

"No matter how many observations you have (what the n is) you can't guarantee the sampling distribution will be normal (Joliffe, 1995). You would need to know the rate of recording these observations to know what the distribution is. This is true for both distributions and summed Poisson distributions. Even with a very large n, it might not be enough to bring the sum total lambda (from the combined Poisson distributions) high enough to approximate a normal distribution.

The CLT can't save everything
The graph below shows a simulated Poisson distribution with 1,000,000 random values, λ=1. Even with a n of 1,000,000, the data is heavily right-skewed. If the rate (λ) is low enough, no sample size can guarantee a normal distribution. This holds true for summed Poisson distributions.

The same is true for a binomial distribution. When P=.01 (the event happens 1 in every 100 trials), the distribution is heavily right skewed. So, the distribution of the statistic of a relatively unlikely event with two potential answers doesn't approximate a normal distribution. This is also true when n=100. Therefore, it shouldn't be analysed using statistics that assume a normal distribution. This clearly shows the CLT doesn't apply without reservations."

Statistics don't lie, but actuaries can give you any answer you want, by using statistics.

Statistics don't lie, but actuaries can give you any answer you want, by using statistics.

One of the universities I went to, the psych department offered their own statistics class. The math department accepted that class as at least equivalent to their own offering, but the psych department didn't accept the math department's version of the class as equivalent. So it seemed interesting (particularly since it was worth more credits). One of the things that stuck out was the claim that - when sampling a human population (and remembering that your sample is representative only of the smallest possible identifiable population) once your sample size is at least thirty you can assume that whatever binary response will be roughly normally distributed WRT the actual population (ie: ask 30 people "are you a Ashkenazi Jew?" and the odds of the proportion of the actual population you've sampled answering the same way is normally distributed around your response rates).

Nobody average together state and national polls, to my knowledge. They do it by state, and national polls in they own category. Although I think 538 had tried to infer national margins by combining state polling and then comparing that to national polls as an analysis excercise, once or twice.

I believe the accuracy of polls have been declining over the last decade or so

The accuracy of polls has never been what the pollsters think and/or claim it is. Dewey Beats Truman anyone?

While a lot of that is true, and truly representative sample is practically impossible now, polling remains rather accurate if isn't even improving actually.

Even the infamously "wrong" polling of 2016 were, by polling standards quite accurate: it had only about 2% systematic error across many polls (believed to be caused mostly by unexpected turnout and voting consistency of non-college white men, and thus those were weighted "incorrectly" in hindsight) at declared margin of error of arousal 4% (with is the usual they give). And even historically, they weren't at all bad. (That analysis is done and redone many times in many places.) They did call the winner "wrong" but as far mathematical accuracy goes, that is immaterial. Within margin of error of parity there just is that uncertainty.

Even the infamously "wrong" polling of 2016 were, by polling standards quite accurate:

No, what you describe is precision, not accuracy. They aren't the same thing.

Dominions Son
10/30/2020, 6:34:19 PM

@LupusDei

Even the infamously "wrong" polling of 2016 were, by polling standards quite accurate:

No, what you describe is precision, not accuracy. They aren't the same thing.

For those of us who don't understand the significance of the distinction, could you please explain.

For those of us who don't understand the significance of the distinction, could you please explain.

I'll give you an example using marksmanship.

Fire 10 rounds at a target.

Precision = The size of the grouping, how close all the bullet holes are to each other.

Accuracy = the distance between the center of the grouping and the center of the bulls-eye.

Polls like to claim a 4% margin of error, but that's not the whole story.

1. They never tell you how they arrived at that margin of error, and yes it matters.

2. There should be a confidence interval supplied with that.

95% is the typical confidence interval used in most statistical work.

What that means is that, assuming they did everything correctly (and that's a pretty big assumption), there is a 95 percent probability that the real answer is within +/- 4% of the answer they got.

What happens in the other 5% of cases? The real answer is out side of that +/- 4% and it could be anywhere outside that +/- 4%.

Maybe you didn't notice, but your reply cut off in mid-sentence.

Perhaps the prison let him out early for good behaviour.

My sentence got overturned. :)

Maybe you didn't notice, but your reply cut off in mid-sentence.

I did notice. And I couldn't remember where I was going with that last sentence so I removed it.

for what ever reason a number of Trump supporters don't admit to supporting him.

There are areas where open Trump supporters have gotten death threats.

There are areas where open Trump supporters have gotten death threats.

There is a ad of a GOP candidate's rally where one of his supporters holding up a sign right near him that says "Vote red or wind up dead."

There is a ad of a GOP candidate's rally where one of his supporters holding up a sign right near him that says "Vote red or wind up dead."

There is a big difference both qualitatively and legally with a generalized treat not aimed at a specific person and specific and particular threats targeting a specific person and their family sent to their home.