I wrote several blog posts yesterday that aren't going to get posted. I wrote one about people with bad manners and a lack of common decency. I wrote another about my growing frustration with the divide between Right and Left and how I was starting to feel sympathetic towards the Left until they started calling me names. I also wrote one about how I do indeed understand basic statistics but that I hadn't planned on 2000 people descending on me like vultures so I perhaps didn't word my post in a way that everyone could understand. And I wrote about how chilling I think it is that every one of the commenters wrongly assumed that I bring my politics into the classroom and force it on my students.
In the end, none of these topics matter. Those who came over from Atrios will have forgotten about me by now, save the occasional one who will pat himself on the back for calling me "intellectually bankrupt" and "a gathering threat to democracy". All that really matters are my regular readers, the faithful who understand what I was trying to say even if I didn't phrase it as well as I could have.
Gemini was one of the only commenters that I appreciated hearing from. I would like to address what he/she had to say.
That's an interesting point, because I no longer trust the organizations who do the polling. None of them. If I have learned anything in the past two years, it's that all sources are biased, even the ones with the best intentions. I don't put much faith at all in the objectivity of any person or organization. In contrast, it seems that lots of people do blindly assume that these polls are conducted by robots who have no political agenda. That's why in my class we discuss how every single media source has bias of some sort, from Fox to the BBC. Every single one. We discuss how it's impossible to avoid but as long as we're aware of it we're ahead of the game. (I don't tell my students which sources I think are more biased than others; that's what bad teachers do.)
I stated already that I don't think that all the questions were worded objectively. Some of them were decent straight-forward questions, but some were not. I mentioned Q48, but I also think that we might have seen different results for several questions if the words George Bush had been substituted with United States (as in Q16: Do you approve or disapprove of the way George W. Bush is handling the situation in Iraq?). The mere mention of our President's name can send some people into a frenzy, regardless of what the question is asking. These questions with President Bush's name in them were split pretty hard down the party lines; I imagine that the answers might have been slightly different if his name had been left out. It's subtle things like that which will affect the outcome of the already-too-close-to-call results.
The sample was not purely random, since the LA Times states that "the entire sample of adults was weighted slightly to conform with census figures for sex, race, age and education." I'm no expert, and I've wasted enough time on this topic already, but I'd guess that voting patterns do not correlate perfectly with census data. My guess is that things like age and education are indicators of who votes, so making the poll fit census data instead of voter demographics is less realistic. But whatever, I can let that one slide.
What I cannot let slide is that, while the LA Times felt it necessary to weight the sample based on demographics, they did not feel it was important to weight it based on party affiliation. According to Mickey Kaus, "the party breakdown in the LAT poll was 38% Democratic, 25% Republican, 24% Independent." (Thanks, Tanker.) Don't you think that might have an effect on the poll results as well? The LA Times spent time and energy tweaking for race and sex; why couldn't they poll an equal number of people based on party affiliation? That seems to me to be a much bigger indicator of political opinion than race or sex, so the LA Times should have tried to minimize that effect.
That's just my point; for all the hatred I incited, few people actually discussed the poll itself. I'm not sure any of them understood what it represents. No one at all commented about the missing 55% in the original AP report. That's the type of bias I was pointing out in the first place. Why comment on what <20 + 25% of people think we should do, without mentioning the rest, unless you've got a point to make? Why not report that 73% said that there should be no specific date for withdrawal, unless that's not considered newsworthy? I don't think any of these results actually mean anything (the only thing that matters is the actual vote in November), but if they must report on it, they could do a better job.
What I was trying to originally point out is that many people are headline readers; they see the headline Poll: Voters Say Iraq Didn't Merit War, and they don't actually read the article or think about how the questions were worded or how the poll was conducted. They place great faith in the polling process because it's "up to statistical standards", without thinking much about the fact that opinions and feelings are not easy to scientifically measure. A sample size of 1230 for coin flips or jellybean colors or dice rolls is absolutely acceptable, but public opinion is a much trickier thing to measure.
All I tried to say is that polls don't really matter, especially close ones. If a poll of 1230 people showed that 80% of people thought Iraq didn't merit war, then perhaps there would be something substantial to worry about. But polls about presidential popularity and opinions on war where the percentages hover around 50% are worth a grain of salt, in my opinion. A slight change in wording could tip opinion the other way. I'm amazed that Atrios' readers spent their time calling me names rather than entertaining the possibility that I could have an honest point here.
Incidentally, one of the things I teach my students when doing research is an attention to detail. If you're writing a paper on marriage and you use Britney Spears as an example, then you'd better spell her name right. I repeatedly tell them that when they ignore the easily-checked details, it really weakens their argument; how can we trust that your argument is sound when we can't even trust you to check spelling and details? I found the same thing going on in the comments; why should I take advice on paying attention to details from people who didn't bother to notice that 1) I am a female 2) I don't teach at a DoD's school or 3) that I'm not a Christian evangelical?
Come to think of it, why should I take advice from people who call me "clueless fucktard dumb" or say that I write "incredible dumbfuckery", that I "have no business teaching anyone anything", that I am "a gathering threat to democracy", that I should "shut the fuck up", and that I probably "don't even have a high school diploma, let alone a college degree"?
Darkwater wrote me a huge email and included a lengthy comment here. I have read both, and I'd like to point out something that I think is apples and oranges.
Darkwater does military testing for the Department of Defense. For him, I assume close calls are controlled experiments measuring, as he said, whether 'system A' is better than 'system B'." That's science. Opinion polling is not science. There are way too many variables that can't be controlled for. A large part of our population relies on caller ID and answering machines to screen their calls; those people will not be participating in the polls. Neither will people whose primary phone number is a cell phone, where they'd be paying for the call. But let's say the poller does get through to someone who agrees to do the poll, and three questions into the call that person realizes he's not informed enough to accurately answer the questions. He can either 1) stop the poller and excuse himself, 2) answer "no opinion" on everything, or 3) guess at what he thinks the "right" answer is. Based on what's been going on in the media, do you think someone will guess that President Bush is doing a good or bad job with Abu Ghraib? People who don't pay that close of attention might not know that the investigation into Abu Ghraib was nearly complete and court martials were already beginning before CBS ever got ahold of the story. But the way the story splashed across the front pages, someone who is not as informed might assume that it's being handled poorly and that "bad job" is the answer the poller is looking for (the poller works for the media, the media says it's a bad job...). That's just one example. There are many people out there who don't read blogs and don't stay up-to-date on politics, yet they might still give the poll a stab. And they might try to guess what the "educated" and "right" answer to each question would be. That's not scientific. Opinions are not hard facts. Using the scientific method to check System A against System B is science (the implementation of either system might then have to mingle with politics, as Darkwater implies, but the actual research is scientific.) Calling a bunch of people and asking them questions is deeply flawed. Though the statistical analysis of the data might be sound, any experiment where someone can try to guess the "right" answer is not hard science.
That's all; I'm done discussing this poll.
1
As always, you said it perfectly. If I were ever able to go to college, I'd feel extremely honored to take one of your classes.
Posted by: Shannon at June 14, 2004 04:42 AM (bqG6g)
2
I've read over the 'controversial' posts in question and find nothing objectionable in what you say, either in terms of methodology or plain candor.
My experience with weblogs, and the state of public rhetoric in general, is in line with what you print: comments like clueless fucktard dumb are cute and clever and signify only someone's passion, not their intellectual acuity.
Graduate school taught me that methodology is at best a set of rules and cautions that may guide the social scientist towards 'truth' but can't guarantee one's arrival at the goal. But in the webworld, truth isn't so relevant as agreement or disagreement, and we are too often stuck reading what is usually entertaining and only occasionally enlightening or informative.
Your columns--at least the ones I've read so far--seem to be a serious attempt to discuss some aspects of public discourse and communication that get scant attention in the mainstream press as well as the web. But I've found that anybody caught actually trying to practice something akin to 'objectivity' will be pilloried and skewered by the web's resident wits without mercy. Your offense seems to have been to question the methodology of a poll conducted on behalf of John Kerry, the anti-Bush.
(Disclosure: I dislike Bush intensely, but will vote for Kerry only because it seems the best way to end the stench of corruption and arrogance of power emanating from our capital.)
My favorite rhetorical resources include Ockham's Razor and the Falsifiability Principle of Karl Popper (or whatever it's called these days). I have found that usually Popper and always Ockham are reliable companions when navigating the thicket of excited rhetoric found on the web.
I like reading Atrios, and Billmon, and Talking Points, and Daily Howler, and the websites, materials, and resources they link to. I found your website via link from Atrios, and found your posts interesting and a welcome change from the usual team-oriented name-calling, although I wouldn't call any of the above bloggers careless or irresponsible writers.
I'm am puzzled by the link from Atrios: I can't figure out if you are being slagged or not. Maybe that makes me a 'clueless fucktard' but I'm willing to live with that.
Keep your head high and say what you believe. That's all anybody should be doing out here in the wild and wooly web, the only place where democracy is practiced with any passion and vigor in the poor old US of A.
Posted by: Jon Koppenhoefer at June 14, 2004 05:01 AM (A094q)
3
Lady, you're dumb as a hammer, for gods sake shut up or put this stuff in a private diary!
Ever So Sincerely,
Mike
Posted by: Mike Adams at June 14, 2004 06:16 AM (ZgqPR)
4
It is interesting to me that about twenty people reading comments on that post made it over to see what Bunker had to say. None left a comment there.
Posted by: Mike at June 14, 2004 07:53 AM (cFRpq)
5
I understood what you were saying in the first few lines of your original post. It was obvious that you were taking apart the newspaper article, and the way the poll was reported, not the poll itself.
As I read through the comments, I just had to smile at the rank hypocrisy evident in so many of them, starting with the wrong assumptions about you, and ending up with the claims that you never read or consider anything outside of your preconceived opinions.
Posted by: NightHawk at June 14, 2004 09:17 AM (5GWma)
6
You're correct, those of us that read your postings regularly got it. Promise. I know I say this too often, but I don't get people that only read or listen to find something negative to say.
We think you're wonderful!!!
Posted by: Tammi at June 14, 2004 10:14 AM (GqPls)
7
So, Sarah....
You were swinging towards the Left politically, but are now swinging towards the Right because some people whom you regard as being left-wing called you names? Forgive me, but that's got to be the most idiotic reason to change one's political colors I've ever encountered.
To call you 'dumb as a post' would be an insult to the many fine timber posts I've seen over the years.
Posted by: Pastormaker at June 14, 2004 10:33 AM (SC/gu)
8
Perhaps, if you consider the fact that what you wrote yesterday was:
"The U.S. population is estimated at close to 300 million right now, and we're supposed to get worked up over what 1,230 people who are registered voters have to say? Hell, I only just registered yesterday, so I would've been ineligible. And if the margin of error is plus or minus 3%, and 53% of these 1,230 people thought war was not necessary, then perhaps only 615 people in the whole USA said this.
615 people. How on earth is this supposed to be representative of the voice of America?"
and it makes no sense at all, other than to assert an equation between rightwingnuttery and ignorance? This might explain why folks outside your immediate jolly little ex-pat circle were irritated by the presentation.
Incidentally, it -still- makes no sense. Saying that you really truly do understand statistics doesn't help. Good luck in your studies.
Posted by: GWPDA at June 14, 2004 10:55 AM (kpVNh)
9
You talk too much. Try to shorten it, and have a point.
Posted by: dan at June 14, 2004 11:39 AM (ZSyWW)
10
"I wrote another about my growing frustration with the divide between Right and Left and how I was starting to feel sympathetic towards the Left until they started calling me names."
Deep stuff! Just thank your lucky stars that Ben. Franklin, Thos. Jefferson, &c. were'nt quite so F-ing thin-skinned.
Posted by: Buck at June 14, 2004 11:42 AM (0K5pN)
11
NightHawk,
It's difficult not to make unkind assumptions about someone's intellect when presented with something like this:
"The U.S. population is estimated at close to 300 million right now, and we're supposed to get worked up over what 1,230 people who are registered voters have to say? Hell, I only just registered yesterday, so I would've been ineligible. And if the margin of error is plus or minus 3%, and 53% of these 1,230 people thought war was not necessary, then perhaps only 615 people in the whole USA said this.
615 people. How on earth is this supposed to be representative of the voice of America?"
This is putting aside logical thinking to accommodate ideological blinkers to an extreme. Sarah was in blatant denial about the fact that people are capable, in large numbers, to see the world differently from the way she perceives it. In her updates and subsequent post, she is once more in denial, this time of the fact that she could have said something so completely undeserving of an intelligent person.
The points she has retreated to - that the media have their biases and that polls can be manipulated by leading questions - are obvious and unremarkable. To a blogger, finding examples of them is usually easy fodder.
What Sarah said in the post that sparked this, though, went way beyond that. Some of those that have commented on this may have been too rude in their responses, but they were fundamentally correct: unless meant as obvious satire, this is simply not worthy of an educator, and it deserves to be ridiculed.
Sarah presented these offerings with not a hint of irony or sarcasm, but at least she has seen fit to acknowledge her misjudgement.
Posted by: MaitreDee at June 14, 2004 12:07 PM (AKgpm)
12
These comments have made me so sad
. Most have come here only to toss out insults, not have an interesting discussion where we could possibly come away with anything other than new curse words.
I understood Sarah's statement concerning the left. She did NOT say she was swinging toward the left. I find it to be extremely telling that those throwing the worst insults cannot even comprehend what they read. She stated she was "starting to feel sympathetic towards the Left." Reading comprehension - it must be a thing of the past.
And now, something I've noticed over and over when the left is involved has happened here: Have no real discussion (just as we saw during the democrats' primary), just throw out rhetoric and insults. The anger and hate from the left for any and all who do not tow the party line is quite frightening. Middle America sees this and they are slowly migrating ever so slightly to the right of center. It's example after example of those on the left being so hateful that most Americans think Michael Moore is the idea of what the left is, not Joe Lieberman. When that special day in November arrives, I think America will unite and say we do not want this type of mentality running our country.
And as an Army wife myself, the distasteful comments concerning her registering to vote just shows that you have never served your country or even had a family member who has. Military spouses are not like service members when they move. We do not have the ability to keep our home of residence. We must establish residency each and every move - which includes having to register to vote AGAIN. My husband has served this country for 21 years and I can promise you that when I have registered to vote at EACH NEW place we've called home, it was not the first time and was not indicative of my never having voted. Just another example of the hateful and downright mean things I've read here. Jon Koppenhoefer, thank you for your thoughts. I enjoyed reading something from someone who could share his thoughts without having the need to belittle someone to make themselves feel superior. It was a most welcomed change.
Posted by: Shannon at June 14, 2004 12:12 PM (O2yvW)
13
Your new post is much better than the previous post. I still disagree with some things, such as the suggestion that "George Bush" could be substituted with "United States" without changing the inherent meaning of the question. But overall the post comes off much better.
You are correct that bias is something to consider in any presentation of fact or opinion. One thing I would add is that the reader should also consider his/her own biases in formulating responses to these presentations. I admit that my biases significantly colored my view of your previous post, though my biases also made me ashamed of many of the comments made against you. I still think your view of statistics is somewhat naive, but I acknowledge that that view could be colored by my biases.
Posted by: Scott Spiegelberg at June 14, 2004 12:34 PM (Z4IrA)
14
I'm sure you already realize this, but most of the people spewing this bullshit are intellectual flunkies that picked up on a single atom of your original post and extrapolated it to encompass the entire message. While your original essay was good, that part probably could have been written better, or at least less idiot-incendiary.
Yes, we all understand that 1300 people is a statistically relevant sampling rate, but these 600-some people are now suddenly the VOICE OF AMERICA. We all read the paper and suddenly you say "Oh! The rest of my country believes we are caught in a quagmire which cannot be won". See that often enough and it becomes true.
Remember kids, just because you think you have a firm grasp of statistics, doesn't make you an Internet supergenius-- it just means you're a average person who understands basic statistics.
Posted by: Matt at June 14, 2004 01:42 PM (x42+z)
15
You state that you don't bring your politics into the classroom. But you and your students "study media bias". Now, you are admitidly biased to the right of center yourself. You said everyone that conducts polls has a bias, because all of us have biases. Is is not safe to say, then, that everyone that "teaches media bias" brings their own biased politics into it.
I think if you have such biased leanings, you should lay off the media bias studies in your classroom. I don't see how you can avoid bringing your own politics into the classroom.
Posted by: Far North at June 14, 2004 01:49 PM (CTbF3)
16
Matt:
"Yes, we all understand that 1300 people is a statistically relevant sampling rate, but these 600-some people are now suddenly the VOICE OF AMERICA."
Since it is a statistically relevant sample, those 600 people do happen to represent roughly half the American public. They also represent a majority, which is what the article called it.
"We all read the paper and suddenly you say "Oh! The rest of my country believes we are caught in a quagmire which cannot be won"."
Not the
rest of the country (which would be about 99.9999%), but
more than half.
Why do some people bother even defending this ridiculous argument, just because she is on the same side as they are politically? She basically pretended that statistics is such a grossly misguided field that it has a margin of error of 99.9% - that a poll indicating that 53% of the people believe one thing with a margin of error of 3% can mean as little as only those people actually polled holding that opinion. Does anyone here, including Sarah, seriously want to defend that assertion?
Those that you call the idiots who were incensed by this... do you think they're wrong in pointing out that what Sarah said was, well, less than an intelligent thing to say?
Of course such a misguided quote is going to obscure what you perceive as her larger point - not that that was a particularly original point, because it is so obvious.
Posted by: MaitreDee at June 14, 2004 02:06 PM (AKgpm)
17
I would fight to the death defending your right to express your opinion. However, sadly, you're one of the people that make me question that stance rather than proud of it.
May I suggest some
critical thinking resources? You might find that critical examination of ideas tends to yield reliable facts, as opposed to unsupportable assertions or outright falsehoods (preferable though these may be).
Good luck,
R.
Posted by: Rob at June 14, 2004 02:31 PM (2Hrux)
18
i think i'm pretty, but my parents are so mean. what should i do?
thanks,
pjl
Posted by: philip at June 14, 2004 03:26 PM (h+561)
19
I'd like to see it acknowledged (so I guess I'll do so) that any time an essay with a political bias has the misfortune of being exposed on a high-profile venue on the internet, a significant degree of empty and hateful response will result. The direction of the original bias is insignificant. I have seen every puerile statement you cite said verbatim by people representing themselves as conservatives.
My own opinion is that a relatively small number of individuals and organizations effectively wield a vastly inordinate degree of power in this nation by manioulating the political system and that one of the key aspects of this manipulation is the dominance of the two-party system. When we succumb to the artificial division of our opinions into two camps and have at each other like unruly children on a particularly nasty playground, we perpetuate this system we laughably call democracy.
Posted by: nanojath at June 14, 2004 04:49 PM (9Y3U3)
20
I would think it heartening that a student didn't know how to spell Britney Spears, as long as they still knew how to spell more worthy names relating to marriage like, oh I don't know, Massachusettes residences Stan Chagnon and Joe McCoy, or Tanya McCloskey and Marcia Kadish....
Posted by: J at June 14, 2004 04:57 PM (5gN2v)
21
I'm one of those Atrios readers that followed the link posted on his site. I'd like to add my voice to those few who have apologized for the insults directed your way. I'm proud to be a liberal, but disgusted with the way my peers have treated you.
More and more, I'm beginning to worry about the future of our country. The partisansip that I continue to see from both sides of the aisle seems to only be intensifying. If something doesn't change, I'm worried that we'll come to blows (literally) within our own country.
In my own family, we've stopped talking about politics because it has become so personal for all of us. I miss the discussions we had because I like respectful debate. I like being around people who see things differently than I do.
I don't know how to change it but I just wanted to let you know that, even if we're not of the same political bent, you come across as a thoughtful and good person.
Posted by: Chad at June 14, 2004 05:57 PM (1OeHR)
22
Well, so here's a question.
If you're convinced that all polls are biased, and that we shouldn't trust any single poll, then why not chalk this up to a simple outlier of a poll?
Besides, if every poll has some bias inherent to it (which is a possibility), wouldn't it make much more sense to consider a single "ur-poll," which conflates the results of all of the polls? If you had the same question on enough polls, you could probably whittle away at both the accuracy and precision of the polls. [No, they're not the same thing.]
Finally, if you insist that polls be corrected for party affiliation, how do you do so? I don't think you can use any specific criterion for establishing a true party affiliation, and I think that it's such a fluid label that it makes no sense to even attempt to correct a poll based on that characteristic.
For example, in MA, in 2000, I registered as a Republican--only so I could vote for McCain in the primary. Does that make me a Republican? Hardly. It was a purely political move, that has no reflection on my actual voting preferences. But, according to your suggestions, affiliating as a Republican would effectively bias the poll.
So, don't sweat a single poll. Sweat the fact that all the polls are starting to turn up results you don't seem to like too much.
Posted by: Can't win at June 14, 2004 06:07 PM (aQOKC)
23
One more point. Why should there be equal numbers of Democrats and Republicans? Are we, in fact, split 50-50? In some states, maybe, but in other states, the split is 60-40 one way or the other. And, nationwide, I don't think there's an equal split--in general, Democrats outnumber Republicans (but Republicans tend to outnumber Democrats at the voting booth). So, it doesn't seem to make sense to "correct" results based on an assumption that may not even be valid.
Posted by: Can't win at June 14, 2004 06:09 PM (aQOKC)
24
J:
"I would think it heartening that a student didn't know how to spell Britney Spears, as long as they still knew how to spell more worthy names [...]"
The fact that you then go on to misspell both 'Massachusetts' and 'residents' is pretty darn funny in and of itself...
Posted by: MaitreDee at June 14, 2004 06:17 PM (AKgpm)
25
I took an intro stats course in college, so that’s the extent of my formal stats education. But it seems to me that using a small random sample and extrapolating to a much larger population makes sense when you’re measuring objective facts—how many children are in the average American family, for example, or the average level of education of retirees, etc. But, opinion polling must be inherently inaccurate, because pretty much each adult has his own individual thoughts, which—because he has a separate brain--must differ at least slightly from the next person’s. Of course, two people could give exactly the same answers on the same poll, but that doesn’t mean that they have exactly the same opinions—especially if the poll contains leading questions, confusing answer possibilities, or limits a respondent’s ability to answer "don’t know" or "no opinion."
The real problem with what happened in yesterday’s comments was not at all that people disagreed with Sarah, but that there was an obvious and intentional flood of obvious trolls: in a period of approx. 5 hours, a blog that previously had received a maximum of—what? 35 comments? —on any single post was deluged with 88 more, a great number of which only expressed their appraisal of Sarah’s intelligence, with no reference to what they thought she was mistaken about. Furthermore, I counted about 20 email addresses that were obviously false (there might have been more), implying that those posters knew that what they were doing was insulting and indefensible.
Posted by: Carla at June 14, 2004 06:59 PM (r5M6F)
26
In response to some of the comments in this thread—
From "Mike Adams":
"Lady, you're dumb as a hammer, for gods sake shut up or put this stuff in a private diary!
Ever So Sincerely,
Mike"
Um, Mr. Adams? There are plenty of websites/blogs available where I’m sure you’ll find more of your own kind. When I don’t think a blog is worth anything, I—gasp!—just stop reading it! Try that strategy sometime—isn’t it nice how one is able to move freely about the Internet?
From MaitreDee at June 14, 2004 07:06 PM:
"Those that you call the idiots who were incensed by this... do you think they're wrong in pointing out that what Sarah said was, well, less than an intelligent thing to say?"
ItÂ’s fine to point out that you disagree with someone or even that you think they are factually wrong. ItÂ’s another thing entirely to leave comment after comment with obscenities directed toward someone who did not insult you and whom you clearly did not take any time to understand. Almost every blog IÂ’ve ever looked at required some background to understand who the blogger is and where sheÂ’s coming from; itÂ’s obvious that the trolls from that spree did not do any of that kind of work before shooting off their mouths (e.g., referring to Sarah as "he," etc.).
Posted by: Carla at June 14, 2004 07:01 PM (r5M6F)
27
hmmm... let's see... 53% said blah. Now, ignoring the sample size for a moment, err.. ummm.... bear with me, I'm not a statistician. errr... 3% possibilitiy of error makes it.. ummm...
Right. I think I got it. Somewhere around 50 percent of a polled sample, maybe a hair above, maybe a hair below, but somewhere damned close to that line responded a certain way. How the hell dare anyone consider something so inconclusive to be representative of the VOICE OF AMERICA!!??
Mind you, I ask myself this self-same question just about every day when I think about the 2000 presidential elections.
Posted by: foog at June 14, 2004 07:03 PM (5GN6y)
28
Foog: I fear you're making the same ignorant mistakes as Sarah.
Yes, a 3% MOE might mean those polled are equally divided. It also might mean 56%, vice 53%, said "blah."
You see, the MOE is distributed around the 53%.
Again, if you believe this poll--and others like it--are "irrelevant" as Sarah believes--I can assure you the current occupants at 1600 Pennsylvania are digesting them with mega-doses of antacid.
Posted by: Jadegold at June 14, 2004 08:55 PM (M9P7+)
29
"My own opinion is that a relatively small number of individuals and organizations effectively wield a vastly inordinate degree of power in this nation by manioulating the political system and that one of the key aspects of this manipulation is the dominance of the two-party system. When we succumb to the artificial division of our opinions into two camps and have at each other like unruly children on a particularly nasty playground, we perpetuate this system we laughably call democracy."
Here here! Buy this man a drink! Excellent comment.
Posted by: Bolo at June 14, 2004 09:09 PM (Tgr46)
30
Polls are always biased to some extent, even when the questions are made as neutral as possible and a large, hopefully representative sampling of people is used. However, that bias pales in comparison to what can be achieved if you're actually trying to manipulate the polls, as in the examples given by Sarah, and I found her article and the attached links quite interesting. How do you explain changes in wording from one poll to the next? How do you explain the leaving out of certain questions in the published results? The only possible answer is media bias, and if you guys would stop spewing flames and propaganda long enough to have your heads examined, you'd see it too.
Sarah: I highly doubt the people posting here are a statistically valid sampling of the US population, so ignore them. They're all idiots.
Posted by: Theodore Pride at June 14, 2004 09:22 PM (fOVHf)
31
How different would this whole conversation be if the insults were replaced by constructive criticism? Can someone explain how the article was not biased, rather than your opinion on Sarah's intelligence?
Posted by: Amy at June 14, 2004 10:45 PM (aJ30v)
32
Wow.
There is nothing but rank humour rolling off the walls when the majority of critics of a blog which extols the value of critical thinking can do nothing more than blab dogma and insults.
I'm amazed that so many "smart" people can miss the whole point about bias.....maybe you need to dummy your writing down to the grade two level?
Or better yet, have some fun with stats! Why not count the negative comments in this post and extrapolate how many assholes there are in America!
Keep up the good work!!
Posted by: Jimmi Jazz at June 14, 2004 11:33 PM (fd5Gy)
33
I see that you've left the comments on for this post (thank you!) and I hope you've had time to read my previous e-mail to you. Your post today makes much more sense than your previous one; still, the proposition that you have a thorough understanding of statistics strikes me as dubious. If, indeed, you have an understanding of p-values, sampling, and the requirement that one's sample be IID before you perform inferential statistics, why didn't you approach the LA Times article in that manner? If you're tossing around the term "p-value", you're in the position to write a post that would educate your usual readership about some of these issues. Why not take the time to do so? The LA Times article and the AP wire it was based on do leave out the level of confidence (as do most print reports on polls) and that in itself is worth a post. (And you could just as easily roll in on TV -- this weekend I was watching Fox News and saw them flash up poll results between programming and commercials which, I think, paraphrased the question asked, didn't list the number of respondants, didn't list the margin of error, and didn't list the level of confidence, and didn't attempt to put the poll into context with other polls or other questions asked in the same poll.) And, the fact that you got the sample size wrong led many people to believe, regardless of their personal opinions of you, that you didn't closely read the article you commented on, let alone have a working knowledge of statistics.
But, if, as you and some of your commenters have written, your post was more on the write up of the LA Times article and not the AP-commissioned poll, why not register for the LA Times and actually *read* the article you critique? (C'mon, Sarah, don't tell me that you needed Groucho's comment in the other thread to goad you into reading the actual article.... I can't seriously believe that you instruct your students that when examining the media for bias they need not read the articles they examine?)
Still, I'll argue the merits of the argument you advance in this post. I *strongly* take exception to your point that "polls don't matter, especially close ones." In my years of dealing with statistics, some of the most illuminating results have been the ones where the p-value is on the hairy edge of the rejection regime. Some of the most interesting tests I've been involved in are the ones where there is no clear answer to the question of whether "system A" is better than "system B". Such tests force the decision-makers to readdress what information they wanted to get out of the test, and reassess their appetite for risk before going ahead with a test. As unfortunate as it is for me as a tester involved with the DoD (and as frightening as it may be for the spouse of a military member) often decision-makers don't think about these sorts of things until the data forces them to. Still, I've been generally impressed by their ability to come to a decision (particularly if it allows me to do more testing) based on the data, and not politics.
And when we report test results, we typically report the point estimate and the lower/upper confidence bound. In the case of the question you write about with a 3% margin of error (roughly similar to confidence bounds), we would be quite justified to say that with 95%/90%/80% confidence, there is a difference between the numbers of people agreeing and disagreeing with the question, and that the point estimate is 53 yes/43 no. (This may not be exact - opinion polls are statistically messy because of the existence of those who answer don't know/no opinion/no response and the fact that you're tracking two reponse variables - yes and no - instead of the single one in traditional statistics, however, the point is that with the margin of error and the given spread between yes and no, we can solidly say that a majority agreed with the question.)
Let me close with my comments on an argument you make both in this post and in your original one. I don't think that the Den Beste article you linked to makes the point that all polls are "irrelevant." Rather, the point at the end of his post is that those polls are eventually proved OBE - by actual election results. Now, that point by itself is too cute by half - what about the enormous resources that political campaigns put into polls and focus groups in order to determine what the campaign should do - but Den Beste also links it only to polls dealing with voters preferences on political canidates - not to the sorts of poll (and specifically the question) that you spend much of your post on - polls that deal with levels of agreement with specific talking points or policies. Since I'll never enter a polling booth (or, more acurately, write in an absentee ballot) that will ask me if I agree with Sen Kerry's assessment of Iraq, I don't believe this poll is addressed by Den Beste's argument.
Posted by: Darkwater at June 14, 2004 11:40 PM (fiEVU)
34
Well, with a few exceptions, the liberals have shown their colors here.
Posted by: Parkway Rest Stop at June 14, 2004 11:45 PM (BjDAE)
35
Well I see Carla has detected "fake" e-mail addresses, hey Carla ever think there are those of us that are smart enough to know how to wander around the "net" and not leave a "footprint". Being the centerist/libertarian I am I do not wish my e-mail posted all over the net. So I use a "fake" e-mail. A fake e-mail does not necessarily invalidate the opinion offered by the individual. Now moving on to Sarah herself I did not gather you were living in Germany until after I made what I thought was a calm and rational post(well at least compared to others). I think a good point for everyone to remember is something Chris Rock says in his latest HBO special, I paraphrase here: "No one is ALL one thing (liberal/conservative) we ALL have issues we are liberal on, and issues we are conservative on...Anyone that is not willing to look at the facts before forming an opinion is a fucking fool". Chris in that second clause is absolutely correct. Sarah as I mentioned in yesterdays post I did my service, and I sympathize with your situation. I was single for the entire time of my military days, so while I do not have first hand experience at the seperation problems the military forces on married couples, I did have many good friends that were married, and I saw how hard they had to work to maintain the relationship. I would ask that you note that while I questioned your teaching ability, I was one of very few to avoid resorting to the ad-hominem attacks . I came here today out of curiosity, and obviously of my own volition, since there was no link to your site today. Please remember that although I might disagree with our being in Iraq, I wish a safe return to all our warriors there. Sarah you hang in there, this situation in Iraq will not go on forever,and though I wish I could express the same sentiment about the conflict with terrorists, I am realist enough to know that the terror front will be a long and sometimes hard slog.
Posted by: Bubba Bo Bob Brain at June 15, 2004 01:22 AM (4pVZJ)
36
Sarah,
You have my sympathies for your misfortune of attracting an Atrios link.
You're still one of my daily blogs.
Posted by: John at June 15, 2004 03:19 AM (crTpS)
37
"I was starting to feel sympathetic towards the Left until they started calling me names."
Wow! This is some great political commentary here. You really thought long and hard about your political leanings didn't you? As someone who holds an MA in Political Science, I would laugh like hell if one of my colleagues expressed the above quotation (and or expressed an utter confusion over simple statistics). And if a professor conveyed the idiocy that you have the past few days I most certainly would drop that professor's courses.
Posted by: Tom at June 15, 2004 04:03 AM (ZKWf0)
38
These comments disturb me.
In dozens of replies, I found a handful of items that could be construed as constructive analysis of the content of the original post -- most of these criticisms targeted at the poster's mistaken assumption that 53% positive responses with a 1230 sample and a +/-3% moe could yield 615. Yes, indicative of room to learn, but by no means an invalidation of the meat of the post which conerned itself with poll construct, sampling methodology, and result presentation, and how these are used and abused to advance the agendas of the poll's funders.
In my opinion, this community (in the interest of productive discourse rather than emotionally-based war -- this is our goal, right?) would be much better served if responses were limited to those that logically and specifically challenge the ideas put forth by the poster, rather than opinions regarding the poster him/herself.
Posted by: cerkit at June 15, 2004 07:51 AM (fQ5dB)
39
I wrote one about people with bad manners and a lack of common decency.
Did you include yourself in this group, considering you called commenters to your blog "assholes?"
Posted by: YatPundit at June 15, 2004 09:25 AM (yv+zz)
40
I found your rants interesting and enjoyed them. That fact that a techer is even thinking about these things is even better! I hope you DO take them into the classroom!
Posted by: OffDWall at June 15, 2004 10:38 AM (rZk1W)
41
Bubba Bo Bob Brain: "Well I see Carla has detected "fake" e-mail addresses, hey Carla ever think there are those of us that are smart enough to know how to wander around the 'net' and not leave a 'footprint'. Being the centerist/libertarian I am I do not wish my e-mail posted all over the net. So I use a 'fake' e-mail. A fake e-mail does not necessarily invalidate the opinion offered by the individual."
My point was simply that fake email addresses + obscenities + blatant disregard of readily available information (e.g., Sarah as "he") [multiplied by] many more comments than this blog usually gets = troll swarm.
Trolling is pretty disgraceful in and of itself; ignorantly echoing 40 other trolls' comments (or one person using multiple names to troll the same thread) is even worse. (Note: I'm not saying this is what you were doing, I'm just commenting on the phenomenon of the troll swarm.)
I use a hotmail account for posting to the 'net. It is not my primary address, but it does provide the blogger about whom I'm commenting the opportunity to get in touch with me if they wish. I think it's handy.
Posted by: Carla at June 15, 2004 11:04 AM (r5M6F)
42
Sarah, your comment:
"It's no lie that everywhere in the world that there's conflict, Muslims are somehow involved."
is demonstrably wrong and ill-informed. Nobody denies that the middle east is a basket case, but Muslims have no monopoly on conflict. (Northern Ireland, North Korea, Vietnam, Colombia, Angola, Tibet/China, etc).
Remember WW2? I don't recall the Muslims starting it. Nope - it was a bunch of
evil white guys, if memory serves.
Never post when you're drunk or angry.
Posted by: yet another lefty at June 15, 2004 04:48 PM (sj6jn)
43
Sarah, thank you for responding to my comments. I'm afraid that I didn't make myself completely clear, though, as you state:
I assume close calls are controlled experiments measuring, as he said, whether 'system A' is better than 'system B'." That's science. Opinion polling is not science.
On the contrary, some of the instances I was thinking of when I wrote those sentences dealt with human factors questionnaires (system usability, quality of training) which are nothing more than opinion surveys. Quite often, there is no substitute for the qualitative information from surveys. Lots of human factors engineers have spent lots of time and money to quantitatively measure usability of GUIs (eye dwell time, eye refocus time), for instance, with very few practical results. But, when faced with loads of qualitative data, we can still analyze that data quantitatively, and come out with numeric results with levels of confidence and margins of error. Qualitative data is not inherently fuzzy. (Of course, it can be.) More importantly, qualitative data is not necessarily fuzzier than quantitative data.
That's another issue lurking in the blog host's (I like that term!) assumption about my work; I'm afraid that she has fallen (granted, unintentionally) into one of the misconceptions of testing and hypothesis testing in particular that I hear all the time.
Most of the tests I'm involved in aren't the highly controlled "experimental"-type breadboard/brassboard test setups that I think she is talking about. (Believe me, as an engineer, I'd kill to be in an environment where I had that level of control over... well, the environment.) I'm used to dealing with very gross measures of performance and getting that data from large, complex, messy environments. When you're relying on a test event involving tens, hundreds, even thousands of people, even limited data is still incredibly expensive to generate.
Very often the only way to get human factors information is to ask the system operators their opinions after operations, in essence, calling people and asking them questions. Now, the amount of human factors data that can be reliably collected is limited; for instance, questionnaires lengthen the workday of the operator, usually right before they leave work, so a lengthy questionnaire can lead to a grumpy respondent. But it is possible to write questionnaires that aren't biased, give the respondent a wide and all-encompassing set of responses to choose from, and can generate data ripe for analysis.
And when we finally get to analysis, the great thing about most statistical processes (particularly hypothesis testing) is that they take the variability and sample size of the data into account. Each time people like me report on whether missile miss distance meets specifications, what the mean time to repair a system is, or on user preference of GUI A or GUI B, (or user preference of Policy A or Policy B), we are performing a hypothesis test, whether the raw data was quantitative or qualitative.
I would say that my experience has shown me that you don't need perfectly controlled conditions and 'perfect' data in order to do analysis, and calling a bunch of people and asking them questions need not be deeply flawed. You can certainly run into problems - but that's true of just about any procedure, or indeed, any thought process. I encourage everyone, whoever they may be, whatever education level they have, whatever they do, to approach how they live and how they think in a critical manner and be aware of those pitfalls - we can't just smugly pile them up on the doorsteps of the Gallups of the world.
Posted by: Darkwater at June 16, 2004 12:40 AM (XYkvR)
44
You still don't get statistics, or survey methods. Sorry.
Posted by: a statistician at June 16, 2004 11:56 PM (TTMxF)
45
Sarah, When you called the commenters "assholes" you lost any chance of the high ground. Easy to lose your temper, perhaps, but that's the result.
The "only 615 people" comment is the striking one. You spell out above that you have the educational background to know what a fallacy this is, so maybe you could have addressed that more. At least you could balance the comment by saying that maybe only 492 people in the USA think that war was justified (by using the same reasoning). Then it wouldn't have seemed such a biased, agenda-driven comment.
For a study of bias in the LAT, you could also look at the 20,000-person anti-war demonstration in LA that they didn't cover. The documentary film "We Interrupt This Empire" would be a suitable starting point for a class on it (Google for it).
You say you don't care what other countries think. That was Saddam Hussein's attitude, wasn't it?
Posted by: Martin Poulter at June 17, 2004 08:41 AM (Qk9Ls)
46
"You still don't get statistics, or survey methods. Sorry.
Posted by a statistician at June 17, 2004 04:56 AM"
Only the Internet can facilitate these wonderful gems of prose.
"Did you include yourself in this group, considering you called commenters to your blog "assholes?""
You can't teach! You're an idiot! You can't teach! You're an idiot! You can't teach! You're an idiot!
You're an asshole.
GHASP!
Posted by: hao2lian at June 18, 2004 11:38 AM (x0PqM)
47
Sarah, though I disagree with your politics, I salute you for making this blog and its comments available. My respect for conservatives has increased ten-fold.
ruben
Posted by: ruben at June 20, 2004 02:28 AM (sTqLM)
Hide Comments
| Add Comment
