THis is from the Environics Website. THe site has lots of useful information on how the poll was conducted plus an easy to read summary of the results just as good if not better than the CBC site.
One wonders how it was decided who to poll. There must surely be districts where it is difficult to poll. The polling company has actually had a pollster shot and killed. I would venture to guess that the area is now not polled at all! It would seem natural to pick people that one knows would probably co-operate and that may give you representation in terms of gender, age, location, etc. but could provide unrepresentative answers. Anyway I am surprised that there is not even any curiousity about these issues in the media. The polling responses as mentioned at 85 per cent if far above what you would get in the west. Why?
On doing a bit of research I found that D3 Systems Inc. is a US firm or at least the president is located in Vienna Virginia. The ACSOR is thus not a native Afghan firm but a subsidiary of D3. D3 was obviously probably set up by some entrepreneurial American after the "regime change" facilitated setting up such a company.
Methodology
The research was designed by Environics, in consultation with its media and academic partners.
The survey was conducted for Environics by D3 Systems Inc. and its subsidiary, the Afghan Centre for Social and Opinion Research (ACSOR-Surveys), based in Kabul. D3 Systems/ACSOR established the capability to conduct country-wide public opinion surveys across Afghanistan following the overthrow of the Taliban in 2001, and includes among its clients the British Broadcasting Corporation (BBC), ABC News, the Asia Foundation, and the United Nations Industrial Development
The Environics survey was conducted by ACSOR between September 17 and 24, 2007 with a representative sample of 1,578 adult Afghans (18 years and older) across all 34 of the country's provinces. The surveys were conducted in-person in people's homes in either Dari or Pashto, the two dominant languages. Female interviewers interviewed women and male interviewers interviewed Afghan men, and the sample was stratified to ensure a 50-50 distribution on gender.
The survey sample consisted of 1,278 interviews conducted across the country, plus an additional 300 interviews to provide over-samples of 270 interviews in Kabul (the country's capital) and 260 in the province of Kandahar, where the Canadian mission is based. The margins of sampling error for these samples are plus or minus 3.8%, 7.3% and 7.3%, respectively (at the 95% confidence level). The response rate for this survey was 85 percent, a rate almost unheard of today for research conducted in the western world. Further details on the methodology used to conduct this survey are available from Environics
1 comment:
Hi Ken,
I just wanted to respond to your comments about the Environics poll, which has raised concern among people who have worked in Afghanistan.
Your research into D3 Systems and ACSOR were useful. If you're still interested, here are some other points that I think bear some discussion on the poll's methodology:
Concerns with Validity -
Methodology involves entering people's homes and ask people's opinions on the military, especially the Afghan National Army/Afghan National Police. While the ANA/ANP are not quite like the militia in Iraq yet, they (esp the ANP) are very corrupt and often seen as dangerous to civilians.
Poll was conducted from September 17-24th, right at the beginning of the Holy month of Ramadan, which for many Muslims represents a period of charity and goodwill, and when the good that is done by fasting can be considered void is one speaks ill of others behind
their backs.
Afghans' oral culture and hospitable nature makes the linearity, aggressively
direct, and confinement of responses into five categories of intensity (highly agree, somewhat agree, etc) bewildering. My own direct attempts at conducting quantitative research in Afghanistan are written up here (Kish grid, audience research survey):
http://cms.mit.edu/research/theses/SarahKamal2005.pdf, pages 42-3, 81-3. The problems I've listed in my Master's thesis barely skim the surface of the research challenges I've continued to have while conducting my PhD.
I have spent 7 years working in and around Afghanistan as an academic, development practitioner, and "undercover Afghan." As a Dari-speaking Afghan-looking woman, I have tended to find that after you scratch the surface of Afghan discourse, something else comes out that could never adequately be captured in as blunt and culturally unfamiliar a tool as a western poll. I usually find that people from other cultures tend not to appreciate the underlying resentment or suspicion felt by many Muslims towards the powerful West, and how quickly it can bubble up over a quiet discussion over a cup of tea.
Finding a good facilitator for polling is hard in Afghanistan. ACSOR has done polls for organizations like the Asia Foundation (said to have been founded with CIA funding) and the US state department, and their polls tend to have eyebrow raising results which run counter to other research but are advantageous for suggesting the military operations are running well. The Environics poll is not the first strange public opinion poll coming out of Afghanistan by ACSOR.
Sometimes the timing of the release of such polls is telling. I did a survey of publicly available public opinion in Afghanistan in Dec 2005, it is available here: http://c4o.unitycode.org/me/PeaceConditionalities.final.20060413.pdf . The studies that I looked at are listed in the appendix. Shortly after I finished this study (which found sharp pessimism and a downturn in public opinion), a new quantitative survey was released that claimed that Afghans were very pleased with the reconstruction process and international presence, released right before a major donor conference. This was in the same year that friends of mine were chased out of a UN compound in Jalalabad by angry mobs, who set fire to the compound. Also the same year as the Koran riots and Afghan Minister of Planning Bashardoost winning major public support in demanding that NGOs leave the country.
Methodology doesn't state how questions were piloted. Were there ways of triangulating responses? For instance, if people are so positive about the future, why is it that in the Environics poll only 40% think the government and foreigners will prevail in the current conflict? (20% believe the Taliban will win, 40% don't know). 20% believe Al Qaida is a positive force in the country - how does that mesh with other responses?
Concerns with generalizability -
Poor to non-existent communications and road infrastructure in rural areas, inadequate mapping, lack of security, illiteracy, widely divergent population estimates and shifting displaced populations hamper statistical generalizability of their poll of about 1,500 Afghans.
--
I have been in Afghanistan many times in the last 6 years, and in my three visits this year I found the security situation to be the worst I have ever seen. I first entered Afghanistan during the time of the Taliban, and even then did not feel as threatened as I did in my most recent journey in October 2007. There is no sense of safety anywhere, and even longtime Afghan friends of mine now feel uncomfortable entering downtown Kabul. Such fears could only have worsened with the Nov 6th suicide bomb killing children and MPs in Baghlan, formerly considered a "safe" area.
I have been wrong more times than I can count when it comes to Afghanistan, which I find a fascinating and unendingly complicated space. I don't object to surprising research findings, but I do object to bad science that run counter to common sense. The Environics poll runs counter to what I and other longtime development workers have found to be the mood in the country (including a practitioner who has lived for 6 years in Kandahar). The poll is also dangerous, in my opinion, because the word for expressing the public's mood that is more and more being bandied about in expert circles, and among Afghans, is "occupation." I was a panelist at the Middle East Studies Association annual conference this weekend, and everybody there agreed with that framing. So I believe it is particularly important to not allow a poll (which, as we understand, even in the best of situations is just a poll and not reflective of anything other than what people choose to say to a pollster) to be taken as more than it is.
Best regards,
Sarah Kamal
2007 Trudeau Scholar
PhD Candidate, London School of Economics
Post a Comment