I didn’t mention that Turkmenistan is run by an ugly, neo-Stalinist regime. They surely knew that, and besides, they didn’t care. As I explained in this month’s issue of Harper’s Magazine, the lobbyists I met at Cassidy & Associates and APCO were more than eager to help out. In exchange for fees of up to $1.5 million a year, they offered to send congressional delegations to Turkmenistan and write and plant opinion pieces in newspapers under the names of academics and think-tank experts they would recruit. They even offered to set up supposedly “independent” media events in Washington that would promote Turkmenistan (the agenda and speakers would actually be determined by the lobbyists).

All this, Cassidy and APCO promised, could be done quietly and unobtrusively, because the law that regulates foreign lobbyists is so flimsy that the firms would be required to reveal little information in their public disclosure forms.

Now, in a fabulous bit of irony, my article about the unethical behavior of lobbying firms has become, for some in the media, a story about my ethics in reporting the story. The lobbyists have attacked the story and me personally, saying that it was unethical of me to misrepresent myself when I went to speak to them.

That kind of reaction is to be expected from the lobbyists exposed in my article. But what I found more disappointing is that their concerns were then mirrored by Washington Post media columnist Howard Kurtz, who was apparently far less concerned by the lobbyists’ ability to manipulate public and political opinion than by my use of undercover journalism.

“No matter how good the story,” he wrote, “lying to get it raises as many questions about journalists as their subjects.”

I can’t say I was utterly surprised by Kurtz’s criticism. Some major media organizations allow, in principle, undercover journalism – assuming the story in question is deemed vital to the public interest and could not have been obtained through more conventional means – but very few practice it anymore. And that’s unfortunate, because there’s a long tradition of sting operations in American journalism, dating back at least to the 1880s, when Nellie Bly pretended to be insane in order to reveal the atrocious treatment of inmates at the Women’s Lunatic Asylum on Blackwell’s Island in New York City.

In the late 1970s, the Chicago Sun-Times bought its own tavern and exposed, in a 25-part series, gross corruption on the part of city inspectors (such as the fire inspector who agreed to ignore exposed electrical wiring for a mere $10 payoff). During that same decade, the Chicago Tribune won several Pulitzer Prizes with undercover reporting and “60 Minutes” gained fame for its use of sting stories.

Today, however, it’s almost impossible to imagine a mainstream media outlet undertaking a major undercover investigation. That’s partly a result of the 1997 verdict against ABC News in the Food Lion case. The TV network accused Food Lion of selling cheese that had been gnawed on by rats as well as spoiled meat and fish that had been doused in bleach to cover up its rancid smell. But even though the grocery chain never denied the allegations in court, it successfully sued ABC for fraud – arguing that the reporters only made those discoveries after getting jobs at Food Lion by lying on their resumes. In other words, the fact that their reporting was accurate was no longer a defense.

The decline of undercover reporting – and of investigative reporting in general – also reflects, in part, the increasing conservatism and cautiousness of the media, especially the smug, high-end Washington press corps. As reporters have grown more socially prominent during the last several decades, they’ve become part of the very power structure that they’re supposed to be tracking and scrutinizing.

Chuck Lewis, a former “60 Minutes” producer and founder of the Center for Public Integrity, once told me: “The values of the news media are the same as those of the elite, and they badly want to be viewed by the elites as acceptable.”

In my case, I was able to gain an inside glimpse into a secretive culture of professional spinners only by lying myself. I disclosed my deceptions clearly in the piece I wrote (whereas the lobbyists I met boasted of how they were able to fly under the radar screen in seeking to shape U.S. foreign policy). If readers feel uncomfortable with my methods, they’re free to dismiss my findings.

Yes, undercover reporting should be used sparingly, and there are legitimate arguments to be had about when it is fair or appropriate. But I’m confident my use of it in this case was legitimate. There was a significant public interest involved, particularly given Congress’ as-yet-unfulfilled promise to crack down on lobbyists in the aftermath of the Jack Abramoff scandal.

Could I have extracted the same information and insight with more conventional journalistic methods? Impossible.

Based on the number of interview requests I’ve had, and the steady stream of positive e-mails I’ve received, I’d wager that the general public is decidedly more supportive of undercover reporting than the Washington media establishment. One person who heard me talking about the story in a TV interview wrote to urge that I never apologize for “misrepresenting yourself to a pack of thugs … especially when misrepresentation is their own stock in trade!”

I’m willing to debate the merits of my piece, but the carping from the Washington press corps is hard to stomach. This is the group that attended the White House correspondents dinner and clapped for a rapping Karl Rove. As a class, they honor politeness over honesty and believe that being “balanced” means giving the same weight to a lie as you give to the truth.

I’ll take Nellie Bly any day.

This article was first published in The Los Angeles Times.

This article was first published at Harper’s.

1. What’s your assessment of the impact thus far of the “surge”?

It’s going about as expected, changing the distribution of violence a bit but not making much difference in the core strategic issues. It would be easy to just look at the trends in violence, but that’s not really the point. When the administration laid out its plan, it said some of the right things, like that success should not be judged on military outcomes and body counts. Their argument was that the surge would create a secure political space that would allow for political reconciliation. So far, the opposite has happened; there’s been little progress towards reaching a new political compromise and if anything the distance between the sides seems to be growing. On the military side, there have been some interesting developments in Anbar province, like you’ve been reading about in the press lately, but that has little to do with the “surge.”

2. What is going on in Anbar?

There really is a palpable turn there against Al Qaeda, that isn’t just the usual wishful thinking that so often takes the place of real analysis. A lot of people have interpreted this as a sign of American strength, that the Sunni tribes are shifting to the winning side. It’s actually just the opposite, it’s a defensive reaction by Sunnis to Al Qaeda’s increasing strength and aggressiveness. Sunni resentment of Al Qaeda in Iraq really dates to last October, long before the “surge,” when Al Qaeda declared the Islamic State of Iraq. A lot of us thought at the time that they did this for strictly propaganda purposes, but it developed into an aggressive bid for hegemony over the entire insurgency. The Islamic State of Iraq became very aggressive towards other insurgency groups and local Sunnis, intimidating ordinary people, declaring them to be non-Muslims, and using that as a justification for seizing property and killing leaders of other groups. This created a backlash; we’re seeing an open turn against Al Qaeda not just by local tribal sheiks and ordinary people but also by the leaders of the insurgency.

The insurgents are very critical of Al Qaeda, its treatment of Sunnis and its extreme interpretation of Islam, but at the same time they are deeply committed to continued resistance to the American occupation.

3. How are the tensions between the local insurgents and Al Qaeda playing out on the ground?

The American media has focused on the Anbar Salvation Council, a group of tribal sheikhs that have asked the American military for help against Al Qaeda. But they really aren’t that important – what matters is that the major insurgency groups have turned on the Islamic State of Iraq project. The split really got serious in early April, when the Islamic Army of Iraq, which is one of largest insurgent factions, openly broke with Al Qaeda and issued a scathing denunciation of the Islamic State in Iraq. A number of other factions joined in, and now they’ve formed something called the Reform and Jihad Front. There are two main issues: local grievances and some real strategic differences. The insurgents are very critical of Al Qaeda, its treatment of Sunnis and its extreme interpretation of Islam, but at the same time they are deeply committed to continued resistance to the American occupation. They want a less divided and more effective resistance, not an end to resistance. But there’s also a real divide in strategy that goes beyond the local grievances, which Americans really need to understand. Al Qaeda wants the United States to stay in Iraq as long as possible. It gets tremendous benefits from having American troops close at hand to kill – Iraq is the primary source of its propaganda and recruitment, and an integral part of its global strategy. They really want to turn Iraq into a base for exporting global jihad. But these major insurgency factions are focused on driving Americans out of Iraq and creating a political system that gives Sunnis a reasonable stake in politics.

The insurgents have made it pretty clear in a series of public statements and private communications that they’re willing to start talking and dampen down the violence if the United States commits to withdrawing from Iraq.

4. Whatever the cause of the split, isn’t it good news either way?

Only if we get our strategy right and learn the right lessons. From an American point of view, if you believe that the “surge” has emboldened the Sunnis to turn against Al Qaeda than what follows is that we should stay for a long time, reassure our allies, and wipe out Al Qaeda. But if you believe, as I do, that the major insurgency groups that are turning against Al Qaeda mainly want the United States to get out and then rejoin the political system, it leads to another conclusion. If you listen to what these insurgency factions are saying, what Hareth al-Dhari of the Association of Muslim Scholars is saying, they couldn’t be making it more clear: make a credible commitment to withdrawing and the insurgency will dampen down and we’ll take care of the Al Qaeda groups out of our own self-interest. Just to be clear, I’m not saying the mainstream insurgents are good guys. They have the most American blood on their hands, but that’s precisely why we can’t ignore them, it’s why we’ve been exploring the possibility of talking to them in the past, and it’s why any sustainable deal in Iraq will require bringing them in.

5. So what’s the best policy choice at this point?

The United States should commit to a withdrawal, not tomorrow but with a clear endpoint – benchmarks, or whatever you want to call them. The insurgents have made it pretty clear in a series of public statements and private communications that they’re willing to start talking and dampen down the violence if the United States commits to withdrawing from Iraq. We’re at a moment where there’s actually a chance for positive developments, because we have a common interest with the insurgents in defeating Al Qaeda and they are putting out clear signals that they are willing to make a deal. But everything hinges on the United States making a commitment to withdraw – politically, they can’t and won’t get in the political game without that because it would destroy their credibility and because, frankly, getting the United States out really matters to them. But there’s a window here that I’m afraid we’re going to let close because of domestic politics. The insurgency factions turned against Al Qaeda because its Islamic State of Iraq project has been growing in strength, and if they can’t show some gains soon the tide may turn against them within the Sunni community.

6. Beyond Iraq, how is Al Qaeda doing in other parts of the world?

Al Qaeda is trying to spread its jihadist world view, the notion of a fundamental clash of civilizations and the idea that Islam is under threat. It’s remarkable how rapidly and deeply this way of understanding the world is becoming entrenched in the Arab world. At the same time, neither Al Qaeda as an organization nor bin Laden as an individual is commanding a great deal of respect or support. When you get these attacks in Algeria and Morocco, it repels people rather than attracting them. But the paradox is that even as Al Qaeda repels people with its actions, its core ideas are becoming more widely accepted, and that’s really troubling, and a real indictment of American public diplomacy. That’s also why the situation in Iraq is so devastating at the wider regional and global level. Killing people in Morocco and Algeria triggers a negative reaction, but fighting Americans in Iraq resonates with a much wider part of the Arab population. The Project on International Policy Attitudes at the University of Maryland conducted a survey of Muslim public opinion a few months ago. 91 percent of Egyptians disapproved of attacks against civilians in the United States and only 7 percent disagreed with the statement that “groups that use violence against civilians, such as Al Qaeda, are violating the principles of Islam.” But 91 percent said that attacks on American troops in Iraq were legitimate, and 92 percent agreed with the goal of “getting the U.S. to withdraw forces from Islamic countries”. That gives you a sense of why jihad in Iraq is so vital to Al Qaeda – it’s a place where their violence is popular.