Back in 2004, Jim Gilchrist, a retired Marine and the founder of the California Minutemen Project, emailed a few dozen friends and family suggesting that concerned civilians personally combat illegal immigration by traveling to the Arizona border with him. Gilchrist lives in Orange County, Calif., but the Arizona border was the most heavily trafficked and sparsely patrolled. That email reached thousands of people and touched a nerve. Hundreds showed up in April 2005 to patrol the border. Some of them brought floppy hats, lawn chairs, binoculars and American flags. Others toted guns and protest signs. The group banned neo-Nazis from attending, though some came anyway. A movement was born.

Gilchrist estimates he did 4,000 radio and TV interviews over the next five years as his group's membership swelled and the media attention exploded. "It was just literally overwhelming," he said.

But today, the once-thriving Minutemen anti-illegal immigration fraternity has all but died out. No one knows exactly why the groups fizzled so quickly, but researchers and former border-watching leaders say infighting and bad press have taken a toll. At the same time, the tea party movement started to rise, which usurped members and stole the groups' thunder.

Still, the movement's message and popularity have left an indelible mark on the Republican Party, whose leaders underestimated the anger in their base over illegal immigration. The GOP, which at the time was considering legislation to legalize undocumented immigrants in a version of Ronald Reagan's 1986 immigration reform law, rejected the popular movement at first. President George W. Bush dismissed the Minutemen as "vigilantes," while Customs and Border Protection Commissioner Robert Bonner said he worried the volunteers would get hurt or hurt illegal immigrants.

Seven years later, Gilchrist tells Yahoo News that the Minutemen Project has petered out amid expensive legal battles over control of the group. Some of his former comrades attempted to fire him as president, alleging that he was using the group's funds inappropriately. He countersued for defamation and lost, but eventually won back control of the group in court. Outspoken activists mainly interested in money and fame infiltrated the ranks and tried to take over, says Gilchrist, distracting from the original goal of border watching. "There are bad apples," he said. "There are some in any group."

Meanwhile, the tea party emerged and absorbed the groups' concerns about illegal immigration and many of their leaders.

"You see an endless amount of splintered groups that range from a membership of one to a membership of 20," Gilchrist said. "No longer will you have a Minutemen organization going to the border with thousands of people." Last year, he began organizing a border rally planned for this May, but dropped the idea due to lack of interest and continued infighting. "I just decided it wasn't a good idea," he said. "There was just too many negative feelings." The last Minutemen Project outing to the border was in May 2011.

The downward trend is reflected in all the local Minutemen groups that sprouted up from 2005 to 2010 and then tapered off.

According to an analysis by Leonard Zeskind, who researches and advocates against what he calls far-right, racist or anti-Semitic groups, the number of Minutemen organizations dropped by more than half from 2010 to 2011. Only 53 Minutemen groups showed signs of activity last year—down from a high of 115 in 2010—and none of them are currently patrolling the border. Zeskind blames the tea party for absorbing a good chunk of the movement. The widely publicized murders perpetrated by a former Minutemen leader against a Hispanic child and her father also contributed to the groups' decline. Heidi Bierich, director of the Southern Poverty Law Center's Intelligence Project, says that the only organized volunteers patrolling the border right now are the ones originally not welcome: neo-Nazis.

But it's not all bad news for border activists that the tea party has effectively absorbed them and their mission. More than 100 leaders of local anti-illegal immigration groups have now joined tea party chapters, according to Zeskind, giving their message a bigger platform. And even as their movement dissipates, their message grows louder. "The Minutemen put immigration on the map big-time," Bierich says. One example of the tea party-Minutemen melding is the Rally for Arizona in Phoenix in 2010, which tea partyers organized and attended. Its mission: to support Gov. Jan Brewer's state-level crackdown on illegal immigrants. Then there was the Mississippi Tea Party's push for that state's anti-illegal immigration law, which is modeled after the laws in Alabama and Arizona.