The deadly siege on the U.S. Capitol on Jan. 6 has prompted U.S. security officials to think the unthinkable as they scramble to secure Washington ahead of next week’s Inauguration: that the enemy is already inside the house.

More than a dozen law enforcement officers and current and former military officials are reported to have taken part in the violent Jan. 6 insurrection that killed a U.S. Capitol Police officer and cost four supporters of President Donald Trump their lives. One Navy and two Air Force veterans are among those being investigated by law enforcement for the attack, as is a junior Army officer by her superiors, while several U.S. Capitol Police officers have been suspended after video showed them appearing to assist some of the rioters who were spurred to action by Trump’s refusal to accept defeat.

Now the FBI is warning of planned armed protests at the Jan. 20 Inauguration in Washington, D.C., and in all 50 state capitals, current senior U.S. officials tell TIME. Current and former security officials say they are concerned that serving U.S. troops or law enforcement officers could pose a clear and present danger to the President- and Vice President-elect and other senior U.S. lawmakers on Inauguration Day. Federal investigators are also trying to track down military and law enforcement members or veterans who took part on Jan. 6, and trace their wider network of associates who may be plotting to turn next week into the mayhem being called for on far-right forums.

But there are too many people to look at, and too little time to do it, says Mitch Silber, former Director of Intelligence Analysis at the New York City Police Department. In Washington alone, up to 20,000 National Guardsmen and hundreds of city, federal and neighboring state police will be on patrol.

“We might be talking one or two bad apples here, not anything systematic,” Silber says. Rooting them out would take an internal affairs-style investigation, possibly of entire agencies that are involved in Inauguration security. “We just don’t know, and there’s just no time to conduct that type of investigation.”

The growing sense of urgency and anxiety was reflected in an unprecedented letter from the Chairman of the Joint Chiefs Gen. Mark Milley and all the service chiefs to military members on Tuesday. They wrote that the “violent riot in Washington, D.C.” was a “direct assault…on our Constitutional process.” The chiefs added that the “rights of freedom of speech and assembly do not give anyone the right to resort to violence, sedition and insurrection.”

Their concern is shared by lawmakers who are incensed over the events of Jan. 6 and worried about security preparations underway for Jan. 20. After an FBI briefing on Tuesday, the Democratic chairmen of the Judiciary, Intelligence, Armed Services and Oversight committees released a statement that it’s “clear that more must be done to preempt, penetrate, and prevent deadly and seditious assaults by domestic violent extremists in the days ahead.”

“There is a crisis issue: the rise of extremism and white supremacy in the ranks,” retired Army officer Rep. Jason Crow (D-Colo.) told Politico on Monday. That rise, he said, has been “fueled by President Trump, unfortunately. So that has to be dealt with right away and unequivocally.”

Extremist experts and former law enforcement officials can only guess at how many of the nation’s police and military are members of militia or other extremist groups, or even hold extremist views they might be willing to act on. Of the nation’s roughly 800,000 police, it’s probably far less than one percent, says Mark Pitcavage of the Anti-Defamation League’s Center on Extremism. “When you have a body that huge, you’re gonna find some people with ties to extremism in it. That’s just a given.”

Nevertheless, that tiny fraction could still mean a sizable number of trained professionals could have the means and intent to cause serious harm or damage to express their anger over Trump’s defeat and the loss of life among rioters at the U.S. Capitol, including military-veteran-turned-martyr Ashli Babbit. More worryingly, many troops seasoned from fighting terrorists overseas know insurgent tactics, such as communicating via encrypted apps rather than expressing their plots over the now-at-least-temporarily-defunct Parler app, says terrorism expert Mia Bloom of Georgia State University.

“I don’t think it’s hyperbole to say that it’s extremely dangerous when you’re talking about people that have actual training in the military, or in law enforcement,” adds Colin Clarke, head of research for the Soufan Group. In reviewing video clips of the assault on the Capitol, he noticed rioters using specialized military tactics. “People were being commanded to move through a broken window in twos,” he noted, reminiscent of how U.S. troops in Iraq or Afghanistan to enter a building.

Frank Figliuzzi, former FBI Assistant Director for Counterintelligence, has also watched the videos, and says while the “vast majority of those inside the building really have no idea of what they’re doing,” there were a handful of people that seemed “very personal purposeful and seemed to know where to go and what to do, and came equipped with flexi ties and other, and other kinds of tactical gear.”

Some who entered the building wore the militia patch of the Oath Keepers, a group he says brags about having a number of military and law enforcement members in its ranks. “When you have a President who holds campaign rallies called ‘Cops for Trump,’ and they are heavily attended…all of this makes the challenge of securing Washington, securing the inauguration, even more difficult than it already is.”

A disappearing target

Ironically, the Jan. 6 insurrection has handed the FBI the best possible blueprint to find future plotters, giving them legal cause to investigate not just those caught on camera storming the Capitol but associates who cheered them on and say they want to take part in up to four days of further, possibly armed insurrection before and on Inauguration Day. The Justice Department has already opened 170 case files with more on the way, officials said Tuesday.

One problem in their investigation, however, is that many of the would-be anarchists are erasing themselves online, according to Army veteran Jeff Bardin of private intelligence firm Treadstone 71.

Military members have always been careful not to use real identities online, but are becoming even harder to spot in the immediate wake of the Jan. 6 attack, he says. In the last week, he has tracked well-known neo-Nazis and other extremists deleting social media posts and taking their conversations “private” on encrypted apps like Telegram or moving to encrypted app Signal or GAB, a site popular with the alt-right, now that Parler has been taken offline.

“Everybody’s scrubbing their sites and trying to remove things if they participated in the insurrection last week,” Bardin says. “They’re