Crispin Blunt lost the whip after he was detained by Surrey Police on suspicion of rape and possession of controlled substances.
The former minister, who denies wrongdoing, is the latest in a string of sitting MPs to be arrested on allegations of sexual offences.
Education Secretary Gillian Keegan insisted the arrest is not a sign of a wider problem in the party.
She said: “I certainly don’t see a cultural issue among Conservative MPs. I see individual incidents which are all investigated as such.
“Due to the serious nature of the allegations… he’s had the whip suspended, but the police are involved now so beyond that it’s not really appropriate for me to comment any further.
“The Prime Minister’s been clear about high standards, he expects high standards, he always follows due process, but all you can do with these things is deal with them as they arise and take the appropriate action.”
Pressed about “sleaze” within the Conservative party, Ms Keegan said: “I’ve been in Parliament now for six and a bit years and I must say honestly, personally, I’ve never seen any impacts or any sort of, this kind of behaviour.”
Mr Blunt, 63, identified himself as the MP at the centre of a police probe on Thursday and said he was first interviewed by police three weeks ago after reporting concerns over extortion.
The Reigate MP said he will “co-operate fully” with the ongoing investigation and is “confident” that he will not be charged.
Conservative chief whip Simon Hart has told Mr Blunt to stay away from Parliament during the process.
In May 2022, it emerged that a Tory MP in his 50s had been arrested on suspicion of indecent assault, sexual assault, rape, abuse of position of trust and misconduct in public office.
The Tory whips’ office said at the time that the MP had been asked not to “attend the parliamentary estate while an investigation is ongoing”.
The latest extension to his bail is to mid-February. One Conservative MP was arrested on suspicion of rape in 2020. He was never identified and police took no further action after an investigation.
Mr Blunt issued a statement confirming he had been arrested. “The arrest was unnecessary as I remain ready to co-operate fully with the investigation that I am confident will end without charge,” he added.
