Civil rights have come under increasing pressure since President Xi Jinping took power in 2012, with widespread arrests of lawyers and activists.
Gui, a Swedish citizen, was one of five Hong Kong-based booksellers, known for salacious titles about the lives of China's political elite, who went missing in 2015 and resurfaced in detention on the mainland.
Chinese authorities said they had released Gui in October, but it was unclear to what extent he was a free man.
His daughter Angela Gui told Radio Sweden that since her father's official release from detention, he had been put in a police-managed flat under surveillance.
She said that he had then been snatched by plain clothes police on Saturday while on a train to Beijing from the eastern city of Ningbo, where he was living, while accompanied by two Swedish diplomats.
The Swedish foreign ministry said it was “fully aware” of what had happened to Gui, and Swedish Foreign Minister Margot Wallström summoned China's ambassador to a meeting.
Rights group Amnesty International described the incident as “absolutely appalling” and called for Gui to be released and allowed to seek medical treatment.
The fact that he had been snatched in front of diplomats should be a “wake up call” to the international community, said Amnesty's China researcher William Nee.
Literary society and activist group PEN Hong Kong expressed “highest concern” over Gui's latest disappearance.
Chinese authorities were widely criticised after veteran Chinese rights activist and Nobel Laureate Liu Xiaobo died from liver cancer while on medical parole on the mainland in December last year.
Rights groups had pushed for him to be allowed to seek medical treatment abroad.
Gui first disappeared in 2015 while on holiday in Thailand and was detained at an undisclosed location in China.
In February 2016 he appeared on Chinese television, weeping as he confessed to involvement in a fatal car accident years before.
In another interview the same year, he also admitted trying to smuggle illegal books into China.