"Vattenfall will discontinue its R&D (research and development) activities regarding coal power with CCS (carbon capture and storage)," the group said in a statement explaining its new research plans.
The state-owned giant had been investing in this technology for more than 10 years, with plans for a power plant equipped with CCS in 2016.
Capturing and liquifying CO2 coming from carbon combustion to later store it underground was meant to curb greenhouse effect gas emissions, but its costs and the energy it requires make the technology unviable.
These difficulties had already forced Vattenfall to give up in 2011 a large project at a pilot plant in Jänschwalde, in eastern Germany.
The European Union then demanded the reimbursement of funding worth €45 million euros (410 million kronor/$62.75 million), but neither Vattenfall nor the EU ever said whether the group complied with the request.
In late 2011, the Swedish company said it still believed in the project and stated that it expected to build a coal power plant equipped with CCS by 2025.
But Tuesday, the group said that CCS was not among its priorities anymore.
"We are evaluating our research portfolio in order to invest in R&D projects which can contribute more quickly to our business development," Research and Development Nordic head Karl Bergman said.
With a capacity of 11,300 megawatt in 14 plants in Germany, Denmark, the Netherlands, Vattenfall is one of the biggest European coal and lignite -- a combustible rock considered the lowest rank of coal -- electricity producers, which accounted for 40 percent of its total production in 2013.