England became the last team to qualify for the World Cup quarter-finals after a dramatic penalty shootout as favourites Brazil and fancied France trained their sights on their last-eight opponents.
England's victory against Colombia takes Southgate's team into what looks like a winnable match against Sweden, who edged out Switzerland 1-0 earlier on Tuesday.
Southgate said he was already looking forward to Saturday's quarter-final against Sweden in Samara, with a last-four clash against either Croatia or Russia up for grabs.
“This was special but I want us to go on,” said Southgate, whose own penalty miss saw England lose to Germany in the semi-finals of Euro 96 at Wembley.
“Sweden is another team we have a poor record against. We have underestimated them for years. They have created their own story and made history. I don't want to go home yet.”
Janne Andersson's Sweden have recovered well from a heartbreaking group-stage defeat by Germany, seeing off Mexico before edging out Switzerland 1-0 in the last 16.
RB Leipzig midfielder Emil Forsberg grabbed the only goal of a tight encounter midway through the second half to send his country into the quarters for the first time since the 1994 side who finished third in the United States.
MEMBERS' QUIZ: Test your knowledge of Sweden at the World Cup