Brewster scored in the 11th and 14th minutes at Goa, fed Gibbs-White for a 3-0 lead in the 64th and converted a penalty kick in the sixth minute of second-half stoppage time.
Josh Sargent scored in the 72nd minute, his third goal of the tournament for the Americans. Sargent, who agreed last month to join Germany's Werder Bremen in January, had four goals at this year's Under-20 World Cup.
England plays Germany or Brazil on Wednesday. Mali defeated Ghana 2-1 on Saturday and meets Spain or Iran in the other semifinal.
Brewster put England ahead after Phil Foden dribbled down his right flank, going past Tim Weah and then Chris Gloster. U.S. goalkeeper Justin Garces dived and used his right hand to parry the angled shot. Brewster, a member of Liverpool's youth program, flicked in the ball from 7 yards.
James Sands' turnover near midfield led to the second goal. Foden passed to an open Brewster, who chipped the ball over an unrushing Garces from 12 yards.
Brewster also scored in the group stage against Mexico.
Garces made a left-handed save to deny Brewster in the 16th.
Sargent had a chance to cut the deficit in the 23rd when a corner kick went to him at the far post, but his bounced shot hit the underside of the crossbar. Taylor Booth's shot was denied by Chris Anderson in the 37th, and Sargent headed the ensuing corner kick over the crossbar.
The U.S. scored when Andrew Carleton passed to Sergino Dest, whose shot was blocked by Anderson only for Sargent to poke in the rebound. England defender Jon Panzo was down at the time after a collision with Chris Durkin.
Dest tripped Brewster in the penalty area in the fourth minute of stoppage time, earning a red card.