Constable's The Hay Wain came second in the poll, organised by BBC Radio 4's Today programme in association with the National Gallery in London.
A Bar at the Folies-Bergere by Edouard Manet came third, while The Arnolfini Portrait by Jan van Eyck came fourth.
Any painting in a British art gallery was eligible, regardless of its origin.
The other paintings on the shortlist were:
The Baptism of Christ by Piero della Francesca
A Rake's Progress by William Hogarth
The Reverend Robert Walker Skating on Duddingston Loch by Sir Henry Raeburn
The Last of England by Ford Madox Brown
Sunflowers by Vincent Van Gogh
Mr and Mrs Clark and Percy by David Hockney
The shortlisted paintings spanned more than five centuries of European art and included works by British, Italian, Dutch, Belgian and French artists.
The full title of Turner's painting, completed in 1839, is "The Fighting Temeraire tugged to her Last Berth to be broken up, 1838".
The 98-gun ship Temeraire played a distinguished role in Nelson's victory at the Battle of Trafalgar in 1805.
The ship remained in service until 1838, when she was decommissioned and towed from Sheerness to Rotherhithe to be broken up.
The artwork, painted by Joseph Mallord William Turner when he was in his 60s, is thought to represent the decline of Britain's naval power.
One of many works Turner bequeathed to the nation, the painting is currently hanging at National Gallery.
It attracted more than a quarter of the votes cast in the national survey - 31,892 out of a total 118,111 votes.
National Gallery director Charles Saumarez Smith said he was delighted with the outcome of the poll.
"I have been incredibly impressed by the amount of public interest it has generated."