It is still unknown when the city was founded. Though the town was not attested before 1174, municipal authorities chose to celebrate the town's 1100th anniversary in 1999, for reasons unknown. The settlement is indeed an old one, as archeologists unearthed a Paleolithic dwelling as well as Scythian remains within the city limits.

The present name of the city is traditionally connected to the settlement Ltava which is mentioned in the Hypatian Chronicle in 1174. The region belonged to the Grand Duchy of Lithuania from the 14th century. The Polish administration took over in 1569. In 1648 Poltava was captured by the Ruthenian-Polish magnate Jeremi Wiśniowiecki (1612–51). Poltava was the base of a distinguished regiment of the Ukrainian Cossacks and served as a Cossack stronghold during the Khmelnytsky Uprising. After the pro-Polish hetman Ivan Vyhovsky came to power and a civil war broke out, Poltava in the year 1658 under polkovnyk Martyn Pushkar was the leading town of the rebells, but was ultimately burned down and pillaged by the troops of Vyhovsky while many of its women and children were enslaved by the Crimean Tatars. In 1667 the town passed to the Russian Empire.

In the Battle of Poltava on June 27–28, 1709 (Old Style), or 8 July (New Style), tsar Peter the First, commanding 53,000 troops, defeated a Swedish army of 19,000 troops led by Field Marshal Carl Gustav Rehnskiöld (who had received the command of the army after the wounding of the Swedish king Charles XII on June 17). The battle marked the end of Sweden as a great power and the rise of Russia as one.

In 1775, Poltava's Monastery of the Exaltation of the Cross  became the seat of bishops of the newly created Eparchy (Diocese) of Slaviansk and Kherson. This large new diocese included the lands of the Novorossiya Governorate and Azov Governorate north of the Black Sea. Since much of that area had been only recently conquered by Russia from the Ottoman Empire, and a large number of Orthodox Greek settlers had been invited to settle in the region, the Imperial Government picked a renowned Greek scholar, Eugenios Voulgaris to preside over the new diocese. After his retirement in 1779, he was replaced by another Greek theologian, Nikephoros Theotokis.

In World War II, after the Red Army had cleared the Wehrmacht out of the Eastern Ukraine by the end of 1943 during the Dnieper–Carpathian Offensive, by the summer of 1944 the allied USAAF conducted a number of shuttle bombing raids against the Third Reich under the name of Operation Frantic, and used purpose-built bases in the Poltava area, as well as near Myrgorod, as eastern locations for landing B-17 Flying Fortress heavy bombers involved in those operations.