Lydham lies in the valley of the Onny River, a tributary of the Teme, with hills to north and south and Bishop's Castle situated in the latter, two miles away. Lydham manor, with its church-focused village, was part of the extensive estates held by Earl Roger de Montgomery at the time of Domesday. Domesday does not refer to the church, but mentions a priest, along with the village reeve – one of only a handful of whom Domesday takes notice – which could suggest some importance to the village, held before the Conquest by a nephew of a Mercian ealdorman. Any such importance may derive from Lydham's position on a natural east-west route through the valley, whose existence prompted settlement in the area since prehistoric times. Also mentioned in Domesday is a mill, presumably that (close to the castle) which was powered by a channel cut from the Onny to flow into Lydham, while a further channel cut from this provided a partial moat for the bailey of the castle put up, at date unknown, west of the church.

After the rebellion of Robert de Bellesme, Henry I annexed Lydham – except for one portion named More, which was held by a long-lasting family taking that name – to the Honour of Montgomery, so that it passed through the hands of the various lords of that honour; two of whom, William de Cantilupe father and son, between 1227 and 1245, founded markets and/or fairs at five locations in south-west England and the Midlands, including the existing borough of Stafford, and the emerging borough of Bromsgrove (Worcs.), though they did not attempt the same at Lydham, perhaps because their title to it was not secure. After the death of the younger William (1254) the king had his escheator take Lydham into Crown custody; following that, it was first in the hands of farmers, then those of Prince Edward, as Lord of Montgomery.

In 1265, while gathering his forces for Evesham, Prince Edward granted the manor to Adam de Mongomery as a reward, or incentive, for faithful service; Adam was not a descendant of the lords of the honour, but, as Adam Fitz-Philip, was castellan of Montgomery castle; in that capacity he had successfully held the castle against Montfort's nominee for the post. In further gratitude for Adam's loyalty during the civil war, Henry III in 1267 issued him a licence for a Friday market and early May fair at Lydham manor, and then in July 1270 a second licence for a Wednesday market and four-day fairs in late May and October, with the specification they be held in the town; the second licence was a component of a charter whose principal purpose was to authorize Adam to accord liber burgus status to his vill of Lydham, and to its men exemption from tolls throughout the realm. From the context of this in the Charter Roll, we may suspect Adam's application for this charter was sponsored by Prince Edward, who was in the process of rewarding various of his supporters in the Marches. At assizes held in 1272 the jurors of Purslow Hundred presented the Friday market at Lydham and a July fair, on the grounds that they were ignorant of any legal right underlying them. Although the Calendar of Charter Rolls abstract of the 1267 grant refers only to a May fair, David Preshous ["Lydham: Early history of a cross-border parish", South West Shropshire Historical and Archaeological Society Journal, no.19 (2008), p.13] states that a July fair was granted on the same occasion, though he may only be following Eyton's statement to that effect. If an investigation followed, Adam would presumably have been able to cite the grant to exonerate himself. Lydham was the only manor he strove to develop into a market town, although he held lands elsewhere in the county.

Lydham's church was situated on the west side of a north-south through-road, which connected to Shrewsbury and Clun, and off which forked a branch to Ludlow. Location of the marketplace is unknown, though it might have been somewhere along the stretch of the road in front of, or just beyond, the churchyard – perhaps at the spot, south of the churchyard, of the fork in the through-road. Facing competition from the markets of Bishop's Castle and Montgomery, and with lack of good lordship – after Adam's death his son Thomas attempted to sell the manor to Roger de la More, but a drawn-out legal battle ensued with a third party claiming a share in the manor – Lydham failed to prosper and may have reverted to village status as early as the fourteenth century. If a planned town were ever laid out it would probably have been close to the castle and the church, but the early depopulation of the borough has resulted in no vestiges of burgage plots being evident today, so it is possible (though unlikely) that any new town was founded at some site beyond the village.