When William Fitz-Osbern erected a castle on a low promontory at Wigmore, shortly after the Conquest – part of his mandate to establish a chain of fortifications along his part of the Welsh border – the site was said to be waste land. A couple of miles away on either side were the Lugg and Teme rivers, which meant the castle was well-placed to control the area between them – much of it marshy moorland (the 'more' in Wigmore likely referring to mere or marsh) – and it was also in a fairly central position for military expeditions into Wales. The Anglo-Saxon chronicle entry for 921 has a reference to a fortification created by Edward the Elder at Wigingamere, and a Bury Farm and Bury Lane on the eastern edge of Wigmore might remember such a feature; however, prevalent opinion is that this burh was in a more easterly region of England.

By 1086, when the castle was in the hands of Ralph de Mortimer (ca.1055-1137, although possible there was a second Ralph after 1104), a borough had been established and a church built around the same time (the nave retaining some of its late eleventh century fabric). Ralph's son Hugh founded Wigmore Abbey about 1179, a couple of years before his death, and his son completed the construction of the abbey, a mile north of the town; it became the burial place for many members of the family. The castle, rebuilt more substantially around 1140 on an adjacent site, became the base of the Marcher lordship of Wigmore, cemented by Roger II de Mortimer (great-great-grandson of Ralph) in mid-thirteenth century, thanks in part to royal rewards for Roger's key support during the war with the Montfortians. Roger III de Mortimer, whose leadership of the opponents of Edward II led to the latter's deposition and Roger's elevation to the Earldom of March, redeveloped the castle on a grander scale, so that it became one of the strongest in England. However, after his fall from power, the family transferred its administrative base to Ludlow Castle, which had come to Roger through his wife; Wigmore's castle, although it remained the official seat of the barony, thereafter slowly deteriorated; it could still be used as a prison in the sixteenth century, but only after repairs were undertaken, and it certainly had no military value.

The Mortimers, who took their name from Mortemer in Normandy, were one of the more influential families of the English nobility from the time of the Conquest – they having been kin both to William Fitz-Osbern, the Conqueror's first Earl of Hereford, and to the Conqueror himself – and throughout much of the remainder of the Middle Ages. The Ralph de Mortimer mentioned above held a number of manors in Shropshire as sub-tenant of its earl, Roger de Montgomery, and he served as the earl's steward. After Fitz-Osbern's son and successor Roger de Breteuil, was exiled (1075) for rebellion, the king looked to Ralph to act as protector of royal interests in that region and he was given Roger's lands in Herefordshire and Shropshire, including Wigmore and Cleobury. Ralph profited in similar fashion from the rebellion of Robert de Bellesme, Earl of Shrewsbury in 1102. These gains he expanded somewhat by conquests in what would later be Radnorshire, though taking and holding were two different matters. Although the Mortimer power-base was the Welsh Marches, this sprawling family, whose various branches can sometimes only be tied together through speculation, held lands across England, generating several baronies, some built on marriage alliances with heiresses of other prominent families, Welsh and English, such as the Braose.

No record has survived of market or fair licences, and no licences would have been necessary, given the antiquity of the borough and its descent through a single family across the medieval period. As far as our limited evidence indicates, the market town of Wigmore was doing well enough at the opening of the fourteenth century, though its population does not seem to have expanded greatly. The inquisition post mortem on Edmund de Mortimer (son of Roger II) in 1304 identifies 140 burgages in Wigmore, as well as mentioning its Wednesday market and July fair. Curiously, the Domesday value of the borough at Wigmore was £7, which would translate to 140 one-shilling burgage rents, though we cannot be certain that rents were the only component of the Domesday valuation. Of course, sub-division of burgages may well have increased the number of residents somewhat by 1304, but the extent of the settlement does not appear to have spread out much during the post-Conquest centuries. By the close of the fifteenth century things had changed, in terms of prosperity and commercial competitiveness (notably with Ludlow and Leominster); Wigmore's decline paralleled that of its castle, and in the post-medieval period it lost both its market and its urban status.

Wigmore is situated on one end of a long spur oriented roughly north-west to south-east, settlement extending into a small valley to the south and bounded by the brook running through that valley. The Mortimer castle site is on a low hill at the north-west end of the settlement itself. But another site with a motte is east of it and much closer to the churchyard; this is suspected to represent the original castle put up by Fitz-Osbern. The church stands just west of the crossroads of a north-west/south-east through-road (connecting to Leominster), and a road with a more easterly approach which passes the south side of the churchyard and originally continued to the earlier castle site, but was later redirected to proceed to the later castle site further west. The crossroads itself, widened into a rectangular area, must have served as the marketplace. Around this and along either side of the through-road south of the crossroads are most tenement plots; the through-road may therefore be considered the axial route of the town. Regularized, narrow tenement plots are discernible particularly on the east side of the through-road south of the marketplace, a stretch now known as Broad Street, and on the south side of the street to the castle, with the brook marking a service lane at the rear of the latter. Foundations of medieval buildings have been identified: one beside that part of Castle Street approaching the later castle, a second facing onto the west side of the through-road just south of the marketplace, and a third on the east side of the through-road just north of the marketplace. The extent of town planning is less clear, however; certainly what we see in the topography cannot support the claim that has been made for a grid-pattern layout.