Situated on raised ground, sometimes described as a cliff, just beyond the east bank of the looping River Wye, Ross was close to a key river crossing at Wilton, a short distance west; Wilton did not attract as much settlement because its low-lying site was vulnerable to flooding. Ross was bounded to north and east by brooks feeding into the Wye; its name derives from a Welsh term for a promontory. The manor of Ross, as it was simply called during the Middle Ages (the 'on-Wye' being added as a differentiator and tourism marketing device in the nineteenth century), was part of the estates of Hereford cathedral chapter at the time of Domesday, although it is mentioned in a cathedral document in 1016. The Domesday entry evidences a village, with the usual mill, while a church may be inferred from the presence of a priest. The bishop had by 1166 a residence (manor-house), on the west side of the churchyard, and the bishopric's dungeon for criminous clerics was established in its basement. The present church is a thirteenth-century build, but incorporating some material from the previous century.

In 1138 Bishop Robert de Bethune acquired licence for a Thursday market at Ross, at the same time that he had from King Stephen a market for Ledbury; Bishop Hugh de Mapenor obtained grant of a second market day (Tuesday) in 1218. Henry III reissued the 1138 licence to Bishop Peter of Aigueblanche in 1241 and at the same time granted a July fair; since there was no evident legal need to re-license the market, it is likely that the fair grant was the main purpose of Peter's transaction, and opportunity was taken simply to obtain confirmation of the market – a shrewd move given the quo warranto proceedings that would take place later in the century, and at which Bishop Swinfield would be called to answer for various manorial rights at Ross and the other boroughs under cathedral lordship, such as administration of the assize of bread and ale, but not for the market. Eyton incorrectly assumed that three other fairs also held at Ross in the post-medieval period were likewise granted by Henry III; similarly, his statement [vol.3, p.100] that Henry gave liber burgus status to Ross may be an assumption from later evidence of borough status, which includes summoning of two representatives to the parliament of 1305.

The church and adjacent episcopal palace stood at the southern end of medieval Ross, for that was the highest ground. From the north side of the palace what was later known as Edde Cross Street headed north along a downwards slope. The marketplace, north-east of the churchyard, forms a triangular space at the junction of the later High Street – part of which appears to outline the Domesday settlement, skirting the palace and churchyard before continuing eastwards beyond the marketplace as the road to Gloucester – with a north-south through-road; the market overflowed into the initial part of the descending northern stretch of that road, known as Broad Street. The planned town seems to have been laid out around the marketplace and up Broad Street; here the long plots with narrow ends fronting on the street have the clearest look of a burgage layout. What appears to be later expansion occurred along all three streets leading to the marketplace as well as along New Street and Kyrle Street, which run westwards off Broad Street, in the direction of the river, to connect to Edde Cross Street, where burgage plots may also have been the product of an expansion phase; the Broad Street ends of New and Kyrle Streets narrowed as they were fitted into an existing layout of burgage plots. Market activity then spread down these additional streets, perhaps in the form of commodity specialization; the base of a medieval market cross was uncovered at the junction of New Street and Edde Cross Street.

In 1205 Henry de Longchamps attempted to tap into Ross commerce by obtaining licence for a Tuesday market at Wilton, about half a mile away, on the other side of the Wye. Henry I had granted the manor of Wilton to Hugh de Longchamp and it was likely he who put up a castle there, overlooking the river. Tt is hard to imagine that the Wilton market would not have provoked an objection from the lord of Ross, though no record has survived of one. But some form of extended competition is suggested by, first, the 1218 grant of a Tuesday market to Ross, then in 1231 another grant to Henry de Longchamps of Wilton's Tuesday market, supplemented by an August fair; whether the market grant was just a confirmation in the face of competition from Ross, or a revival following a conceivable cancellation of the original market, we can only speculate.

In 1257 the licence for market and fair was renewed by Reginald de Grey, who had before 1252, when still a minor, married a daughter and heiress of the lord of Wilton, though sources differ as to whether that lord was Henry de Longchamps or the husband of Longchamp's daughter (it being beyond the scope of this study to delve into the confused Grey family geneaology). Reginald's father John, whose family was based in Essex, had served as sheriff of Bedfordshire and Buckinghamshire in 1238/39 and was sheriff of Herefordshire 1252/53. It was presumably because he was not descended from the Longchamps family, that Reginald found it necessary to renew the licence, and he made a further attempt to strengthen Wilton's commerce by establishing a town there. But apparently it proved unable to out-compete Ross and nothing further is heard of this town – the inquisition post mortem on John de Grey in 1324, for instance, gives no hint in its description of the manor either of burgesses or of a market – though the Greys remained barons of Wilton into the Tudor period.

An extent made in 1404 of bishopric properties in Herefordshire found that the borough of Ross yielded annually about 10s. in tolls from its Thursday market, and the same amount from its fair tolls. Another indication that Ross was prospering – if not as much as Ledbury or Bromyard (which probably had much larger populations) – is seen in the survey of episcopal lands in 1288, which records 96 burgesses and 9 selds in Ross, some of the selds being held by a fuller, a goldsmith, and Adam le Mercer. Adam was one of the two townsmen at the only parliament to which Ross sent representatives. That in 1305 Ross sought exemption from attendance, on the grounds it could not afford the daily expenses due representatives, is not necessarily a sign of declining fortunes, but over the course of the following century the same adversities affecting other towns in the county – bad harvests, plague, and attacks by the Welsh – must have taken a toll. Leland found the episcopal palace ruinous, and it probably ceased to have been used for its original purposes of manorial administration and imprisonment after a rationalization of diocesan property in 1356. Despite these things, Ross-on-Wye was one of the nine market towns in the county to continue functioning as such by 1500. It had little competition, as a market for grain, wool, and livestock, in the southern part of Herefordshire, yet lay at a site crossed by a north-south route along the Wye valley, an east-west route connecting central and southern England with South Wales, and a regional road between Hereford and Gloucester.