12th-14th centuries

The first manor house was built on the site sometime between 1199 and 1216 (during the reign of King John)^[2] and the site was owned by William de Rous during the 1220s. It first appears in records in 1275,^[3] when it was owned by William de Morteyn, who gave it to his son Roger when he died in 1283. When Roger died, the castle was passed down to the Baron Bassets of Drayton until the barony went extinct in 1390.^[3][4] During this time, the manor house was built within two phases of construction, with a moat being added later.^[1] Also at this time, part of the building was used for metal-working.^[1]

Walsall Castle was owned by Ralph Basset, 3rd Baron Basset of Drayton during the late 14^th century and the manor house, then known as Basset's "new castle", was rebuilt between 1385-86,^[4] likely on the site of the 12^th-13^th century building, and it was repaired and extended between 1388-89.^[4] It was repaired again during the 1390s, after Basset's death, with a new chamber being built. Lady Neville stayed at Walsall Castle in 1385-86 with her two children while Ralph Basset owned the manor house.^[1]

15th-21st centuries

The chapel was no longer in use by 1417,^[2] the manor house was abandoned during the 1430s and all traces of Ralph Basset's manor house had been exterminated by 1576.^[2] A Mr. Holmes purchased the site in 1763 and had two houses built on the site; the houses built by Holmes were abandoned by the 19^th century and in 1885, the northern section of the moat was filled in but the rest of the moat still contained water until at least 1974.^[2] By the 20^th century, no visible remains were left of Walsall Castle and the parking lot for Walsall Manor Hospital was eventually built over the site, but since the 1970s, archaeological excavations have been undertaken at the site by the Walsall Local History Society.^[2]