The Bedford of Bunyan's time
An 1687 drawing of the bridge across the Ouse

In fact, Bedford was around a long time before then. See Medieval Bedford. If you want to find out more about the town, an excellent booklet to read is "The Beginnings of Bedford" by David and Evelyn Baker, published by the Bedfordshire County Planning Department.