"I am not afraid of ghosts," I said, when I took up the offer to spend a night in haunted Hall Place, one of the oldest buildings in Bexley. If you ask me now, I wouldn't reply so promptly.

Sixteenth-century Hall Place is the perfect setting for a gothic tale. It has a great hall, a minstrel's gallery, long corridors, wood-panelled chambers, rambling attics, dark cellars, rumours of secret passages, and a tower in which the room at the top has been sealed off and the spiral staircase removed by a former inhabitant "to stop the ghost coming down".

This, it seems, did not work. Only recently, a visitor to Hall Place reported seeing a woman in white, archaic costume, by the window of the great hall. The White Lady of the Tower has been seen usually wandering near the tower, weeping and wringing her hands. The story goes that a young damsel was at the top of the tower when she saw her husband gored to death by a stag he was hunting. She died of grief.

The Black Prince is also said to haunt the halls, and seeing him bodes doom and disaster. Other ghosts include a servant girl who searches the attics for a lost child, and spectral music in the minstrel's gallery. The last resident of Hall Place was socialite Lady Limerick, who lived there until her death in 1943. She is said to have had an agreement with the local constabulary and would phone at night "when the ghosts became too troublesome" to ask for a policeman to stand guard.

The building is now run by Bexley Heritage Trust and is open to the public Monday to Saturday from 10am to 4.15pm, with exhibitions of art and local history displays.

I had agreed to meet the caretaker, Steve Tucker, at The Jacobean Barn at 10pm. The Jacobean Barn was once an outbuilding of the estate, but is now run by Beefeater as a pub and restaurant. Arriving early, I asked the bar staff if they had seen anything spooky. They seemed reluctant to talk, but one waitress, who did not want to be named, said: "One of the girls saw a figure in white, but she left after that. None of the staff will work here on their own late at night."

When Steve arrived, he unlocked the tall, forbidding gates and we made our way down the long, dark drive towards Hall Place.

He took me on a guided tour and, although I was impressed with his immense knowledge of the estate and its history, I couldn't help but be slightly suspicious of his dedication to the building which meant he was willing to stay up all night to take care of it and keep an eye on one nosy journalist. "I often work here late, after dark," he said. "I don't go out much any more."

Wandering around an old building late at night, exploring shadowy attics and cellars, looking into rooms normally closed to visitors and peering by torchlight into the empty stairwell of the old tower is a little creepy, and I was quite pleased when Tim, the photographer, stopped by. But, when midnight struck, I was alone once more with the hall's caretaker and, perhaps, the spirits of dead souls.

As I waited in the great hall, I saw shapes in the shadows, movement in dark corners and odd reflections in the glass windows. But it was the strange noises which were most disturbing. There was a thud as if a vast door had slammed although, at the time, both Steve and I were in each other's sight.

Then there were sounds like soft footfalls which started in the area of the tower, then moved across the creaky boards of the attic. It may have been the sound of timbers creaking as the temperature dropped. Perhaps it was Steve playing some horrible practical joke. Or, perhaps, it was the ghosts...