Here's a useful old chant, charm, spell, prayer (you name it!) to recite if you hear a bump in the night! I've seen this poem in so many different versions that I can't credit the author here. But give it a try .... and best to say it three times!
Ghoulies and Ghosties: An old spell
From ghoulies and ghosties,
And long-leggity beasties,
And things that go bump in the night.
Good Lord deliver us!