Bees and wasps usually use unoccupied houses, but it’s best not to spray the house with any poisons. If you have one of our Wild Birds Unlimited easy clean out houses, I would leave the door open so there is a lot of light in the box. If there is no easy clean out you can plug the hole for a couple days until they get the idea. If they have already established a nest it is best to let them be and not take any active measures to exterminate them. Instead, wait to clean them out in the fall when the weather is cooler and their activity has halted.
To stop this from happening again, rub a thin layer of bar soap on the inside surface of the roof. The slippery surface prevents the insects from attaching the nest to the wood. Don’t use any oils like Crisco which could melt and get on bird feathers. A little Ivory Bar Soap rubbed on the inside of the birdhouse roof doesn’t hurt the birds and deters any insects from attaching a nest inside. Good luck!