Provide mice with foraging opportunities, ratrations.co.uk sell the Pick n Fly pack and the Harvest Festival pack, they also sell various millet sprays. Your local pet supply shop might also sell some of these things, they'll probably be in the bird section. Getzoo has a wide variety of different sprays too
Mice are great climbers so they really benefit from having the height of the cage filled with climbing opportunities like ropes, perches, hammocks, sputniks etc. They also really love tunnels on the ground (or cable-tied to the sides of the cage!) and plenty of houses. Also a good sized solid wheel like the 20cm Trixie one.