From "das Zny", his first bottled specimens.
I think the contamination problem is coming from the cloth identification tag. As he points out, bacterial growth inside a specimen intended for show isn't necessarily a bad thing. Creatures made from polymer clay will shrug it off, but any acrylic paints applied to the creature will eventually be destroyed.
The best way to prevent contamination is to give the jar and anything going inside a good washing with hot soapy water. Follow that up by adding a few grains of sodium metabisulfite to the preservative fluid and you'll have a sterile environment that won't support any unwanted growth. It's safe, easy to use, and you can pick it up on Amazon for pocket change. A one pound supply will be enough for a lifetime's worth of bottled specimen props.