The problem with formica is its grooved nature. Applying wallpaper to it directly would result in a wall that, once touched the right way, would be bumpy and have holes in it. To alleviate this problem, you can either paint the formica, remove it, or, after some preparation, wallpaper over it.
Removing formica is really not an option for a do-it-yourselfer, as it requires the use of special tools and highly-flammable materials. The result of this removal will likely be an ugly wall covered in globs of the adhesive used to put the formica up anyway.
To wallpaper over the formica, first you must clean and sand it. Clean the surface with a TSP (trisodium phosphate) cleaner and water. Next, fill in and sand the surface; this will lower the bumps and create a smoother surface. It will also create grooves that the wallpaper glue can sit in. Making this work in a bathroom would be tricky due to the humidity.
Filling in the Grooves
You will need: a pail of ready mixed joint compound, a mud pan, and a 6″ spackle knife, and 80 grit sandpaper.
Fill the mud pan 1/2 way and load the knife edge with a 1″ ribbon size bead of the “mud”. Line up the knife parallel with the groove, cock it just a bit towards you so that it is on a slight angle and slide it down the groove.
You should end up with and area about 2′ long and about 2″ wide with mud over the grooved area. Take the clean knife and place it on the mudded area at the top so that the wide part is at a right angle to the groove and drag it down removing all the excess.
This is much faster than dabbing the mud over the joint in all directions and then removing the excess. Allow this to dry and sand it. Apply another coat and again sand when dry. Two applications should do the trick.
Priming the Surface
Prime the formica with a water- or oil- based priming agent. Water cleans up easily, is reasonably priced, and covers up to 500 square feet per gallon. Oil based primer should always be used if the surface is going to be exposed to water, like in a bathroom. To clean oil based products, use paint thinner or mineral spirits.
Once your priming is complete, you should apply a wallpaper liner. A wallpaper liner is a smooth surfaced, light- to heavy-weight paper like product that is used to hide the grooves in formica. It can be applied over textured walls prior to applying a smooth surface paper.
Now you may wallpaper over your formica wall in the usual way.