Once the mildew forms it is like the mildew on tile and can only be removed by strong chemical action combined with scrubbing.  Mildew is a fungus.  It is in the air and only finds a place to grow where it is warm and constantly damp.  Just like a shower stall, conditions have to be right for the mildew to grow in the first place.  And just like a shower stall, you will have to use a strong fungicide (like Tilex or CLR) and a scrub brush in order to remove it.  Considering the nooks and crannies in the foam surface that is going to be a difficult job at best. Plus the fungicide will oxidize the polyethylene making it brittle and scratchy and eventually the surfaces will slough off.

 

Probably the best solution is to replace them and change the environment so this does not occur again.  The most important criteria is for them to dry out completely and as fast as possible.  The longer it takes to dry, the more likely mildew will form.  Store the hand buoys where they can drain completely and air can swirl around them.  Take them out of the pool area.  Perhaps keep them in another room that is air conditioned or open to outside breezes.  Do not put them in a closet next to the pool.