Have you tried sanding it? My guess is you want a smooth surface in order to place down new tile. Sanding it should work (especially if you have a professional sander).

There are some instructions on this site, but it for concrete:
http://www.parish-supply.com/mastic_removal.htm

Another site suggests: ""Bob Harris, president of the Decorative Concrete Institute and author of Bob Harris’ Guide to Stained Concrete Interior Floors, says caulking compound and mastic are two of the most difficult substances to remove. He scrapes off as much material as possible using a putty knife or floor scrapper and then applies a poultice to remove the remainder. For a poultice, he recommends mixing an inert fine powder, such as fly ash or hydrated lime, with denatured alcohol to make a smooth paste. Once the poultice dries, the caulk or mastic residue usually is brittle enough to remove with a stiff-bristle brush."
http://www.askmehelpdesk.com/advice/t-24515.html

Apparently, you can also buy mastic removers: http://www.franmar.com/mastic_tip_sheet.html

If the mastic is the gooey black stuff that looks like tar then I would use a propane torch and a scraper (Putty knife) to scrape it up. A little acetone should cut it too for a final wipe, and cleaning your tools. This kind of mastic is usually under linoleum tiles, and will clog sanders.

If it is the yellow mastic then I would take a floor scraper, which it the kind on a broom handle, and scrape away as much as you can, and then take a putty knife and run it along the ridges and take them down as far as you can. Then I would sand the rest with a palm or belt sander till it is flat, but be careful not to take anything but the mastic off the floor. If you do just use some self leveling compound and float the floor prior to installing the new tile.

You can also just use a little leveling compound over the mastic, and then screw down quarter inch Hardibacker into the plywood floor and re-tile right to the Hardibacker, if it does not raise the height of your floor to an undesirable level.

Hmm, the best way, and this is from experience, is to hire a high school kid. My experience is from the perspective of the high school kid...

The other option is use a sander. Or a pry-bar and hammer as a chisel. And then some leveling compound.

If the mastic is only in specific spots, you'll be better off with a mastic remover than sanding--as with sanding you run the risk of creating an "uneven" floor. Depending on what you're doing next--this may or may not be an issue.

Be careful when selecting a mastic remover, however, as some are designed specifically for concrete, and may damage wood.