That's right, sanding is putting small scratches in the wood, a scraper will leave a perfectly smooth surface - makes a big difference when it's polished.
The problem with scrapers is that they are flexed when scraping (generally), so they leave slightly concave scrape marks. Not usually that big a deal, but depending on the wood, and more particularly, grain orientation, might be seen. For 90% of jobs, I sand to 320 grit, varnish, and sand between coats to 4/500 grit. Wipe off the residue. 3 coats, jobza. The other 10% is usually looking for some kind of rougher finish anyway, and is generally oiled and/or waxed.