I can't offer any back-to-back comparisons, but I always thought the concept had merit and finally got around to using one a year ago in my IHRA Stock engine. Some of my bracket engine pans had them integrated into kick-out pouches while others were used in regular-sided oil pans. I'm sure different engine designs will vary their effectiveness, but a scraper is going into my next engine also. A combo scraper/windage tray may be more effective, but that can take some careful fabrication. As with many performance mods, there are a lot of "it depends".
http://www.crank-scrapers.com/index2.html