I would say the PQ between the Roku and PS3 is comparable, however, as my PS3 is hooked up to better TV (46" Panny 800u plasma vs. a 32" 720p Samsung LCD), I give the edge to the PS3.
I've also noticed when I hit the Display button on my BD remote, the bitstream rate hovers between from 3.0 to 5.0 mbps. On my Roku the highest bitsteam possible for HD content from Netflix is 3.8 mbps. I don't know if this makes a difference PQ-wise or it's just a factor of how the PS3 receives and buffers the stream.
I can't speak on the PS3, but I do have the XBox360 and Roku to compare against. Watching the bandwidth usage on my router between the 360 and the Roku it looks like the 360 pulls more bandwidth, so I'm not sure if there are different streams for the Roku and 360. From what I remember hearing from the Roku guys they use a slightly different codec stream for the HD...not sure if it is good or bad. It is hard to tell though on the screen between the two. Bandwidth wise it is hard to tell on the router as they grab the movie in chunks, and both have different size buffers.
I can tell you this though, I prefer the size of the Roku, power consumption, and there's no fan on the Roku, not that I hear the 360 since it is in the other room. Another big plus for the Roku is what is coming to the Roku in the next couple of months. Lots of screen shots floating around the web with Pandora, MediaFly, Revision3, etc. shown on the screen.