That's a loaded question...depends on the quality of your component cables, your specific HDTV and, to be quite honest, your eyes and your interpretation of the term Big Difference. As a rule of thumb, DVI/HDMI cables are best used with a digital set (LCD/DLP) and component work better with analog sets (CRT RPTV). Personally, I don't notice any difference between HDMI and a quality set of component cables on my Sony GWIV - quality component cables look slightly better over a cheaper (freebie Adelphia/Dish/VOOM) set of component cables on the Sony GWIV - however, a cheap set of component cables look horrible on my Mits CRT RPTV (quality component cables are a must).
I would recommend buying quality component cables and testing it for yourself.