Looks like a mismatch between the STB output and the TV input. In the television production world, we would see something similar if the composite input to a monitor or other device was not terminated. Your TV probably doesn't have a termination selection switch, but the picture sure looks like it is seeing the too high of signal. The TV may be expecting lower amplitude signal and the STBs might be providing too high of signal for the TV to handle.
Are you using the same AV RCA breakout to connect each STB to the TV? Might be a defective cable or plug. Swap to a different cable?
Did you check the same TV input with a DVD player or other video source as KE4EST suggested?
Reduce the menu setting for the video output?
Install a resistor across the pin/shield (unknown value) would match the two components?