Filling the pole with concrete will definatley help. Have a 2.7M solid with quite a heavy duty mount sitting on a pole about the same height as yours. (The pole is 89mm. 3 and 1/2" I think, sorry metric brain here) I just poured the the concrete in litte by little and rodding as I went. Also when I first concreted the pole in the ground made a rock can around the base of the pole about 2' or high which looks quite attractive. The cement holding the rocks together and the cement in the ground are as one. As I did it at the same time.
Havenx's idea has a lot of merit as with time the ground can shift. The guy wires with bottle screws could be used to keep the thing plumb.
Anyway, this thing of mine has been though a cyclone/hurrican with no movment.