I think the worst negative side effect of cannabis use is complacency. And with complaceny comes dependence. It affects all users to some extent.
I speak here as a reformed "addict" and many years removed from my last consumption. In my own situation I can see how complacency put my career on hold, as well as my quest to be financially independent. One can weigh for him/herself which is more important, but as for me I would feel a whole lot better now knowing that I had made myself independent and set the stage for better opportunities and economic security for my heirs and assigns if I had never imbibed in the first place for such short-lived gratification. If only I knew then what I know now...
I think back to the Vietnam war. I did not serve, but IIRC our enemy helped make drugs including cannabis readily available to our soldiers in harm's way...to make them complacent and take the fight out of them. Did that have any impact on the outcome...?
America did not become the world leader it once was on complacency. But complacency may very well hasten our current decline...