I went from 6s to X a year ago.
Positives: noticeably faster, much better camera, wireless charging, water resistance, somewhat bigger screen, Face ID that works well.
Negatives: the notch, missing home button, no finger print reader, different UI, no audio jack, much heavier, very pricey.
How significant those factors are, that's up to each user. For me personally, surprisingly, the biggest negative is the extra weight. A year later, I am still reminded about it every time I grab the phone or put it in my pocket.
The missing audio jack is an inconvenience, especially when you use multiple headphones. That tiny adapter is not just a connector, but it actually is your DAC. And the quality of the DAC is important. The Lightning port doesn't have analog outputs. So if you want high-quality audio you need to buy a high-quality adapter/DAC. You can't use iPhone's built-in DAC anymore.