As someone in the field the last 20+ years I would put my stressors as follows:
(1) Docs are mostly employed these days and it is a tough place to be in. There's a profound loss of independence although you're the first one responsible if the proverbial crapola hits the fan. I have never had a malpractice case (humble brag alert!) but it's still always lurks in the background.
(2) Most of my difficult patient encounters take place with a small percentage of my patient panel (maybe 1-2%). However, in a system focused on pleasing people if you're not making 100% of them happy, you've got a problem. This has created a lot of stress where I want to do the right thing but feel pressure knowing the patient will probably complain if I do.
(3) Patient panels are insane. Where I work there is no limit to the amount of patients you can get on your panel. No limit! I'm in my third year at this job and still see 2 or more new patients per day. I've given up keeping track. The virtual patient care is one of the biggest burn outs of my job.
(4) The knowledge gap alluded to by Pinchaloaf is very valid. It's amazing in my field (primary care) how much this has exploded. I spend the equivalent of 1 workweek yearly in CME.
(5) Workload has been squeezed by efficiency experts to get you to bill the maximum while seeing the most patients you can per day. I'm under the gun because I insist most of my geriatric patients get 40 minute visits and it lowers my productivity. Many doctors see an excess of 20 patients per day and I can't see how you can deliver quality care like that.