Too tame a life, perhaps that is it. I would rather have the life of Sir Richard Burton, or someone like that. He, too, was a scholar.
Notice, though, the vocabulary of debt and obligation that goes with academia. You did a Ph.D. People say the price you pay is to teach. You owe the field another book. If you do other things, you have abandoned your companions, including those less fortunate than you, for whose sake you must keep that tenure track job since you, at least, got one.
It is a strange set of attitudes and I would rather be breaking through underbrush with a forked stick, or peering around big leaves.