You know if all else fails, you can go to your doctor and ask for Cialis, or order some generic online. I preferably would like to take Yohimbe, it really revs the engine for me, but when I was younger I had ED problems, similar to what you are talking about now, can get an erection but lack of drive, asked my doctor about Cialis, after taking it at least once a week for several months, I didn't need to take it anymore, it really restored my drive, I just managed to regain the feel of what its like, I used it as a way of recognizing what it was like to be in the mood, and tried to utilize my body to do it naturally.
"What about adrenal fatigue? Could that be responsible?"
You have no idea how many people brush off adrenal fatigue, if anything, just take 3-5g of Vitamin C per day (even as much as 10g), till you notice a difference.