Quite possible that you actually DO have 25%. Not trying to be mean or anything, but the average BF% in the country is something like 23-27%, for males.
If you LOOK lean, that's what I'd worry about. I have the opposite problem, see. I used to be exceptionally fat (try 35% on for size), and now when I look in the mirror (even when I was down to 9% last fall) I still see that 35% fat, regardless of how much I actually carry. In your case, I'd worry more about what you see when you look in the mirror, whereas I'm exactly opposite - I need to see numbers because I can't see myself.
All in all, don't worry about it, just keep up the diet and cardio.