Short term vs. long term
As an author, many opportunities will present themselves, and within them, negotiations will occur. Speaking opportunities will be provided with trade-offs for fees: exposure, book sales, etc. Some will demand big payments right off the bat. Others will build resources and audience through relationship development. The point is to get your idea out there, so that it can grow to higher and higher levels, more opportunities, higher fees, and widespread credibility.
Geno Church wrote today about the Spinal Tap clip where the guitarist shows off his amp that goes to a volume level of "11." Most amps, as you may or may not know, only go to "10." When asked about why he doesn't just turn his regular amp to "9" and when he needs the extra boost, then turn it to "10," he replies, "But this one goes to "11." The Spinal Tap guy wants something bigger right now. I think many authors feel the same way.
Short term high cash vs. long term raving fans is an equation you'll have to solve on your own. Think about the consequences of both routes.