But regardless, what's a good way to develop cool consequences, other than actual play?
This is an interesting question. During play there are definitely times when (as GM or not) I think "this should definitely require a roll, but wth happens if it's failed?" Sometimes the answer is "um no, then, it shouldn't require a roll, just say yes". But sometimes the answer is "your brain is on leave, there are tons of awesome failure consequences, you just didn't manage to think of any of them for some reason".
Players suggesting consequences can help. But due to groupthink there are still times when everyone knows there should be a roll and everyone knows they're failing to think up an awesome failure. What do we do about this? As I see it there are three options.
1) Say yes. The table does not have enough ranks in role playing to succeed every time.
2) Have some mechanical fallback for failure to be used only when the table fails to state some other failure consequence. This means if no consequence is stated in advance, and then the roll occurs, and then the GM fails to state a consequence, and then the GM + table fails to state a consequence. This shouldn't happen often at all, but when it does, it sticks out like a sore thumb.
3) Get better at consequences. Experience through play helps. The question is whether there is another way, a flash-card approach, other brain exercises, etc. that are actually worth doing.