One of the smartest things you can do is to be ready to write down or audio tape things the audience yells out to you when you are having fun with them. These lines can be used by you in future speeches.
One of my stories is about a riches to rags story that I lived. I build up the story of how I lost a whole bunch of money in a failed business and then I injured my leg and was bedridden, and then my girlfriend left me and I finish that part of the story saying, "and my dog got run over." One time when I told the story someone yelled out from the crowd something about the dog went onto write a country music song which obviously referred to all the doom and gloom I was suffering. This got a BIG laugh.
In future speeches, to relieve the tension of people thinking that the dog got killed, I used the line, "Don't worry he's OK and went on to write a country music hit" which always gets a good laugh.
Always be ready for this when you have a really fun loving crowd. One way to be sure that you remember the line after the speech is over is to stop right then and jot the note down while saying something to the person yelling out the line, "Would you repeat that because it's funnier than my line / story etc." which will get a laugh and make the audience member the star.
This is really a win win because you really do need to write things down when they are fresh because you are likely to forget them after the speech (especially if you are not audio taping) and also because you get an interaction with the audience and another laugh line out of the situation.