According to the Online Etymology Dictionary “Off the cuff (1938) is from the notion of speaking from notes written in haste on one’s shirt cuffs.” Source: http://www.etymonline.com/index.php?term=off
While Merriam Webster claims it means “without preparation” and that speaking off the cuff means “to ad lib”, they give no references to where this phrase came from. http://www.merriam-webster.com/dictionary/cuff
And the Free Dictionary lists several citations, including: off-the-cuff (adj.) meaning “Not prepared in advance; impromptu: an off-the-cuff remark.” Source: American Heritage® Dictionary of the English Language, Fifth Edition. Copyright © 2011
“off the cuff” means an informal remark, made without being prepared or thought about in advance. From: Collins English Dictionary – Complete and Unabridged, 12th Edition 2014 © HarperCollins Publishers, or….
off-the-cuff, adj. meaning with little or no preparation; impromptu.  Source: Random House Kernerman Webster’s College Dictionary, © 2010 K Dictionaries Ltd. http://www.thefreedictionary.com/off-the-cuff