Principle thing I've heard about pen drives is the data getting corrupted but this is at the time of writing to them not later so as long as you check the data is OK it should be fine - I recommend removing and then testing on another computer for critical data.
This primarily happens if you don't cleanly unmount the volume. Never unplug a USB mass storage device from a machine if you have just written to it, unless you unmount it cleanly (by running 'unmount /path/to/mountpoint' in a shell, right clicking on it and pressing 'eject' in nautilus or similar, or using Windows' unmount removable device option). Basically, you can't guarantee that writes have been committed to disk when the copy command completes. Disk cache and so on mean that there's still quite a bit going on. You can run 'sync' a few times on the command line to check that all pending writes have been written. This was especially bad on USB1.x devices. USB2, having significantly faster throughput available, isn't as bad, but it can still happen.