the normal version of the USB images we have are not persistent. Basically, everything you do is only for that session. Maybe you change the keyboard layout or install another app, next time you boot to the USB’s live session, those changes will be gone.
A persistent USB acts more like a normal installation that you would have on your hard drive. When you change something, you can expect it to be persistent across boots.