A Live, (or Persistent), USB uses the FAT32 filesystem.
Windows can see and access files on a FAT32 partition just fine, (as long as it is the first partition on the drive).
Adding files to this partition will not break the Live install unless the Ubuntu files are deleted or overwritten.
If you need to access any of these stored file while booted from the USB, open filesystem/cdrom as root.
All of my USB drives have an install of Linux on them.
If you are talking about doing a Full install to the USB, use manual partitioning, (Something Other), and make the first partition on the drive FAT32 or leave it NTFS.