In Linux, we can do
dd if=/dev/sdb of=bckup.img
but if the disk is of 32GB with only 4GB used, the 32GB image file is waste of space-time. Is there any way or tool to create images with only valid data?
![](data:image/svg+xml,<svg xmlns="http://www.w3.org/2000/svg" width="32" height="32"><rect fill-opacity="0"/></svg>){kind=small}
Pretty good and simple way to deal with this is simply pipe it via gzip, something like this:
# dd if=/dev/sdb | gzip > backup.img.gz
This way your image will be compressed and most likely unused space will be squeezed to almost nothing.
You would use this to restore such image back:
# cat backup.img.gz | gunzip | dd of=/dev/sdb
One note: if you had a lot of files which were recently deleted, image size may be still large (deleting file does not necessarily zeroes underlying sectors). You can wipe free space by creating and immediately deleting large file containing zeros:
# cd /media/flashdrive
# dd if=/dev/zero of=bigfile bs=1M # let it run and quit by disk full error
# rm bigfile
-
1
-
2and ur solution still writes 32GB of data to sd card. a very time consuming process. – Necktwi Oct 14 '13 at 7:41
-
2Well, you can simply
tar cvfz mybackup.tar.gz /media/flashdrive. It will capture everything except for bootable USB drives they will lose bootable status – mvp Oct 14 '13 at 7:41 -
2Yes,
tarwould lose MBR, and you will need to useboot-repairto make drive bootable again. But, other than that, tar should preserve everything and OS should be fully usable (but be sure to run tar as root!). – mvp Oct 14 '13 at 7:52 -
2You can usually zero out free space on modern sdcards using 'fstrim', this has the advantage of being many times faster, and also avoids flash wear.For fstrim support on VFAT partitions, you will need at least Linux kernel version 4.19. Unfortunately, many USB SD Card reader/writers don't support discard/trim. If you wish to try this, try
fstrim -v /path-to-mounted-filesystem/. I use this during production image writing on an embedded air quality sensor which I work on professionally. – Tim Small Jan 9 '19 at 12:24
The best thing to do is
Copy all the files from all the partitions preserving meta data
mkdir -p myimage/partition1mkdir myimage/partition2sudo cp -rf --preserve=all /media/mount_point_partition1/* myimage/partition1/sudo cp -rf --preserve=all /media/mount_point_partition2/* myimage/partition2/Extract the MBR
sudo dd if=/dev/sdX of=myimage/mbr.img bs=446 count=1replace
/dev/sdXwith the corresponding device.Partition the destination disk into partitions with sizes greater than copied data and should be of the same format and same flags using
gparted. Google how to partition a disk.Mount the freshly formatted and partitioned disk. On most computers, you just need to connect the disk and you can find the mounted partitions in
/mediafolder.Copy the previously copied data to destination partitions using following commands
sudo cp -rf --preserve=all myimage/partition1/* /media/mount_point_partition1/sudo cp -rf --preserve=all myimage/partition2/* /media/mount_point_partition2/Copy back the MBR
sudo dd if=myimage/mbr.img of=/dev/sdX bs=446 count=1
Now njoy Ur new disk!
-
1Has anyone tried this and confirm that it works? Otherwise, I would supply only one of
-ror-Rtocp: they are the same switch. – Jealie Apr 15 '15 at 17:08 -
1I've tried this and it worked. Made a clone of my raspberry pi's boot SD card. The card had 2 partitions, one fat and one ext4. I have dumped those with commands provided, then I dd'ed entire SD card with zeroes, created new partiion table, created my partitions which had different from previous parameters: size, spacimg, labels. And after that I restored it from dumps and it worked. – CoderFF Apr 28 at 18:01
Using the bs and count parameters of dd, you can limit the size of the image, as seen in step 2 of answer 1665017.
You may already know what size image you want to create. If not, you can get a good idea from df:
df -H --total /
Substitute / with a space-separated list of all the mount points relating to the disk partitions.
A more accurate way might be to use fdisk or your preferred partition editor and get busy with a calculator.
$ fdisk -l /dev/mmcblk0
Disk /dev/mmcblk0: 7.4 GiB, 7948206080 bytes, 15523840 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x00057540
Device Boot Start End Sectors Size Id Type
/dev/mmcblk0p1 2048 186367 184320 90M c W95 FAT32 (LBA)
/dev/mmcblk0p2 186368 3667967 3481600 1.7G 5 Extended
/dev/mmcblk0p5 188416 3667967 3479552 1.7G 83 Linux
Total used space in bytes = end sector of last partition X sector size (here that's 3667967 x 512).
Total used space in GB = total used space in bytes / 10243 (here that's 1.749023 GB).
If you decide, for example, that your image should be exactly 2 GB, the following command will do that:
dd if=/dev/mmcblk0 of=/path/to/pi_updated.img bs=1M count=2048
The resulting image will also include the random noise beyond the greatest extent of the last partition.
Don't forget to sudo the above commands if your account doesn't already have sufficient privileges.
For my purposes, I don't need an image that is perfectly trimmed down to the last bit of data so when the real size is 1.75 GB then a 2 GB image is near enough for me. This cuts out the other 6 GB (or 30 GB or whatever the device has spare) of unused space that would otherwise be in the image.
I have seen advice in many places that dd should not be performed on a mounted partition and I followed that because it seems intuitively correct; it does seem rather like trying to sketch yourself making a sketch in a mirror with the sketch you're making also visible in the sketch. It's a bit sketchy.
-
2IMHO, while calculating, the sector size should be based one End+1 instead of End. In your case, it should be 3667968. – scegg Nov 15 '16 at 7:12
If you have a big SD card 16 GB, 32 GB etc but you want to save space with backup you can use:
sudo apt-get install gnome-disk-utility
Open disk utility to check witch letter is your usb drive actually has:
gnome-disks
In my case a 32GB SD card with Raspbian image on it recognised as: /dev/sde
So I run with /dev/sde:
sudo dd bs=4M status=progress if=/dev/sde | gzip > \
/you-selected-full-path-here/raspberry-images/`date +%Y%m%d`_rpi_image_backup.gz
status=progress gives you progress bar indication
| gzip > compresses the 32 GB total size and not writing the empty space from the 32 GB
`date +%Y%m%d` writes today date in the filename
Output: 20190529_rpi_image_backup.gz
And the size is only 3.5GB. If you want to write this image to a new SD card use:
Also you can write this image made from 32 GB to a 16 GB or 8 GB disk, it is not complaining that the image is too large anymore.
-
I tried your steps for my 64 GB SD card which has 2 partitions. In total their size is about 7GB. However, after several minutes I see the process is not finished and the size of the created image is more than 25GB. I did again with
gzio -9and the result is almost the same. Why don't I have an 7GB image? – ywiyogo Nov 9 '19 at 10:34
After trying multiple different methods, I found the following article:
https://medium.com/platformer-blog/creating-a-custom-raspbian-os-image-for-production-3fcb43ff3630
It's created to shrink and resize (on first boot) a raspberry-pi image but could be easily adjusted for any other Linux distribution. I successfully got it working with Debian 9 on a custom arm based chip.
The rc.local created by the pishrink script first uses raspi-config to resize the rootfs and then falls back to a method using parted (which I had to install ahead of time on my machine). I commented out the section of code where raspi-config.
My sd-card image was shrunk from 15 GB to 1.1 GB. I flashed the shrunk sd-card image with etcher. Took less than 5 mins as opposed to over half an hour for the full 15 GB image
