[APP][ROOT][5.0+][1st Nov 2020] Migrate - custom ROM migration tool

Jumping from one custom ROM to another can be a challenging task. So I decided to make an app to simplify the process.

Introducing
Migrate!

Migrate-GPE (v4.0) is on Google Play
It was temporarily suspended. Read more here: https://xdaforums.com/showpost.php?p=82230451&postcount=1273

Download from Google Play Store
Download from AndroidFileHost
Download from XDA labs (removed as XDA Labs has shutdown)

CODEBASE ON GITHUB




With Migrate, you can back up:

1. App apk
2. App data
3. App runtime permissions
4. Contacts (as vcf file)
5. SMS messages
6. Call logs
7. Current ROM screen DPI
8. Current default keyboard option
9. App installer (Migrate 3.0)
10. Adb state (Migrate 3.0)
11. Font scale (Migrate 3.0)

... everything in one app.




Ok. So its just another of those backup and restore apps, right?

Sort of... but with a twist.
Restoring everything requires flashing the backup via TWRP.

ALTERNATELY
You can use the Migrate Flasher app (if your phone does not have a compatible recovery, or your recovery is broken).

If you are flashing via TWRP, see the below screenshot.



All your apps and data are restored to a directory under /data/balti.migrate, /data/local/tmp/migrate_cache (from v2.0+) and a helper app is injected into the system.. Once you boot up your new ROM, the app will spring up, ask for root permission and do all the job while you can relax.



IF YOU ARE USING MIGRATE FLASHER
The process is similar. Only difference is Migrate Helper is not injected into system, but installed as a user app. There is no use of recovery environment.
Once Migrate Helper is installed, just press RESTORE APPS AND DATA and proceed as usual.

Please see the HOW TO FLASH section on Migrate Flasher for complete illustration.

IF you have already flashed using TWRP (with no errors) then the above sections about Migrate Flasher is not applicable.

Donations:
Well, the app already has ads, so you need not explicitly donate anything
But if you are feeling reaalllyyy grateful for the app, you can donate here: paypal.me/sayantanrc
Thanks.

Migrate Gemini (v4.0) release

Hello everyone.
Migrate 4.0 is now released.

There are 2 editions of Migrate.

Migrate NG: This is the Non-Google and Beta edition. It is not published on Google Play store. It will continue having in-app updates. Moreover new features will be tested on these builds first. People who did not like firebase analytics, this edition is for you as it does not have firebase SDK anymore. However, Google admob ads are still present.

Download link (NG): https://www.androidfilehost.com/?fid=10763459528675575696

Migrate GPE: This version is Google Play Edition. It is the stable version. It does not have in-app updates anymore as it violated Google Play policies.
Download on Google Play: https://play.google.com/store/apps/details?id=balti.migrate

Coming from alpha 3.1, if you update via the in-app updater, you will be switched to the Non-Google Edition.

Thank you.

Want to translate the app?
The strings.xml file is available here: https://github.com/SayantanRC/translations

Telegram group!
We now have a Telegram group for migrate:
http://t.me/migrateApp
This is not to rival XDA but only another form of communication between users and me
Thanks.

FAQ:

Click here for Troubleshooting options

If you are facing issues while flashing, refer to this video: https://youtu.be/0Cp6TNayXnM
Full manual extract guide: https://xdaforums.com/showpost.php?p=82198185&postcount=1269

How is this app different from Titanium backup?
This app is a ROM migration tool. It not only backs up app and data, but also contacts, SMS, call logs etc. Moreover, the restoration process also involves flashing via TWRP. Titanium backup is an excellent app. Migrate just views the restoration process differently.

In recent times however, Migrate Flasher app can be used to flash a zip even if you dont have a compatible TWRP recovery. Everything is almost the same other than just the flashing part, which is done from the Flasher app.

Do I need to download the Migrate app to restore the apps?
No. Only your ROM needs to be rooted. The restoration process is handled by a helper app, which deletes itself after restoring everything.

Is it necessary to use Magisk?
The app works best with Magisk. Other options like Lineage OS su addon causes weird errors like incomplete file copy, etc.

All apps are restored from recovery?
No. That is not possible. Because Recovery is a completely different environment from Android. Think of it as installing a Windows game from a Linux distro. Instead from recovery, an app is injected into system which takes up the restore process when android boots up. Most things are automated, so you need not worry much.

A quick word on SD card support...
SD card support in v2.0 requires a Magisk module "ExSDCard Access Enabler", made by developer Romain. According to him, this module requires FUSE support on ROMs. FUSE is supported on some ROMs as a legacy feature as Android has moved away from FUSE to SDCARDFS. As such, the current implementation may or may not work based on your device and ROM.
If it does not work for you, please wait until either the module developer rewrites the module or I find a different implementation Thanks for your patience.

From version 5.0+ (Not yet released, available on GitHub)
Users will be able to natively store backups on SD Cards, especially on Android 11+.
For lower Android versions, Storage Access Framework is used (along with root access) to store backups on SD Cards as well as on USB-OTG.

Changelog

Version 4.0 [1st November 2020]

# Compatibility fixes for Android 11
# Add Message center to display important announcements.
# Migrate helper now allows to retry failed apk installs
# Add a "Migrate flasher only" backup option
# Save app icons as png files
# Show number of apps selected for backup in Extra backups screen
# Keep screen on when calculating backup size
# Save last state of selection in extras
# Don't remount /data partition in system test
# Many under the hood changes

Version 3.1.1 [25thApril 2020]

# Fix issue in v3.1 where restore would not be possible for apps with unsupported characters in the name.

Version 3.1 [24thApril 2020]

# Probable fix to data restore problems
# Probable fix to helper crashing during loading apps
# Record raw list of files in a backup
# Many other minor improvements
Use v3.1.1 above

Version 3.0.3 [18thApril 2020]

#. Slightly tweaked logo.
#. Probable fix to No metadata found after flashing.

Version 3.0.1 [9thApril 2020]

#. Bug fixes
#. Added translations

Version 3.0 [20th March 2020]

#. The app is rewritten in Kotlin
#. Many new preferences have been added.
#. Helper app is now available on Google Play.
#. Many fixes in recovery. Support for SAR and Android 10.
#. Added options for backup of app installer, font scale, adb state
#. Set max zip size in preferences.
#. Verification of zips after backup is complete.
#. Many more changes and improvements…

Version 2.1 [15th Jan]

# Minor errors are now suppressed, but recorded in log.
# Fix icon retention form previous backup
# Fix wrong backup duration display.
# Fixed a bug in error correction mechanism.
# Probable fix for some apps not being backed up.
# Fix for error 6 in TWRP for some locales.
# Fixed a long standing wrong /data space calculation in TWRP.
# Migrate helper no longer uninstalls app before installing.
# Added Telegram group link!

Version 2.0.1 [30th Dec] (christmas_beta)

# Fixed a probable crash in Migrate helper (needs a magisk module)

Version 2.0 [26th Dec] (christmas_beta)

# Experimental SD card support (needs a magisk module)
(please read the previous post)
# Support for split apks
# "Abort" option in Migrate helper
# "Temporary disable" in Migrate helper
# Apks are now unpacked under /data/local/tmp/migrate_cache
(probable fix to some devices unable to install apks)
# Each backup zip now has a busybox file as fallback

Version 1.2 [12th Nov] (beta)

# Completely avoid "dumpsys" to backup app permissions
# Now app data in tar.gz is unpacked in /data/data
(probable fix to "Operation not permitted" issues in Google Pixel and some other devices)

Version 1.1.1 [10th Nov] (beta)

v1.1 (skipping upload)

# [New] Backup verification before zipping
# Major code refactoring and under-the-hood changes
# CANCEL button now kills the backup process
# Fix to "set_progress()" in recovery
# Probable fix to "Failed to change locale for db"
# Probable fix to missing updater-script
# Probable fix to some "Operation not permitted" issues
# Remove illegal characters from backup name
# "Share scripts" option should now work properly.

v1.1.1

# Fix to a no-closing-quote issue in helper

Version 1.0.5 [7th Nov] (beta)
# Probable fix to "android.os.TransactionTooLargeException"
# Better in app protection to prevent zips from not being made
# App version is now written in logs
# App now targets Oreo instead of Pie to potentially avoid some crashes
# Fixed unexpected ( in restore process

Version 1.0.4 [5th Nov] (beta)
# Improvements to logging
# Ignore tar socket warnings
# New update-binary to prevent TWRP error 3
# Should fix a null String issue while reading backup data

Version 1.0.3 (beta)
# Fixed log sharing
# Share script file in log
# Blacklist Android system from system app backup list (prevents tar socket error)

Version 1.0 (beta)
# Initial release

Some plaguing bugs on some devices


"dumpsys not found"
(Severity: mild. Only permissions are not backed up)
Mainly in ASUS devices. Probable solution is to not use dumpsys at all.

Probably fixed in v1.2

"tar: can't execute 'gzip': Permission denied"
(Severity: severe. Do not restore this backup)
I still have no idea why this is happening. Probable solution is use something other than tar.gz


Data files cannot be moved during restore. "Operation not permitted"
(Severity: severe)
No idea yet. Prominent in Google Pixel devices and some Xiaomi devices

Probably partially or completely fixed in v1.2

Error 7 (or 6) in TWRP
(Severity: severe)

This issue which was supposedly fixed in v1.1.1 seems to occur again on few devices
Since I cannot reproduce the issue and the number of error reports are pretty small, please send the recovery.log from TWRP:
- Goto Advanced -> Terminal in TWRP
OR
open an adb shell inside TWRP
- Type the following and press enter: cp /tmp/recovery.log /sdcard/
- Reboot and upload the recovery.log file (from internal sd card) here.


Stuck at calculating size.
(Severity: severe)

This is being reported multiple times. I am still unable to reproduce the issue on my device.
That being said, I will be adding new controls to skip size checking of its stuck on any device, in the next release. Also, I will add more fine logging to see what is actually happening behind the scenes, which cannot be detected by a logcat.

Probably fixed in v2.0

Pocophone TWRP recovery has issues
(Severity: severe)

I need someone to do the following and then upload the file:/sdcard/test_log.log
- TWRP main menu -> Mount -> Data and System should be mounted
- TWRP main menu -> Advanced -> Terminal
OR
open an adb shell inside TWRP
- Type the following commands:

getprop > /sdcard/test_log.log
echo " " >> /sdcard/test_log.log
df /data >> /sdcard/test_log.log
echo " " >> /sdcard/test_log.log
df -k /data >> /sdcard/test_log.log
echo " " >> /sdcard/test_log.log
df -h /data >> /sdcard/test_log.log
echo " " >> /sdcard/test_log.log
df /system >> /sdcard/test_log.log

The following should work as an alternative:

SayantanRC said:

You have to manually do the following:
1. Open the backup zip file
2. Edit the "package-data.txt" file
3. Put the "data_required_size" as "0"
4. Save the file.
5. Repack the zip file.

Click to expand...

Click to collapse

"Segmentation fault"
(Severity: unknown)
Mainly in Xiaomi devices. I cannot reproduce it. Neither do I know how much this affects the backup. Please restore a backup and tell me if it works.

candiesdoodle said:

Quite promising
So what does the helper app really do? Move the apps and data from /data/balti.migrate to their respective directories? How does that work in a booted up ROM?

Click to expand...

Click to collapse

The helper app installs the apks using package manager and then unpacks the data of the respective apps in /data/data. Finally it restores all the apps runtime permissions.

ceelos218 said:

I got this issue

Click to expand...

Click to collapse

Please tell how to reproduce the issue.

m0d said:

Wifi?

Click to expand...

Click to collapse

Maybe added in future

m0d said:

Does it handle folder permissions when there are changes in app uid for data restore
when pm installs app, app uid can be different than before, so there might be folder permission discrepencies?

I.E.

Code:

#!/bin/sh
if [ "$1" != "" ]; 
then 	
#The folders in /data/data each have a different owner. The next line gets the owner and the strips the whitespace 	

appusername=$(stat -c "%U" /data/data/$1 | xargs) 	
#The next line constructs the chown command to repair the permissions for all the files in the target folder 	
fixperm="chown -R $appusername:$appusername /data/data/$1/*" 	
#Display the command to the user 	
echo "$fixperm" 	
#Uncomment the next line if you would like the script to fix the permission 	
eval $fixperm 
else echo "Missing target folder parameter" 
fi

Click to expand...

Click to collapse

It takes care of the new uid for the app. No problem with that.

ceelos218 said:

I just clicked next.. nothing else I selected all apps to backup then next page ticked boxes to back up.. click ready and I got that error and It didn't back up anything.

Click to expand...

Click to collapse

This is weird. Please open the green circle button on top left in the first screen of the app -> Contact by email and send the issue.