I would like to rename/move a project subtree in Git moving it from

/project/xyz

to

/components/xyz

If I use a plain git mv project components then all the file history for the xyz project gets lost.

Is there a way to move this such that the history is maintained?

share|improve this question
4  
    
I just want to note that I just tested moving files via the filesystem, and after committing (via intellij) I can then see the whole history (including history when it was at a different location) when viewing the history (again in intellij). I'm assuming intellij isn't doing anything particularly special to do that, so its nice to know that at very least the history can be traced. – B T Oct 15 '16 at 1:34
up vote 454 down vote accepted

Git detects renames rather than persisting the operation with the commit, so whether you use git mv or just a plain mv doesn't matter.

The log command, however, takes a --follow argument that continues history before a rename operation (that is, it searches for similar content using the heuristics):

http://git-scm.com/docs/git-log

To lookup the full history use the following command:

git log --follow ./path/to/file
share|improve this answer
49  
I suspect this is a performance consideration. If you don't need the full history, it will sure take longer time scanning the content. The easiest way is to setup an alias git config alias.logf "log --follow" and just write git logf ./path/to/file. – Troels Thomsen Feb 23 '10 at 7:36
11  
@TroelsThomsen this e-mail by Linus Torvalds, linked from this answer, indicates that it's an intentional design choice of Git since it's allegedly much more powerful than tracking renames etc. – Emil Lundberg Sep 6 '13 at 14:58
55  
This answer is a bit misleading. Git does "detect renames," but very late in the game; the question is asking how you ensure Git tracks renames, and someone reading this can easily infer that Git detects them automatically for you and makes note of it. It does not. Git has no real handling of renames, and instead there are merge/log tools that attempt to figure out what happened - and rarely get it right. Linus has a mistaken but vehement argument as to why git should never just do it the right way and track renames explicitly. So, we're stuck here. – Chris Moschini Apr 10 '14 at 18:19
18  
Important: if you rename a directory, for example during renaming of a Java package, be sure to execute two commits, first for the 'git mv {old} {new}' command, second for the updates of all Java files that reference the changed package directory. Otherwise git can't track the individual files even with the --follow parameter. – nn4l Oct 4 '14 at 16:31
14  
Though Linus probably makes very few mistakes, this does appears to be one. Simply renaming a folder causes a massive delta to be uploaded to GitHub. Which makes me cautious about renaming my folders...but that's a pretty big straight-jacket for a programmer. Occasionally, I HAVE TO re-define the meaning of something, or change how things are categorized. Linus: "In other words, I'm right. I'm always right, but sometimes I'm more right than other times. And dammit, when I say 'files don't matter', I'm really really Right(tm)." ...I have my doubts about that one. – Gabe Halsmer Sep 11 '15 at 16:04

It is possible to rename a file and keep the history intact, although it causes the file to be renamed throughout the entire history of the repository. This is probably only for the obsessive git-log-lovers, and has some serious implications, including these:

  • You could be rewriting a shared history, which is the most important DON'T while using Git. If someone else has cloned the repository, you'll break it doing this. They will have to re-clone to avoid headaches. This might be OK if the rename is important enough, but you'll need to consider this carefully -- you might end up upsetting an entire opensource community!
  • If you've referenced the file using it's old name earlier in the repository history, you're effectively breaking earlier versions. To remedy this, you'll have to do a bit more hoop jumping. It's not impossible, just tedious and possibly not worth it.

Now, since you're still with me, you're a probably solo developer renaming a completely isolated file. Let's move a file using filter-tree!

Assume you're going to move a file old into a folder dir and give it the name new

This could be done with git mv old dir/new && git add -u dir/new, but that breaks history.

Instead:

git filter-branch --tree-filter 'if [ -f old ]; then mkdir dir && mv old dir/new; fi' HEAD

will redo every commit in the branch, executing the command in the ticks for each iteration. Plenty of stuff can go wrong when you do this. I normally test to see if the file is present (otherwise it's not there yet to move) and then perform the necessary steps to shoehorn the tree to my liking. Here you might sed through files to alter references to the file and so on. Knock yourself out! :)

When completed, the file is moved and the log is intact. You feel like a ninja pirate.

Also; The mkdir dir is only necessary if you move the file to a new folder, of course. The if will avoid the creation of this folder earlier in history than your file exists.

share|improve this answer
33  
As an obsessive git-log-lover, I wouldn't go for this. The files weren't named that at those points in time, hence history reflects a never-existent situation. Who knows what tests might break in the past! The risk of breaking earlier versions is in pretty much every case not worth it. – Vincent Jan 6 '14 at 16:52
5  
@Vincent You're absolutely right, and I tried to be as clear as I could about the unlikeliness of this solution being appropriate. I also think we're talking about two meanings of the word "history" in this case, I appreciate both. – Øystein Steimler Jan 7 '14 at 13:41
4  
I find there are situations where one might need this. Say I developed something in my own personal branch, which I now want to merge upstream. But I discover, the filename isn't apropriate, so I change it for my whole personal branch. In that way I can keep a clean proper history and have the correct name from the beginning. – user2291758 Mar 4 '15 at 7:52
1  
@user2291758 that's my use case. These more powerful git commands are dangerous but that doesn't mean they don't have very compelling use cases if you know what you're doing! – philix Sep 1 '16 at 12:06
git log --follow [file]

will show you the history through renames.

share|improve this answer
20  
It appears that this requires you to commit just the rename before you start modifying the file. If you move the file (in the shell) and then change it, all bets are off. – yoyo Dec 13 '11 at 22:15
14  
@yoyo: that's because git doesn't track renames, it detects them. A git mv basically does a git rm && git add. There are options like -M90 / --find-renames=90 to consider a file to be renamed when it's 90% identical. – vdboor Jun 13 '12 at 12:29

No.

The short answer is NO, it is not possible to rename a file in Git and remember the history. And it is a pain.

Rumor has it that git log --follow--find-copies-harder will work but it does not work for me, even if there are zero changes to the file contents, and the moves have been made with git mv.

(Initially I used Eclipse to rename and update packages in one operation, which may have confused git. But that is a very common thing to do. --follow does seem to work if only a mv is performed and then a commit and the mv is not too far.)

Linus says that you are supposed to understand the entire contents of a software project holistically, not needing to track individual files. Well, sadly, my small brain cannot do that.

It is really annoying that so many people have mindlessly repeated the statement that git automatically track moves. They have wasted my time. Git does no such thing. By design(!) Git does not track moves at all.

My solution is to rename the files back to their original locations. Change the software to fit the source control. With git you just seem to need to git it right the first time.

Unfortunately, that breaks Eclipse, which seems to use --follow.
git log --follow Sometimes does not show the full history of files with complicated rename histories even though git log does. (I do not know why.)

(There are some too clever hacks that go back and recommit old work, but they are rather frightening. See GitHub-Gist: emiller/git-mv-with-history.)

share|improve this answer
    
I believe you are correct. I was just trying to use php-cs-fixer to reformat the source for my Laravel 5 project but it insists on changing the capitalization of the namespace clauses to match the lowercase value of the app folder. But namespaces (or composer autoloads) only work with CamelCase. I need to change the capitalization of the folder to App but this causes my changes to be lost. This is the most trivial of examples, but shows how the git heuristic is not able to follow even the simplest of name changes (--follow and --find-copies-harder should be the rule, not the exception). – Zack Morris Jul 9 '16 at 3:25

I do:

git mv {old} {new}
git add -u {new}
share|improve this answer
2  
The -u doesn't seem to do anything for me, is it suppose to update the history? – jeremy Jan 24 '13 at 22:47
    
See here: git-scm.com/docs/git-add – James M. Greene Jan 26 '13 at 1:04
    
Perhaps you want the behavior of -A instead? Again, see here: git-scm.com/docs/git-add – James M. Greene Jan 26 '13 at 1:05
1  
It does add the files, however it doesn't update the history so that 'git log file name' shows the full history. It only shows the full history if you use the --follow option still. – jeremy Jan 26 '13 at 1:45
1  
Oh, I see. Bummer.... – James M. Greene Jan 27 '13 at 2:48

Objective

This answer is based on git am inspired from Smar's answer borrowed from Exherbo’s docs.

  • Keep history of files copied from one directory to another
  • Or files moved from one repository to another
  • But does not keep tags/branches
  • History is cut when file is renamed (i.e. directory renamed)

Summary

  1. Extract history in email format using
    git log --pretty=email -p --reverse --full-index --binary
  2. Reorganize file tree and update filenames
  3. Apply new history using git am

1. Extract history in email format

Example: Extract history of file3, file4 and file5

my_repo
├── dirA
│   ├── file1
│   └── file2
├── dirB            ^
│   ├── subdir      | To be moved
│   │   ├── file3   | with history
│   │   └── file4   | 
│   └── file5       v
└── dirC
    ├── file6
    └── file7

Set/clean the destination

export historydir=/tmp/mail/dir       # Absolute path
rm -rf "$historydir"    # Caution when cleaning the folder

Extract history of each file in email format

cd my_repo/dirB
find -name .git -prune -o -type d -o -exec bash -c 'mkdir -p "$historydir/${0%/*}" && git log --pretty=email -p --stat --reverse --full-index --binary -- "$0" > "$historydir/$0"' {} ';'

Unfortunately option --follow or --find-copies-harder cannot be combined with --reverse. This is why history is cut when file is renamed (or when a parent directory is renamed).

Temporary history in email format:

/tmp/mail/dir
    ├── subdir
    │   ├── file3
    │   └── file4
    └── file5

Dan Bonachea suggests to invert the loops of the git log generation command in this first step: rather than running git log once per file, run it exactly once with a list of files on the command line and generate a single unified log. This way commits that modify multiple files remain a single commit in the result, and all the new commits maintain their original relative order. Note this also requires changes in second step below when rewriting filenames in the (now unified) log.


2. Reorganize file tree and update filenames

Suppose you want to move these three files in this other repo (can be the same repo).

my_other_repo
├── dirF
│   ├── file55
│   └── file56
├── dirB              # New tree
│   ├── dirB1         # from subdir
│   │   ├── file33    # from file3
│   │   └── file44    # from file4
│   └── dirB2         # new dir
│        └── file5    # from file5
└── dirH
    └── file77

Therefore reorganize your files:

cd /tmp/mail/dir
mkdir -p dirB/dirB1
mv subdir/file3 dirB/dirB1/file33
mv subdir/file4 dirB/dirB1/file44
mkdir -p dirB/dirB2
mv file5 dirB/dirB2

Your temporary history is now:

/tmp/mail/dir
    └── dirB
        ├── dirB1
        │   ├── file33
        │   └── file44
        └── dirB2
             └── file5

Change also filenames within the history:

cd "$historydir"
find * -type f -exec bash -c 'sed "/^diff --git a\|^--- a\|^+++ b/s:\( [ab]\)/[^ ]*:\1/$0:g" -i "$0"' {} ';'

3. Apply new history

Your other repo is:

my_other_repo
├── dirF
│   ├── file55
│   └── file56
└── dirH
    └── file77

Apply commits from temporary history files:

cd my_other_repo
find "$historydir" -type f -exec cat {} + | git am --committer-date-is-author-date

--committer-date-is-author-date preserves the original commit time-stamps (Dan Bonachea's comment).

Your other repo is now:

my_other_repo
├── dirF
│   ├── file55
│   └── file56
├── dirB
│   ├── dirB1
│   │   ├── file33
│   │   └── file44
│   └── dirB2
│        └── file5
└── dirH
    └── file77

Use git status to see amount of commits ready to be pushed :-)


Extra trick: Check renamed/moved files within your repo

To list the files having been renamed:

find -name .git -prune -o -exec git log --pretty=tformat:'' --numstat --follow {} ';' | grep '=>'

More customizations: You can complete the command git log using options --find-copies-harder or --reverse. You can also remove the first two columns using cut -f3- and grepping complete pattern '{.* => .*}'.

find -name .git -prune -o -exec git log --pretty=tformat:'' --numstat --follow --find-copies-harder --reverse {} ';' | cut -f3- | grep '{.* => .*}'
share|improve this answer
1  
BEWARE: This technique splits commits that change 2 or more files into separate fragmented-commits, and furthermore scrambles their order by sorting on filename (so the fragments of one original commit do not appear adjacent in linear history). The resulting history is therefore only "correct" on a file-by-file basis. If you are moving more than one file, then NONE of the new commits in the resulting history represent a consistent snapshot of the moved files that ever existed in the history of the original repo. – Dan Bonachea Nov 15 '16 at 17:00
1  
Hi @DanBonachea. Thank you for your interesting feedback. I have successfully migrated some repos containing several files using this technique (even with renamed files and files moved across directories). What do you suggest to change in this answer. Do you think we should add an WARNING banner at the top of this answer explaining the limitations of this technique? Cheers – olibre Nov 16 '16 at 15:12
    
I adapted this technique to avoid the problem by inverting the loops of the git log generation command in step 1. Ie. rather than running git log once per file, run it exactly once with a list of files on the command line and generate a single unified log. This way commits that modify 2 or more files remain a single commit in the result, and all the new commits maintain their original relative order. Note this also requires changes in step 2 when rewriting filenames in the (now unified) log. I also used git am --committer-date-is-author-date to preserve the original commit timestamps. – Dan Bonachea Dec 4 '16 at 3:44
    
Thank you for your experimentation and sharing. I have updated a bit the answer for other readers. However I have took time to test your processing. Please feel free to edit this answer if you want to provide examples of command lines. Cheers ;) – olibre Dec 5 '16 at 6:08

I make moving the files and then do

git add -A

which put in the sataging area all deleted/new files. Here git realizes that the file is moved.

git commit -m "my message"
git push

I do not know why but this works for me.

share|improve this answer
    
not working for me :( – dr0i Jan 19 at 16:35
    
The trick here is that you do not have to change a single letter, even if you change something and press Ctrl+Z, the history will be broken. SO in that case if you write something revert the file, and the move it again and make a single add->commit for it. – Xelian Jan 20 at 8:56

While the core of git, the git plumbing doesn't keep track of renames, the history you display with the git log "porcelain" can detect them if you like.

For a given git log use the -M option:

git log -p -M

With a current version of git.

This works for other commands like git diff as well.

There are options to make the comparisons more or less rigorous. If you rename a file without making significant changes to the file at the same time it makes it easier for git log and friends to detect the rename. For this reason some people rename files in one commit and change them in another.

There's a cost in cpu use whenever you ask git to find where files have been renamed, so whether you use it or not, and when, is up to you.

If you would like to always have your history reported with rename detection in a particular repository you can use:

git config diff.renames 1

Files moving from one directory to another is detected. Here's an example:

commit c3ee8dfb01e357eba1ab18003be1490a46325992
Author: John S. Gruber <JohnSGruber@gmail.com>
Date:   Wed Feb 22 22:20:19 2017 -0500

    test rename again

diff --git a/yyy/power.py b/zzz/power.py
similarity index 100%
rename from yyy/power.py
rename to zzz/power.py

commit ae181377154eca800832087500c258a20c95d1c3
Author: John S. Gruber <JohnSGruber@gmail.com>
Date:   Wed Feb 22 22:19:17 2017 -0500

    rename test

diff --git a/power.py b/yyy/power.py
similarity index 100%
rename from power.py
rename to yyy/power.py

Please note that this works whenever you are using diff, not just with git log. For example:

$ git diff HEAD c3ee8df
diff --git a/power.py b/zzz/power.py
similarity index 100%
rename from power.py
rename to zzz/power.py

As a trial I made a small change in one file in a feature branch and committed it and then in the master branch I renamed the file, committed, and then made a small change in another part of the file and commited that. When I went to feature branch and merged from master the merge renamed the file and merged the changes. Here's the output from the merge:

 $ git merge -v master
 Auto-merging single
 Merge made by the 'recursive' strategy.
  one => single | 4 ++++
  1 file changed, 4 insertions(+)
  rename one => single (67%)

The result was a working directory with the file renamed and both text changes made. So it's possible for git to do the right thing despite the fact that it doesn't explicitly track renames.

This is an late answer to an old question so the other answers may have been correct for the git version at the time.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.