nemo significantly slower opening folder content #1907
Comments
|
I can't agree with the nautilus comparison. Testing side-by-side, using the same style view they appear to me nearly the same. Both programs load their list view fairly quickly, and the icon view slowly. One difference is that nautilus uses the list view as its default, whereas nemo uses the icon view. This can be changed in the first tab of preferences. Additionally, the number and type of extensions in use can vastly affect load times. Efforts are being constantly made to improve these speeds. We're aware of thunar's performance, but it's not trivial to just say 'do it the way they do it' and snap our fingers. |
|
Although you may not agree with nautilus comparison. many have complained nemo is slower than nautilus. I continuously click on the the sidebar shortcuts they both use about the same CPU load but nemo is slower in refresh visually. Thunar uses the much less CPU. I made the test in the past, I disabled extensions and made no difference |
|
Since upgrading to Mint 19 (and now using Nemo 3.8.5), handling files in folders with a large number of files has become practically impossible, it uses one CPU to almost 100% to make any action (moving a file, creating a folder, ...) and can take up to 5-10 seconds to respond. |
|
@donlencho Is not that serious with me where it uses 100% CPU but it does use alot when you browse folders. Not only high CPU load but visually slow. It is terrible to use when you are working on projects where you browse to many levels of subdirectories, I need to wait for folder to appear before clicking on it. |
|
@mtwebster There are certainly many ways of doing things. The disappointment is after several major updates to Linux Mint there is no improvement of nemo's performance, not even to know what the issue is causing the slowness. Perhaps to do with the way gtk3 works. |
|
@howdev @donlencho Is there any thing you can do to help pin down where the problems you experience are ? So far you have identified 1) folders with a very large number of files 2) folders with many levels of subdirectories. Could you try, for example, a) disabling plugins, actions [edit, plugins] b) turning off thumbnails c) turning off folder counting [edit, preferences] and feed back if any of those make a difference. Similarly clarify what other preferences you might be using, such as tooltips and try changing those modes d) say if you get differences with different types of nemo view e) clarify what types of disk storage you are using, and how full you are running the disks Regards Simon |
|
@brownsr The issues is not about many subfolders. The slowness is from loading folder contents whether they are files or folders. It is slow to display even the content only contain few items. Plugins disable tested before no difference. GTK3 had a similar performance issue in the past with slow appending liststore. Someone suggested to use insert_with_valuesv instead. This may have to do with the way the model is populated for the iconview And by the way I think this may have to do with sorting of the model as well. The populate and sorting of the model needs to be investigated. |
|
This is another problem I found on is to do with sorting in treeview. The sorting is to do with model and regardless of the view whether is treeview or iconview. |
|
Not sure we are talking about the same problem, as said before. I'll keep posting in #1902 instead. |
|
you may say can't compare gtk2 and gtk3 app, but I just installed Thunar 1.8 is gtk3 and still as fast as Thunar gtk2 @mtwebster therefore should investigate the way Thunar does it. There is serious problem the way nemo does the iconview with the model |
|
I have serious difficulties with opening folders which are on a mounted separate disk. Nemo works fast on opening one of two external disk, but not the other. The entries in fstab are: Nemo goes to 100% cpu only on one folder in this second disk. What information could I provide to help tracing this problem? What could be different in this folder from the others? |
|
First I would eliminate any hardware issues, e.g. try different cables. Does it happen with other file managers, e.g. nautilus. |
|
Does this folder have a lot of pictures or other file types that have a thumbnail generated? Try a couple things: This gets rid of existing thumbnails (in case there's an issue with one that's being hung on) and/or Try disabling thumbnailing: Preferences->Previews->Show Thumbnails - Never Ensure you've got a new instance of nemo: Also, you can try disabling any extensions temporarily (Edit->Plugins, lower section) and again, restart. |
|
Thank you for the advice - nothing helps. No thumbnails in the file, not shown anyhow; disabled all extensions (nothing was present beyond the defaults). Most surprising, if I open the folders from outside by unfolding them in the same windows, no undue delays occur. it is only in this folder for one folder, if it is opened (not unfolded), then nemo goes into 100 % cpu. What could be special about this folder (all others in the same folder open ok)? |
|
Open a terminal in that folder, run The fact it opens ok from the parent folder makes me think there's something wrong maybe inside one of the children of problem folder. Under normal circumstances, a view's 'directory' doesn't delve deeper than one layer deep of children by default (you get 'deep' counts only when doing things like right-click->properties or copy/move operations). Though I'd think no matter how you view the folder, you'd run into the problem (whether expanding from the parent or some other way). Regardless, check the members of the folder as mentioned, and one deep in any child folders. Does this happen in the icon view also? Also, what version of nemo is this and what distro are you on? |
|
I have hundreds of folders and nemo takes roughly 20 seconds to respond after start. The only thing that works for me (Mint Linux 19.1, Nemo 4.0.6) is edit-preferences-plugins- then under extensions disable nemo media columns. No thumbnails but nemo it is very responsive now in all folders. |
|
Add my name to the list of 'Nemo is slow and getting slower'. I got a folder with around 10,000 files in it. It opens almost instantly on Windows 7/Windows 10 and displays thumbnails instantly too. On Nemo, it very slowly adds thumbnails to the file browser, and before even displaying all of the files, it maxes out CPU usage of one core and eventually becomes completely unresponsive and must be 'Force Quit'. I know 10,000 files is a lot of files, but on a fast 8 core CPU, an M.2 SSD, and with 32GB's of RAM, and high end GPU, I expect to at least be able to open the folder and display all of the files in it. And when the performance on Windows 7/10 is so much better for this kind of thing, almost near instant results, I can't help but think Nemo must be doing something very inefficiently. Mint 19 was meant to improve this, but from what I've seen so far, it seems like it's actually slower and worse. It was slow before, but this behaviour of slowing down to the point of becoming completely unresponsive is new. Don't know what to suggest folks, but IMO, performance improvements for Nemo should be the number 1 priority right now above literally anything else. |
|
The bug hits me as well... I have a photo folder containing several thousands of JPEG spread over about a dozen of subfolders. What I want to do is use Nemo for browsing the photos, and copying some of them into a different folder. It's supposed to be rather an easy task for a file manager. However, the more photo folders I browsed into, the slower Nemo gets. As soon as I unfocus the Nemo window, Nemo consumes 100% CPU for several seconds (up to minutes), and all Nemo windows become unresponsive. I used gdb and backtracked the Nemo process when it happened. It turns out that the stack is filled with some 10,000 (!) recursive calls to This is all I could figure out due to my very limited GTK and gdb skills. It seems that Nemo is mostly busy computing CSS stuff. I don't know though why it gets worse and worse the more image folders I have visited. |
|
Addition: I have tried different themes (Arc-Darker, Mint-Y, Adapta-Nokto), but the issue persisted, so it does not seem to be related to the theme used. It rather seems to be the way Nemo uses GTK3, as if each icon shown adds another CSS layer (and thus another CSS parent) to the stack. |
|
I tried to time Windows 7 to see how long it takes to open and load a folder with 12,000 photos in it and display the thumbnails but unfortunately the bugger is too quick for me and I can't tap the timer button on my phone fast enough to keep up with it, it's probably less than 0.2 seconds or so. That's on an SSD with 500MB/sec read/write. Meanwhile, Nemo on the other hand, simply can't open the folder without crashing, and that's after a minute or two of trying to load all the icons into view. This really is a problem. This is on the same computer too (dual booted), but the sad part is, Windows 7 is on a slower SSD, Mint is on the faster M.2 SSD, with 3500MB/sec read/write speed. Shred sounds like he's onto something with the CSS. |
|
I've tried every suggestion in this thread, but nothing makes a difference. Nemo simply performs poorly. It takes long to list the contents of a folder with many files, even with thumbnails disabled. If I open and close the same folder within seconds, it still goes through the same "refresh" process of slowly displaying all the contents. However, with Thunar, the same exact directory lists all its contents IMMEDIATELY. There are no "accidental clicks" in Thunar because all the items display immediately and they do not "shuffle around" as more icons are being displayed. The scrollbar in Thunar never "shrinks", as compared to Nemo which has a scrollbar that shrinks and shrinks because it's still in the process of listing all of the files. It's night and day. I love Mint 19, but I can't stand to use Nemo. It's sad because the file manager is an integral part of any desktop session. Mint 19.1 Nemo 4.0.6 Thunar 1.6.15 Fully updated system. |
|
Nemo is just too slow and performs poorly. I sit with some folders with a few hundred photos within and scrolling / copy / cut / paste actions is a nightmare. It literately becomes unresponsive for 10 seconds at a time. I really hope the developers will prioritize this as their number 1 bug fix, otherwise the best solution is to use another file manager with each new release of Mint. I'm currently looking at other file managers as I can't wait 10 seconds at a time per copy / cut / paste action. |
|
"I really hope the developers will prioritize this as their number 1 bug fix, otherwise the best solution is to use another file manager" Agreed. It's so bad that I installed Manjaro (KDE) on my other computer, and I might end up leaving Mint. This is too important, as dealing with files and directories is probably one of the most common usage of a desktop distro, second only to using a web browser. Thunar? INSTANT listing, even of many files and large directories. SpaceFM? INSTANT listing, even of many files and large directories. Dolphin? INSTANT listing, even of many files and large directories. Nemo? Horrendous. Yet simply using another file manager isn't a great solution, as you lose some integration with the desktop, and the visual layout of Nemo is excellent. |
|
Unfortunately, this was also my "solution". I'm sorry I couldn't help find the source of the problem and I understand it's difficult to solve for the developers if the cause is not clear, but this is precisely what made me switch to another desktop environment: I get behaviour in Nemo that I can't predict and reproduce if I try to! Another bug I get is also the same kind of strange behaviour: #1401 (it's been closed but unsolved). |
|
Well, I think the cause is rather clear (see my posting above). I rather guess the problem is that no core developer has the time to look into it. (Sadly I cannot help, as this problem exceeds my GTK knowledge by far.) I have switched to Thunar in the meantime. But I would love to return to Nemo, as I like its design and its integration into Cinnamon. |
|
Right, sorry, I missed your post. Seems like a probable cause indeed. |
|
Nemo 4.06 is still way faster than Nemo 3.x |
|
Why wouldn't a developer want to deal with and respond to these posts? I can't imagine. We did quite a bit last cycle improving performance, and you wouldn't believe the amount of time I've spent this cycle working on it (You wouldn't, I'm certain). I've targeted quite a number of places where I can significantly improve performance, but unfortunately it's not as obviously simple as it's been made out to be. To a significant degree we're tied to a certain program 'model'. It's not trivial to rip that all apart and reconstruct it, but we've been doing it a little at a time, and will continue to do it. As an example, last dev cycle we reworked how spacing is calculated in the icon view. There's much more efficient use of space now, and it gives us an opportunity now to further simplify some ridiculously complicated layout code. Another issue is that nemo is pretty popular because it's feature-rich. For instance, I could really speed up the detail/list view if I got rid of the expanders next to the folder names (and the entire functionality of being able to expand the view to see the contents of that folder), but then a lot of people would raise hell (rightfully so) for us removing it. I'm considering an option to disable it if desired, but I haven't decided for certain. There are a number of other things I've targeted as well, but I've no idea yet what of this will make it in or not. I'm sorry to say, but you guys are not the only users here. I'm not sure what you expect out of us. I apologize if for your particular workflow nemo performs unsatisfactorily. Be happy you have the option of other file managers. I don't want people jumping ship, obviously, but I'm also not that motivated by threads that devolve into this. We do what we can as we're able to. You are not going to see nemo performing like thunar for 4.2. If this is unacceptable to you then so be it. All I can promise that it is being worked on, even if we're not participating in these fun threads and responding regularly. Lastly, have you all evaluated this performance with all extensions disabled? (actions and scripts don't matter). Thanks. I would be happy to see any hard data on the performance issues. Some tips/suggestions: Kill nemo: Then clear your disk cache (I've made an alias to keep this handy: Then run nemo as: You'll get a few times printed out to the terminal. I'd be curious to see those times, along with pertinent info like how many files in the folder (reasonably exact please), are they mostly image files? Thumbnails enabled? Which view - list or icon. The more data the better, when I'm evaluating things, I look at those times in primarily four ways:
If there are lots of images I might blow away Only if you guys are interested, nothing I'm doing hinges on feedback here, it would just be handy to have more test data (and situations I can reproduce for myself). Obviously this is all hardware-dependent, but useful info can still be gleaned. |
|
Thank you Michael for this reply and sorry if some comments were offensive or at least not very constructive. It's true that not seeing reactions from people involved can lead to thinking that devs are not reading this or "caring". |
|
@mtwebster Icon view is how I choose what I want to move about but the task is not time critical so I'm happy to wait for the fix (and happier that there might be one, thanks to you chaps). I'm sure that if this is the long-standing problem that's been responsible for Nemo repeatedly "pausing" with a maxed-out thread in icon view and it's fixed it'll make a lot of people very happy. Thank you for taking the time to verify and investigate it. |
|
Have to admit that I don't do a lot of moving of images - typically I might upload a few dozen images into a sub-directory and view them with xviewer or GIMP (sometimes) - I occasionally edit (without moving) to produce a cut-down version. If you are doing a lot of image processing, long term, then it might be worth using some proper image management software. For example |
|
Maybe offtopic but I experienced another use case scenario suggesting that nemo is not the culprit : OS: Linux Mint 20 Ulyana
When restarting it nemo takes about 15 to 20 seconds to appear, and it has difficulty displaying the contents of a directory. Following the suggestions posted here I had hoped to see some improvments with thunar but it displayed the same slowdowns. Listing files from the terminal display no slowdowns. When unplugging the phone from the USB port : the slowdowns both disappeared from nemo and thunar. During the slowdown the following error messages appeared :
|
This is still as true now as it was more than two years ago when you wrote it and I could not agree more. |
I'm being very serious when I say I switched to KDE on Manjaro because of this. We use the file manager every day. We rely on it to handle directories with few files or numerous files, and to handle it consistently without mis-clicks due to "populating" the view. Even with thumbnails and extensions disabled (which isn't a "solution" regardless), it's the same issue as it was years ago, as early as 2018, and maybe even earlier. For all intents and purposes, the file manager practically is the operating system when it comes to the end-user interacting with their computer. I'd argue the web browser perhaps more so, but the file manager is a close second. Because of Nemo, and only Nemo, I have left my favorite Linux distro (Mint). I can't believe I reached that point. I migrated my two main computers to Manjaro KDE, and ever since then using my computers have been infinitely less frustrating. I might revisit Mint and Cinnamon in the future one day, but I doubt it. From what I and many others have seen, it appears this is going to be a perennial issue with Nemo. Nemo is simply sluggish. Milliseconds combine into seconds, and when you're dealing with that regularly while using the file browser, it negatively affects workflow. |
|
I now have a clean install of LM 20 Cinnamon with all updates applied. The default 4.8.6. List view, not icon view. No sign of any big slowdown with Nemo. Differences from last post. I have not tested Nemo from a fresh boot before file cache built. |
|
I can't agree with the actual thunar comparison.
It seems so, that the problem has been solved. Thanks a lot! |
|
@flansuse I've tried all of the major DEs and all of the FMs. To me, KDE is unbearable and feels like a Windows knockoff. I just can't get used to it after using it for a full week. Its FM, while good, is lacking many things I like about I currently use |
|
I use |
|
In Mint 20.3, I'm finding that Nemo is generally fast at everything but opening folders. This is something I've tested with all add-ons disabled. When navigating, there's a 0.3-0.5s pause before anything displays whenever I make a navigation. |
|
I'm on Mint 20.3 Cinnamon with its Nemo 5.2.4. I have got a folder of 120,000 small images (up to 570 kB, but most < 40 kB), and I am suffering heavily - even with deactivated thumbnails. I see this on ext4 and btrfs, both on SSD. Also, thumbnails would be quite useful here. My project partner is using Windows 10 with similar (or even older) hardware and has no problems at all with this folder (even with the thumbnails), so he doesn't want to change the directory structure :-/ |
|
If you create your thumbnails over a period of time, they will appear relatively quickly. I wonder why the sub-folder feature was invented? |
Just saying, other file browsers, such as the file browser of Windows 10, have absolutely no issue loading a list of 120,000 files or more. I'm on Windows 10 right now, and if I go to C:\ and search '*', results begin to instantly, and in a couple of seconds exceed 200,000 files. I can scroll through all of them, change between views, from tiled list to max size thumbnails, with what feels like almost no lag at all. Scrolling has an instant response time, and thumbnails for images within the scrolling view appear in less than 1 or 2 seconds. .. About 30 seconds has passed now, and it's showing roughly 600,000 files and again, still no noticeably troubling lag to worry about.. I can drag the scroll bar from the top to the bottom and back again without an issue. Still no issues. If I did this in Nemo, Nemo would have probably frozen and crashed by now, assuming it managed to get up to 600,000 files in the first place. The issue isn't so much that it takes time to generate file thumbnails, the issue is that Nemo can't seem to handle processing the work of displaying a large number of files, in a way that doesn't result in extreme lag, unresponsiveness and eventual crashes. Just looking at the Windows file manager again, and it's now up to about 1,200,000 files, and I can scroll through them with no issues. As I scroll to files such as image files, thumbnails are being generated and loaded for what's on screen very quickly. So something Nemo is doing must be very inefficient if other file managers can handle millions of files, while Nemo (from my own experiences) can barely handle 10,000 before lag and crashes start to become an issue. We should investigate what is causing this lag. Here are just a few questions to put out there to anyone familiar with the codebase of Nemo for possible causes:
(Disclaimer: I support the Linux Mint project, I donate to it, and I love Linux Mint, and Nemo is my preferred file manager for Linux, because I feel the UI is probably the nicest looking. But Nemo's performance is definitely still an issue in Mint, even if it has improved a bit in recent years.) |
|
Are your speed tests immediately after a boot when the file cache is empty or later when there is
stuff in the file cache?
I find Linux slow for file searches the first time then fast after the file cache is loaded. On some
older versions of Windows, the fast initial speed appears to be a background read ahead for
directory and file information. Just a theory.
The test would be to run the same 120,000 file process on Windows and Linux after a clean boot then
repeat the test a couple of times after the file cache is full. I do not have dual boot for a good
comparison.
On my machine, there is about 14 GB of RAM for file cache and that only just fills up with a full
disk file search by name of about 500,000 files. The results will vary if the file cache cannot hold
all the file/directory info.
|
|
Thumbnails are stored in |
|
Hmm - PNG, huh? I never paid attention to that. I see an opportunity of massive performance improvement there. Just tested with a few samples from my While this may not be the core factor to the issue being discussed here, I wonder how much improvement could be harvested from changing the thumbnail format alone. The substantially lower file sizes should mean faster thumbnailing and loading due to reduced disk IO and the lower computational complexity of JPG encoding/decoding. We can also investigate further, I tested with JPG because that's what first came to mind. I should add that, currently, my Is this something you guys are in favor of looking into? I may give it a shot myself, during the week. (Now on a more off-topic note, can someone confirm that the thumbnail folders are themselves not scanned/processed by the thumbnailer when browsed on? i.e. are they blacklisted?) |
|
I run (via cron) a daily command: This deletes any thumbnails more than 12 days old, which keeps my thumbnails down to about 70MB. Provided that you don't have gigantic folders, it's probably worth doing something along these lines occasionally as the cached thumbnail doesn't get removed if the file gets moved/edited(?)/deleted etc. The cached thumbnail folders can be browsed in nemo and viewed with an image program, but even if you request thumbnailing, no thumbnails get created - so in that sense they seem to be blacklisted. |
One downside of JPEG is that it doesn't support transparency. And thumbnails can have transparency backgrounds. One solution would be to use WEBP, it has all the benefits you mentioned of JPEG but with the benefit of transparency support. I recently switched from using PNG to WEBP for thumbnails on one of my web app projects and the file size difference between the two is very significant. As an example, here is a picture of a cow that is 1722 x 2073 with a transparent background (subject matter of picture is not relevant for this example). As a PNG this file is 594.2 kB
As a WEBP this imge is only 92.2 kB with virtually no loss of quality. (I'd upload the image here but github doesn't support that format yet). |
That's because Explorer.exe relies on the "Windows Search" service. You can 100% confirm this by disabling the aforementioned service. ON-TOPIC When I'm dealing with thumbnails, I don't use Nemo anymore, because of this exact issue. I installed and use |
|
Looks like Nemo is trying to determine each file's type by reading their header. This deteriorates performance significantly on non-flash media. Also, an unresponsive MTP device (e.g. smartphone) can cause Nemo to get very slow. When I click on a directory on any device while the MTP device is connected, I see a blank list for half a minute. No high CPU or I/O usage. No indication of what might be wrong. To its credit, Nemo behaves better here than Windows Explorer. Windows Explorer locks up entirely if an MTP device is unresponsive, and since the task bar and desktop are the same process, they become unresponsive as well. |
How else would you do this? |
|
Is the file type in the Linux file cache? Seems like a good place for it after it is determined.
…On 19/6/22 20:39, Jeremy7701 wrote:
Looks like Nemo is trying to determine each file's type by reading their header. This
deteriorates performance significantly on non-flash media.
How else would you do this?
The Windows method of deciding a file type entirely from an "extension" is very flakey.
—
Reply to this email directly, view it on GitHub
<#1907 (comment)>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ACUA7SVDWIZ6KJIW7I6XM5DVP32FJANCNFSM4FLJEVAA>.
You are receiving this because you commented.Message ID:
***@***.***>
|
The file header checking is indeed helpful and indeed more accurate than that of Windows Explorer, but I suggest having an option for temporarily turning it off to improve performance when necessary. |
|
If the mimetype of a file is unknown (which is mostly deduced from the filetype), you won't know what program(s) are capable of processing the file. So switching off this function would be pointless. If you have a desperate need for speed, switching off thumbnailing via the relevant toolbar button would be more effective if you have a very slow PC. |
I think what they meant is: A switch to toggle between determining the filetype by file extension or by reading the file header. Determining filetype by file extension would be accurate for most types of files, most jpegs have a .jpg extension, most text files have a .txt extension, most html files have a .html extension, etc etc. Or it could be a progressive enhancement strategy. The filetype could be guessed by file extension initially, and a background thread could determine the mimetype of files more accurate by reading their header in a non-blocking background task. |
Excellent point. If the file's contents haven't changed, no need to reopen it and read the header again to determine it's filetype. And I imagine most times a folder is open, most of the files in the folder have likely not changed between previous visits. Could save a lot of file reads to simply cache the data. The mimetype itself is a relatively small amount of data to store per file. |
I have already deactivated thumbnails, and indeed, the performance is better.
Sometimes, I don't want to open files, but only manage them (rename, move, etc.). If Nemo did not try to determine file types by header, that would improve performance and reduce wear on a hard disk. |
|
You are trying to second-guess what portions of a hard disk that a file system / driver / firmware / hardware is going to access. Make sure that you are reading all important files when you are taking your regular backups. |
and others
Issue
This issue has been reported prior to Mint 18.
Nemo open folders very slow. slower than Nautilus. much slower than Thunar.
We expected the issue to be resolved in Mint 19, but still hasn't
Steps to reproduce
Open nemo filemanager open some folder with contents.
Especially can be test with /usr/bin directory. While Thunar opens instantly, there is a significant delay with nemo. to browse any folder with low item count still can see blank page then folders showing.
Expected behaviour
instant show of folder contents
Other information
The text was updated successfully, but these errors were encountered: