libtracker-miner
So Carlos and I have been working on libtracker-miner for the last few months. Since tracker-store (formerly known as trackerd) is now handling all reads/writes from/to database and doing it much faster than ever before with a much more expressive language to query with (SPARQL), we had to merge the old tracker-indexer and parts of trackerd from the 0.6 branch into one binary that could crawl the file system, insert file specific metadata and call tracker-extract for file type metadata (for example: none “file” data, but actually data like image height, width, etc.).
As we had to do this anyway, we took the opportunity to refactor the parts we were unhappy with and to make libtracker-miner a library which other “data miners” could use. This gives the following things:
- DBus integration for free
- An API to find other miners both available and running
- An API to get/set status, progress, name and description for each miner
- An API to pause/resume each miner
- Signals to know when all miners or specific miners start/stop/pause/resume/error/progress.
More recently, Adrien Bustany has been working on “bridges”, which in fact are the same principle, they are miners of data but for web applications like:
- Flickr
- etc.
We are working together to integrate this into the “miner” framework we already have set up in master right now and it is quite exciting to see integration in other areas than just desktop applications.
Additionally, Philip is making Evolution use the same miner API so we will have support for 3 miners as standard out of the box for:
- Email data
- File data
- Application data
tracker-status-icon
Formerly known as tracker-applet, this has been refactored by Carlos recently to work with the new miner API too, so now you can see (much like the network manager) a list of miners and their state/progress. It also allows pausing/resuming of ALL or single miners at a time which is very useful.
tracker-preferences
The tracker-preferences application was also really out of date. The whole configuration system has changed since 0.6 so we decided to use Vala and GtkBuilder to build the new dialog. This dialog only services tracker-miner-fs preferences right now because they are really the only settings that make any difference to the user at this point. There is some polish that is needed here, but it looks good so far:
0.7 Development Release
The current roadmap is mostly done now with a few exceptions which we have decided to not worry about for the 0.7 release. Next Friday we plan on doing this release now that most of the UIs are in reasonable states and people should be able to start using it normally now all the big features have been integrated. This has been put off by 2 weeks already but we don’t want to delay any further. So look out for a new version of Tracker next week!
Comments are closed.
Sounds like a great update for tracker! Looking forward to it.
Thank you very much.
Great stuff!
The wording in the dialog is somewhat minimalistic and some of the options are not obvious.
Some suggestions:
* Delay [15] minutes could use a more verbose description.
* “enable thumbnails” could be renamed to something like “generate thumbnails”. Enable is pretty much a passive verb.
* Enable when running on battery -> “index also when running on battery”
* Not sure what is the effect of the “for first time”
* Include mounted – index also mounted directories, perhaps? Same for removable media
* The disc space limit in percentage requires me to think how much disk space do i have and if i can really put a number on the thing. I mean – i don’t really care
* What is the effect of “throttle”? I sure want indexing to be faster, but will it cost me anything? Maybe the oposing sides could be set “faster indexing” and “lesser use of cpu” or what are the consequences of faster throttle? Throttle could be also renamed to something more meaningfull.
Keep rocking!
Can you tell something about the experience of using Vala? Was it good, were you more productive with it?
[…] that Martyn has updated his blog with some sweet tracker info, I figured I could do the […]
Why is there an option to delay startup time? If the default value is a problem, perhaps it should be increased or is there some more intelligent heuristic available? It seems like an awkward question, considering how it (IIRC) it does other things (niceness, IONice?) to not get noticed by the user.
I know someone mentioned on the discussion that you don’t want to do a simple search bar. But really, that’s what helps an actual user. Also, it might actually
Ofcourse i’m just complaining and not doing any work
Too much in pref dialog. Example: How can I figure out what i shouldn’t index if i need a search tool to find it? You need a very good overview(=expert) of your system to do that + I have to remember it for future use). I guess it might be needed as last resort for users, but it does remove the motive/need to fix the underlying problem.
Complete an actual usercase, ie. just search and make it work. No web 2.0 integration. no separate search gui window that provides all kinds of stuff. A simple search bar is harder because it’s simple. Relevance becomes very important.
Just a simple search bar that’s lightning fast.
PS: Make it possible(easy) for distro’s to ship a preindexed install) This way first time use doesn’t suck ;).
PPS: An important case is to minimize time between recently saved files and when it’s possible to search for it. (Ie, downloads).
Gabor was kind enough to ask translators on gnome-i18n@ to update the files in/po, but for future reference It would be better if the tracker team could do this themselves. Following the GNOME release schedule is another thing I’d like to see but Vincent mentioned that already on d-d-l.
aklapper: Which files are you referring to? the POTFILES.{in|skip} or the .po files? If you mean the .po files, what command should we be using there. I was under the impression only translators should be doing that sort of thing.
Robin: I think I could be, but it is just a case of getting used to it. If you have had to learn python, it is just as quick as that, maybe even quicker if you know C already.
Adam: Because otherwise, when the desktop is starting up, tracker starts mining your hard disk and slows down your UIs loading, etc. This is purely to not start until the rest of your system is expected to be up and running. I agree with your point about nice/ioprio/etc, but boot time decreases are often noticed and really we would rather wait until after then anyway. There might be some other reason I am forgetting here – this is a legacy option after all
Andreas: Our pleasure
Frej: Hej, the preferences dialog does need an update actually, I recently added the XDG special user directories to tracker-miner-fs – so you can specify VIDEOS or PICTURES or DOCUMENTS in the config and it will find out where those are kept (based on the XDG standard). The UI needs a bit of love here to add some obvious buttons like “Index pictures”, “Index music”, etc. Also web integration is coming, Code think have a GSoC student working on that right now for integration with facebook, flickr and others.
tm: Hi, yea we have improved this over the last few weeks. Now we have tooltips to better explain some of the options. I am not a big fan of long options explained to the nth degree because the clutter up the dialog. Also, they are only useful for the first time you use the dialog when you don’t know what options do. So I sided with tooltips
I agree with your idea about the thumbnail rename too. The “for first time” is about indexing while on battery for the first time. This is essentially an exception for new installations. The distinction between mounted directories and removable media is made in the tooltips now. They are not the same. The throttling now has a tooltip too – the effect is simply that we don’t use so much processing power – which might strangle other processes running on your machine. This shouldn’t need to be changed, but on some older or slower machines it may be necessary. I think you’re right about the naming there “Lesser use of CPU” is much better. Thanks for the suggestions!