Sunday, 25 February 2007

Introducing the new Wikipedia TomeRaider file.

Hooray!

We have just finished the first cut of the new Wikipedia TR3 file and it really rocks. A few years ago Eric Zachte converted the entire Wikipedia text to TR2 and then to TR3. You can still download or get on CD the files from Eriks website (see below).


But as the wiki grows it becomes more and more cumbersome to handle and the returns of relevance diminish with this growth, it is also now over that SD important 1 gigabyte mark. So, last month we made a decision which is to supply regular cuts of the full Wikipedia text but only the introductory paragraph and then a link to the online article for those who want to read.

It works really well:

  • The files is less than 250 megs so it will fit on most SD cards.
  • It is super fast to browse and index search, even with 1.3+ million entries.

There are some text errors in the file, and to be honest there probably always will be – that’s a drawback of a content source written by so many people. But we will be doing regular updates and constantly removing glitches ion the text. If you find and “significant repeaters” then drop us an email at the Yadabyte support address.

We currently have these cuts of the Wikipedia data, all available for immediate download:

  1. Wikipedia (226 MB) - This is the new file which holds the first paragraph of the articles, all 1.3+ million of them. For most people this will be the most useful. (File Version 01)
  2. Wikipedia Complete (1.1 GB +) - various cuts made by Erik Zachte. Its, contains the full wikipedia with full articiles
  3. Wikipedia Compact (49MB) - For this one we took a number of indexes from various encylopedia sources, merged them and then compared against the wikipedia. Close to a traditional encyclopedia in terms of scope.
Download TomeRaider for Pocket PC, palm OS, WM5, Windows etc at www.tomeraider.com
Email support issues to support at www.yadabyte.com.

Thanks:)

25 comments:

Hedy said...

When I cliked to download the file, I keep getting redirected to the same page.

mstpc said...

None of the download links work. I'd really love to try this again. Please fix the links!

Also, three suggestions:

1) 2 gb SD cards are less than $20 shipped at places like newegg.com and buy.com, so I can't see why most people wouldn't want the full text version. (at least until the picture/graphic version is updated, see below)

2) I did try TR3 briefly and had one major complaint that was a deal-breaker for me. If it hasn't been fixed already, tops on your priority list (besides fixing the download links) should be making a version that scrolls up and down with the "up" and "down" buttons of a handheld. When I'd tried TR3 on my Axim, it was the one thing that drove me nuts about the software -- you had these wonderfully comprehensive articles on Wikipedia in the palm of your hand, yet to scroll down through them you couldn't just hit the "down" button with your thumb -- you had to continually tap the scrollbar on the screen to see the rest of the article. It would be so much easier if one-handed operation were supported via the hardware buttons that are built-in to every handheld!

3) Last but not least, I'd really like to see an updated wikipedia version with pictures and graphics. There is one out there, but it's from Dec 2005. Even though it would need an 8gb card, SD and CF 8GB cards are less than 90 dollars shipped now. From what I can gather, the decision to update this version is not in tomeraider's hands, but FWIW, myself and a bunch of my friends would really like to see a 2007 version as soon as is realistically possible.

mstpc said...

I did a little searching and found the page to download the various versions of wikipedia for PocketPC's and Palms, here it is:

snipurl.com/tomewiki
(Handheld Wikipedias)

Peter said...
This post has been removed by the author.
Mat Ripley said...

Hi Folks, especially MSPC.


Regarding your points:

1) The full text version is available. Its not made by us however.
2) We use left and right to move through the article list and up and down to go through the hyperlinks. Hows this?
3) So would we. Adding the images exponentially adds to the work. But your right, a new once would be great.

We are uploading the latest as I speak.

Thanks

Mat Ripley

Juan Manuel said...

I'd also support 100% a new, full text + images version, as stated by mstpc

In dec-2005 Erik Zachte already offered a 4GB version for 21 euros, the size of the images being 275 pixels, so and update of the wikipedia for PPC would require 8GB or maybe a 4GB version with smaller images (130 pixels?)

the problem is, Erik seems to be unable to do that process right now because he's working in several projects, and even though he stated the dec05 version wasn't that old in comparison with new editions of paper encyclopedias (2 years), I think it'd be about time to get an up-to-date version :-)

will be waiting for your news

Hamish MacDonald said...

I'm a registered user of TomeRaider, and I have to say that the new beta is a very nice improvement. The interface is very clean, and it's much more straightforward navigating around the Wikipedia now (the search bar used to keep disappearing on me before, and clicking on the button wouldn't bring it back). Thanks!

Paul said...

I would as a minimum want the full text. I can live without graphics but the text is important to me. I have a 4GB card and these are so cheap now so there's no pressure to keep it down to 1GB anymore.

michael said...

What a shame. I'm still waiting for the bug fixing of copying text function. It used to work on my old WM2003 PDA, but not on my new WM5 PDA Phone.

wikipedialover said...

I'm really sad to see that we no longer have the full version of wikipedia (with no cut-down of text or image). I wanted the full version so badly that I bought a PDA with 12 GB storage (Dopod U1000) and is just about to order the full image full text version but see that you only have a condensed version.

Can you bring back the full version with full image please?

Paul said...

Get the version with the whole articles. It's only ~5x bigger than the abstracts only version but is a hundred times as usefull. For example, if you are looking up Mel Gibson, then you probably already now he is an actor, which is basically all the abstract will tell you. Sometimes the first paragraph only clarifies something (ex. it has been suggested that this page should be joined with another, telling you nothing about the entry). In conclusion, don't download the 250 MiB abstract version, for the dozen or so people I know that have (including myself) ended up deleting it due of its uselessness (it's not even worth burning onto a 5 cent CD).

RealEstate said...

I love this !!! I use it on my iPaq 4155 all the time...

* where can I get the very latest release?

* I would very much want to see a version with images...

* up to which date doest the "WP_EN_(P)PC_TXT.tr3 2006-Oct-13 20:24:21" include? I see that some articles added even in May 2006 are not included :( ( example http://en.wikipedia.org/wiki/Magnolia_Bakery )

Thank you all!

Douglas said...

Much thanks for compiling/posting the .tr3 files. Unfortunately I'm having a problem with the Tomeraider text search (text box on lower-right) and finding any article beginning with the letter "b". All others work great. Any idea of the cause of the problem? A manual scroll will allow me to see the "b" articles.

Francisco said...

But these files are like a year old!!!! Is not there a more recent version?

Thanks.

David said...

Is there any way that the database can be updated? The main file is almost a year old.

Now that 4GB, 8GB cards have dropped in price, there are a lot of people who would enjoy having all of the articles (pictures too, if possible) available in the .tr3 file.

Thanks.

Paul said...

Its been 6 months without an update now. This is very disappointing. I only purchased TR3 to get Wikipedia. I'm feeling like I wasted $38.

I feel that Wikipedia should be updated at least once a quarter (because it changes so rapidly) and a range of sizes should be offered for those of us with large (4GB+) memory cards.

vertigo said...

+1 to having different versions for different people, including full-text and full-text with pictures versions. I use wikipedia so much and find it so incredibly useful that I would definitely buy a card large enough to store it if it were made available. Also, it really should be updated more often, at the very least twice annually, if not quarterly. Finally, I also noticed the bug with searching the b's, a problem which was reported almost 4 months ago. As another poster said, my only reason for purchasing TomeRaider (I haven't yet) is for Wikipedia, but I am not going to spend/waste the money for a product that is so limited (lack of full-pictures version), out of date, and bugged, not to mention the fact that it's been 4 months and the bug still has not been corrected. I might be able to deal with one of these issues, and be willing to spend the money despite it, but not all three.

vertigo said...

Also, back and forward buttons and a history of more than the last one or two items would be great. And I don't know if I'm missing something, but if you go into the Find screen, there's no way to start the search or even get back to the article, you have to completely exit out of the book. Finally, it keeps reverting back to taking up only half the width of the screen, and I have to open then close the typing box to make it take up the full screen. This may be due to the fact that I have a Tapwave Zodiac, so I view it in widescreen, which is the equivalent of landscape mode on a normal palm, and so I'm not sure anything could really be done about it.

Daithi said...

I'm waiting for a 2008 update to your wikipedia tr3 file before i purchase tomeraider. As is anyone i know. Its a deal breaker that your version of wikipedia is very old now.

djpao said...

tengo casi intala en mi palm os


www.palmos.net.tf

mitchelln said...

I've just paid to download the March 2008 version from Tomeraider.

It seem to me to be an increment of on Erik Zachte's free version.

A very large proportion of the articles are identical to Erick's even though the Wikipedia articles from around March 2008 are substantially different.

So is the 2008 version a true new snapshot of Wikipedia or an increment of on Erik's (i.e.only new articles added)?
You should make this clear as it will disappoint people like me.

Surinder Rinku said...

When will we have the 3.7 Gb version of wiki for palm users? We see it listed for windows but the site says that the palm version would be out soon.

mitchelln said...

Unfortunately it all seems to have gone quiet at Yadabyte. My support emails have gone unanswered as well :(

Surinder Rinku said...

Mine too

buy wow gold said...

When Wow Gold wolf finally found the wow gold cheap hole in the chimney he crawled cheap wow gold down and KERSPLASH right into that kettle of water and that was cheapest wow gold the end of his troubles with the big bad wolf.
game4power.
The next day the Buy Wow Goldlittle pig invited hisbuy gold wow mother over . She said "You see it is just as mygamegoldI told you. The way to get along in the world is to do world of warcraft gold things as well as you can." Fortunately for that little pig, he buy cheap wow gold learned that lesson. And he just k4gold lived happily ever after!.