Take the 2-minute tour ×
Personal Productivity Stack Exchange is a question and answer site for people wanting to improve their personal productivity. It's 100% free, no registration required.

There are some questions similar to this. But i would like to know, what software one should choose, if you go to university for example and are building a knowlegde base of everything you learn & read for your whole coming scientific career?

I pretty much like Tiddlywiki, but alot of small open-source projects die after some years or hardware & software technology changes and the program data cannot be ported? Its a bit unsure.

Also Tiddlywiki is not best solution for knowledge bases getting pretty big, as everything is saved in a .html file. Proprietary knowledge programs like Evernote are imho not very future-proof. Wiki-like programs like TikiWiki need a server, but are probably the most future-proof programs for knowledge managment? How do companies deal with this problem?

Tiddlywiki can already be runned on a android smartphone with Firefox mobile. Solutions like Evernote afaik still have problems on mobile browsers or giving access of to all features of cloud-based service on smartphones

Should one use a software storing knowledge data in any case as XML entries, pictures as SVG?

Many questions and likely no easy answer, my main requirements would be future-proof, big database suited, mobile access via smartphone or tablet, hypertext links and tagging of nodes possible, input of mathematical formula/signs possible

share|improve this question

closed as off-topic by Adam Wuerl Dec 5 '13 at 4:31

  • This question does not appear to be about personal productivity within the scope defined in the help center.
If this question can be reworded to fit the rules in the help center, please edit the question.

    
imo this is a software question and not a productivity question (though very interesting) –  Dmitry Selitskiy Jul 18 '11 at 6:45
3  
@dmitry i thought this site is on everything influencing your productivity NOW & IN FUTURE.Choosing the wrong software here will heavily damage your productivity in future, when you cant port, update, repair a knowledge database. I want to ask more software questions,as most of us work with software, improve software with software (Extensions, Autohotkey,...).If such questions get closed here, the site gets pretty pointless for me working every day in front of a Computer But thats just MY VIEW.Im a bit surprised this question gets not more upvotes,as knowledge managing is crucial for prod. –  Hauser Jul 18 '11 at 11:20
    
I'm merely raising a discussion point rather than blatantly suggesting to close the question right off the bat. As I say I'm personally very interested in this question but I am unsure if it belongs on this site. –  Dmitry Selitskiy Jul 19 '11 at 3:17
2  
lets get back to pen and paper. :-) –  tehnyit Jul 19 '11 at 8:50
1  
Just a short comment about "future-proof". This (making software and information future-proof) is subject of ongoing research in the information preservation community. So many people would like to know the answer but nobody actually does. Text files are cool for storing information. But for tagging and searching you need software. While migrating ASCII files from one system to another is easy, keeping a given software product running from one system to another is not. But by storing information in plain text files you make migrating the data easy. –  xmjx Sep 20 '11 at 11:50

14 Answers 14

Text files are future proof. They will always work. Grep is your friend.

See also TODO.txt.

ASCII is the new PDF! - Cory Doctorow

share|improve this answer
    
thx. i will add-edit this in my post. It has to be possible to make hypertext references/links, i think this is crucial for any big knowledge base. Also tagging would be fine. –  Hauser Jul 14 '11 at 21:03
5  
@Hauser you could write your text files in Markdown (the syntax used on SE sites) and then render them in HTML or a text editor with Markdown preview (different answer, more links) so the links would work. Then you can still grep and even store them in a VCS if you wanted for posterity's sake. –  Adam Wuerl Jul 15 '11 at 19:04
2  
@Hauser you can use a wiki like DokuWiki - its "pages" are simple text files and you still get linking, images uploading and stuff like that. I've used for a long time and no worries, works great (YMMV, of course). –  tomeduarte Jul 15 '11 at 22:41
2  
@adam thx also interesting. I think tagging is a fundamental functionality of any knowlegde system. In a database (im no expert here) you would posteriorly probably have more functions to group specific tagged nodes or sort them? Is this a big advantage of sqlite? @tomedu thx i tried doku and pmwiki, but doesnt work for offline mobile phones afiak? would need a php XAMPP server –  Hauser Jul 16 '11 at 11:39

Like first post said, the only future-proof technology is the ASCII (or UTF-8) text file.

However, you will soon find yourself wanting more. Amongst many others, I have made substantial use of the following knowledgebase systems. Presenting in chronological order of when I started using it and how much I've written in it:

  • Freemind (200,000+ words): XML-based mind maps, used it for studying many of my university courses, great with organising facts for memorisation, converts to HTML/dynamic HTML, but incremental search is terrible for ad-hoc retrieval. Also Freemind development has slowed to a crawl.
  • Emacs nxml-mode (200,000 words): Makes it easy to write XML. With a custom schema and generated outline I wrote several years of diary entries in XML.
  • Emacs org-mode (100,000 words): "Your life in plain text", which is a step up from the plain text files but just as future proof. It uses indented starred lists to signal folding for emacs to display the hierarchy. Better than Freemind at storing multi-paragraph texts. Can "filter" hierarchy in addition to incremental search.
  • TiddlyWiki (10,000 words): the antithesis of "future-proof", its updates and fancy plugins regularly broke my wiki (plugins get left behind). I put a few hundred entries in.
  • Google Docs (10,000 words): I put spreadsheets and large documents here. Great for knowledgebase of documents, like tax returns and contracts - not so great for knowledgebase of facts.
  • Emacs reStructuredText Mode (100,000 words): reST is text-based (future-proof), and standardised (unlike the org-mode formatting). Emacs completions make it easy to work with, converts to pretty HTML. But, you have to be at the desktop, and have the same search disadvantages as other textfile-based knowledgebases.
  • Tomboy notes (15,000 words): moves away from text (.note is XML) but runs on Linux/Windows/Web/Android. Adds built-in synchronisation. Lacks hierarchy, compensates with instantaneous full-text search.
  • Evernote (3,000 words) works on Windows/Web/iOS/Android, fully synced, searchable. Downside: requires trusting your data to a corporation, editing is slowed down by extra clicks.
  • Google Scratchpad (100 words), built on Google Docs, Chrome-only and purely web-based, but convenient and fast from the browser. Lacks features.

Currently I value easy export to standard text-based formats (provides future-proofing), variety ways to access (web, desktop, mobile, offline or online), and good search (incremental is not good enough). Tomboy and Evernote are pretty good on those counts, but not great for structured info. If I went back to studies, I'd probably start using FreeMind and reStructuredText again.

share|improve this answer
    
if you like Freemind you might like to check out FreePlane. It is a fork of Freemind with many interesting features and a very active developer community, where you also can ask for features and see how your suggestions are implemented. I've been using FreePlane for some time now and I like it very much - just give it a try, the "unstable" versions should be quite robust: freeplane.sourceforge.net/wiki/index.php/Main_Page Docear will be based on Freeplane and could also be interesting: docear.org/software/details, especially for use in science. –  MostlyHarmless Jan 19 '12 at 11:41
    
I would never class TiddlyWiki with some of the proprietary solutions you cited, and 'antithesis' seems a very strange thing to say! TiddlyWiki is as future-proof as ASCII -- you can extricate all your text with any editor. I would also like to say that I'm handling 1 million words in a TW (using necessary plugins only). –  Smandoli Apr 15 '14 at 17:46

I started building a while ago a personal knowledge base (kb) with similiar requirements and concerns as yours. After considering a lot of different choices, finally settled with a mediawiki installation and haven't regretted since. I'll fundament my choice considering your requirements.

future-proof

In my point of view, building and practical, future-proof, electronic knowledge base means the choice of:

  • open format, with easy conversion to other formats
  • software that is auditable and that gives some guarantees that is can be maintained
  • considerable user base, which promotes it's expansion

For me a proprietary software doesn't provide the assurances I want. Companies go south, formats change, and data portability is not always a concern. Mediawiki is written in PHP and uses a RDBMS (mainly MySQL or PostgreSQL) to store it's data. The uploaded files are stored on disk. It needs a web server and not much else. This requirements are easily met in any of the major operating systems and IMHO the closest to standard software and formats we can get.

I choose mediawiki because I judge it as a mature software package. It powers wikipedia which gives it an enormous user base, is actively maintained, thoroughly documented and has lots of extensions. I think it is reasonable to presume that, as long as wikipedia exists,it will be maintained and developed. If they ever change platform, it's more than sure that there will be a migration path. If wikipedia ends, or the database used (MySQL, PostgreSQL) is discontinued, there will be plenty of time (and interested parties) to migrate to another platform. As a last ditch, the database could be exported to CSV and the data imported somehow to another system. Too far fetched to practicaly consider. I think that no proprietary system can give this kinds of guarantees, and that's why this was my choice.

Being future-proof also means resistance to hardware failures, natural (or man-made) disasters, tampering, etc. Tools that synchronize to the cloud have it a little easier. With a setup like this, the future-proofness encompasses a backup plan, which in any case should already be an existing practice if you plan to devote a lot of work in building a knowledge base. Backing up a mediawiki installation is as simple as making a dump of the database and backing up the mediawiki root directory (which also contains the uploaded artifacts). After this it can be compressed and/or encrypted for size reduction and security, transfered by a network to a remote location, copied to a portable media for offsite archiving, etc. Define and automate a procedure like this, and you won't risk loosing your information.

Any stricter requirements of future-proofness would require considering an text-only representation (which btw can more or less be obtained from the database) or a copy on paper, which continues to be the most realiable knowledge medium, but I guess this scenarios are more or less out of scope for their limitations and impracticality.

big database suited

Since the datastore is a RDBMS (MySQL, PostgreSQL and partial Oracle and Sqlite support), the database can grow pretty large and still be accessed/updated quite efficiently. The Semantic Mediawiki extension discussed below only supports (at least when I installed it) MySQL and PostgreSQL.

hypertext links

Core wiki functionality

mobile access

Assuming you have network access from your mobile device (in contrast with running the software in the device), the data is accessible from anywhere. AFAIK, the wikipedia pages render nicely on all browsers, mobile included.

tagging

I've been using the wonderful Semantic Mediawiki (SMW) extension. This extension enpowers the mediawiki software with semantic web features, enabling the definition of semantic content. This allows the creation of dynamic content (pages or page fragments) using the SMW semantic query language and allows the kb to became a data source that can be consumed by other systems. As a very trivial example, I use the wiki to keep my food recipes. Since I annotate semantically the recipes, I can create dynamic pages that list all the recipes with ingredient X, or all the recipes which are desserts, etc. Another thing I use it for is to enrich my vocabulary. I regularly pick up new words from the dictionary and try to introduce them into my day-to-day conversation. Everytime a new word is chosen, I create a page in the kb and annotate it semantically (verb, adverb, etc... masculine, feminine), which allows me to produce the previously mentioned list pages. I also create other annotations, like relating two homophone words, etc. Since I like to learn new languages, the next step will be, at least in this vocabulary aspect, to expand my mediawiki so that when I learn a new word in my native language, I can look it in the other languages that I speak, create pages for them, and associate them semantically, more or less like the wikipedia does for the articles in different languages. I don't know yet the best way to implement this. I also have a limited knowledge of the SMW. As I learn more, I'll try to expand my kb with the facilities it provides.

Hope this information can be useful. For me this is still a work in progress and I intend to expand on this as I learn new approaches.

share|improve this answer

Since text is a popular answer for this question, I would suggest Emacs, with Org-Mode. It also uses plain text as it's backend but has a lot of features. Links, tags, todo lists, time tracking, tables, basic spreadsheets, publishing/exporting to html and or pdf and support for referencing and embedding source code. Synchronyzing Org-Mode's text files with something like Dropbox gives you a cross-platform always available knowledge base I would argue.

share|improve this answer

There are 2 ways to do this:

  • choose a solution that will last forever
  • accept that you will need to do a conversion every 5-10 years

The first will probably have a much lower functionality than the second.

Choose a solution that works best for you now. Make sure that it has "export information" functionality, that you can use when converting to next solution.

share|improve this answer
1  
I think you are 100% right. I think I can live with evernote's export, but that it has been there is critical, I probably should dump it every few months or once a year, all formats, just in case. –  Ronald Pottol Jul 29 '11 at 0:54

Emacs org-mode is probably the way to go. It's everything and the kitchen sink, including:

  • Stores all data in 100% future-proof plain text files
  • Allows easy manipulation, linking, and sorting of text.
  • Emacs is cross-platform, so put your org files in a dropbox folder and work on them everywhere.
  • Runs locally, so you don't need a server running php as with most wiki platforms.
  • Export to html, latex, and other formats
  • Supports document linking and attachments so you can easily move between documents
  • Great outlining tools
  • Intelligently handles your tasks and deadlines with org-agenda.
  • Compatible with GTD methodology
  • Timetracking with org-timer.
  • and so much more! It's fully customizable and extensible.

The org-mode project was started by Carsten Dominik, a scientist at University of Amsterdam, in the early 2000s because he found that none of the available proprietary options for task and knowledge management suited his needs and working style. That many tasks begin as notes is central to his philosophy in developing org-mode.

You can see him explaining how he uses Org-mode in a Google tech talk here.

I'm still scratching the surface of what it can do, but I just wanted to add my voice to those who've already mentioned it. I've even heard of vim users being lured to emacs because of Org, so you know it must be good. :)

share|improve this answer

If your technically inclined take a look at Fossil It is a single executible that provides a Wiki as well as source code control. While you may not need to manage source code you can use the source control aspect of the program to help organize electronic documents and electronic reference meterial. Backing up is simple since the repository is a single SQLite file. SQLite is a very popular database so even if Fossil is no longer actively maintained your database will still be readable. Fossil also offers the ability to export the repository into text format as well as exporting the Wiki pages.

share|improve this answer
1  
How would one compare future-proofness of text file & sqlite? text files are likely 100% future-proof, squlite 90% (?) as Firefox and many mobile OS use it? It looks to me better for node taking, as you have tags, node-text, links of nodes all in a handy table. Text files look to me still like the most future-proof solution, but sqlite maybe much more handy with more function for grouping, sorting? Maybe there are tools in some years to visualize a database like a 3D mind map. Looks like the best compromise in sense of functions, compatibility and future-proofness? –  Hauser Jul 16 '11 at 11:39
    
Future proof depends on the aplication you use to access the data stored in SQLite. For example Fossil allows you to export your wiki pages as text. If you felt like fossil development was not proceeding at a good enough pase you could export all your data to text and import it into another Wiki. Fossil will also let you export the source code portion of a repository into Git. While it's impossible to say what source control systems will be around in 20 years Git is popular enough that I assume there will be a migration path off Git onto what ever is next. –  Jared Jul 18 '11 at 12:26

Evernote is future proof when it comes to text/images. You can export the data into html or xml. If Evernote gets replaced by a better tool someone will write a parser to convert the Evernote data into the new format.

share|improve this answer
1  
im not totally convinced by proprietary software. Google had a app called Google Notebook, which is now not further developed afaik. I also dont know of a parser porting it onto another knowledge managment software...i think open source like tiddlywiki, docbook, tikiwiki is much more future-proof here. –  Hauser Jul 15 '11 at 15:55
    
There was a parser for converting Google Notebook files into Evernote when Google Notebook shut down. There's nothing that makes open source software inherently future proof. Someone has to support it. Supporting software is much more work than writing a simple parser. Maybe the parser will have a problem with handwritten notes but writing a parser that just get the text doesn't even take a day of work. TidiWiki won't give you handwritten notes anyway. –  Christian Jul 15 '11 at 18:15

I would like to second @mgois's interest in MediaWiki. What is more future-proof than WikiPedia?

But for me, MediaWiki is just a sophisticated front-end for MySQL, and it's more advanced fork, MariaDB. It takes a bit of work to get up to speed with SQL, but once you do, it's pretty amazing. I have 2GB worth of email, dating back to 1992, archived and searchable in MySQL. And with the power of SQL, I can link and embed that email in other data points of interest.

Other things of interest I have in there: vehicle logs going back to 1989, my personal weight log going back to 1982, and all the seeds I've purchased for our farm, with cross-referenced information from the Plants For A Future database, which (for a nominal fee) can be downloaded and imported into MySQL.

A simple MediaWiki extension allows you to run MySQL queries and display the results. And of course, all the stuff that you can do in WikiPedia is there, as well -- catalogs of images, formula and equations, simple and efficient categorization, etc.

In short, MediaWiki/MySQL is my personal ultimate "big knowledgebase." And we use it for our co-op's enterprise resource planning system, as well, tracking labour, harvest, sales, projects, etc.

I hear and respect the arguments for ASCII text. But it will only take you so far. The next step up from there -- at least for me -- is MediaWiki/MySQL.

share|improve this answer

I highly recommend Workflowy.com. It's a web service that is essentially a single page that contains a list of text items. The thing is that the list can be of unlimited depth (it can contain tens of thousands of text items and more). So you can literally outline your whole life there, starting with big sections (projects, thoughts, knowledge base etc.) and zoom to particular tasks, thoughts, notes etc.

I was seeking for a tool to 'contain my whole life' for years. First I used Google docs, text files and mind maps. All of them were good, but they have big unnecessary overheads. I don't think of my information as a number of 'files', so the concept of file does not fit. Mindmaps are great, but they are not easy to use, and I can't easily navigate and search hundreds of mindmaps (and a single huge mindmap is too slow).

Then I found Google Wave. It was great, and I used it for a year (having 1000+ waves). It worked, but then the project was closed.

After all, I found Workflowy. And only then I fully realized all of the limitations of the previous tools. When you search in google wave (or google docs), you first find the wave (or file), and then the line in it. In Workflowy, there is no such overhead, since there are no files. You search instantly the whole list. It saves a lot of time. Second, you can focus on any item (subtree) at any depth level, and all other items are hidden. Third, any particular item can be shared, and you can collaborate on it (the remaining list remains private). Workflowy is very fast and easy to use. There are (hash) tags. And it's future-proof, because at any time you can export the whole list as a huge plain text file, and you can search through it, put it in dropbox etc.

The only big (temporary) limitation of it for me is that there is no native app for Android (for iOS it's been already created), so you have to use the mobile browser version, which is good, but not awesome.

I've been using Workflowy for 1 year (20k+ items): for project management, for personal notes (knowledge base), as a diary, for personal task management, for collaboration. For now, I can't think of a better organizational tool.

Short introductory videos:

share|improve this answer

I have been using JspWiki for the last 5 years. Each page is stored as a text file.

Last year I put my local wiki files into a Dropbox folder and now I can at least see my pages also on my mobile phone.

share|improve this answer

If you just want to dump your thought -- Evernote. It is big enough that even if they go bust someone will come up with something to let you continue using the data. I'd still take offline backup of your Evernote files though -- if your notes are deleted in the cloud, the deletion will be applied to your local copy during the sync operation.

If you want to be able to link like in a Wiki -- Microsoft OneNote. Extensive editing feature and almost guaranteed to be future-proof (you can always open old Office files in the new version) and cloud editing too. But they're not very mobile -- almost unusable on iPad/iPhone/Android or Mac. Also, make local backups of your data.

Stay away from all-cloud solutions. As someone already mentioned -- Google Notebook was killed when Google decided to change their corporate priorities. (unlike EverNote or Microsoft who have more motives to keep their products updated)

share|improve this answer

I've been using Evernote for a long time (20,000+ items now stored in 50+ .enb files) but never upgraded past the v2.2 version because lots of my stuff isn't appropriate for the cloud, plus they never IMO got hierarchical tagging right with the cloud-based versions.

The backend storage is standard XML, and there is a flexible export facility, the 2.2 version is still downloadable, including as a "portable" apps (runs with no installation from a changing-drive letter, e.g. flash drive), and the matching "Universal Clipper" works fine in Windows 7.

I'm dreading the day I am forced to export/convert, but this apps' functionality is so excellent and easy out of the box I'll stick with it as long as I can, but have started researching/planning/experimenting with the following in anticipation of the future, perhaps will start migrating some of the more current knowledge domains over sooner rather than later if I get a toolchain that I think will be more future-proof.

===========

Note toolchain - I don't expect a single app to give me what I want, since the only truly future-proof file format is plain text.

I prefer txt2tags to markdown for inline formatting markup. Once a given set of plain text files has grown to the point where I want more structure, I can easily convert to asciidoc, which offers all the functionality of the Docbook format, including tagging (index terms) navigation (Table of contents), glossary of terms, cross-referencing (see also).

Docbook is most suitable for "canonical" storage format for organized writing, definitely as full-featured and as future-proof as it gets at the moment - it is O'Reilly press' standard, and has been adopted as the "archive format" for many large publishing houses. It also outputs to HTML, HTML-help, PDF, LaTeX, EPUB and just about anything else if you're willing to get into XML/XSLT.

Otherwise it is a very daunting techie learning curve, but if you keep Asciidoc as the "master" storage format, that's very manageable as is, just plain text. Plus you can keep your docs under version control.

When I have more time, I'd like to explore the different Wiki systems - the ones that claim to allow you to just store text as text in a file system rather than using a database seem promising, as long as the data files can still be read and written to from outside the Wiki interface - I'm not sure if Dokuwiki or any of the others actually allow that.

share|improve this answer
    
thanks, I know docbook, unfortunately no tagging. Its imho fine for Documentation, but not a knowledge base you want to quickly edit and search and watch even on mobile device. Take a look at PMWiki (popular php-Wiki text file based) –  Hauser Nov 17 '11 at 13:04
    
Actually "tagging" is just good-old index terms, and in any reasonably complex topic domain, having a traditional hierarchy of term/sub-term is very helpful. The only newfangled tweak is finding a way to show the "backlinks" (what terms point to to this text), either inline, in the margins, bottom of the "page" etc. –  HansBKK Nov 22 '11 at 10:54
    
I've been looking at DokuWiki recently in preference to PMwiki, as it stores all the "meta" stuff in a separate dirstruc from the items' content, which it leaves 100% untouched in the filesystem as plain text, and uses the dirstruc hierarchy for its "namespace" feature. You can manipulate (create/delete/edit/rename) the source files to your hearts content, tell DokuWiki to rebuild and badaboom badabing everything's back in your wiki (I don't use its history feature, have my text files under VCS anyway.) –  HansBKK Nov 22 '11 at 10:55
    
And talk about "tagging on steroids", have a look at DokuWiki's SubjectIndex plugin dokuwiki.org/plugin:subjectindex –  HansBKK Nov 22 '11 at 10:55

I'm a big fan of MediaWiki (a little hefty for a Wiki, but it's well-supported and open-source, plus I already have a dedicated webserver).

On my personal computer, I keep track of notebooks using Basket Notepad which combines note taking with tagging, hierarchy and if so inclined, custom spatial orientation. Again, open-source, cross-platform and can be synced with Dropbox. Note files are HTML and supports linking to/copying in system files. This software has served as my lab notebook for 3+ months now and I'm very happy with organization and looking up historical entries.

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.