From Evernote to ZimWiki via Python

For many years I’ve struggled with finding the best way to collect and store knowledge…from articles I read online to what I learn from PyBites code challenges.

I was a fan of Evernote for several years. There is no beating it for convenience: a random thought, an article on the web, a photo of a brochure can be quickly saved anytime and anywhere with a tap or two. It has an extremely powerful search system, including searching for text within documents and PDF files. Its structure consisting of notebooks, sub-notebooks and tags is extremely flexible, allowing a user to come up with almost any organizational system they please. It utilizes the cloud but unlike Google Keep, it is not cloud dependent; the desktop app syncs all content to a local database that is fully accessible and usable without internet access.

Despite all these benefits, in the past couple of years I have become discontented with Evernote, for the following reasons:

  • Its rising annual subscription cost, which doubled from $35 in 2017 to $70 this year. This is in addition to the $99 per year I am already paying for Dropbox to do much of the same thing: sync my content online.
  • I have found that Evernote’s convenience has almost become a liability. It’s so easy to impulsively save articles and such that I quickly forget what I have because it gets lost among my 3000+ notes. If I have completely forgotten an article,including its title, content and subject matter, and it’s buried among thousands of notes and tens of notebooks, it might as well not be there.
  • Searching Evernote using the Windows desktop app is slow. Typing a search term such as intitle: (for searches in note titles) causes the app to freeze for several seconds before I even finish typing the word.
  • All content, including attachments such as PDFs, is all squashed together in a proprietary format called exb that only the Evernote app can access. It is possible to export notes out of Evernote into a browser-friendly set of HTML pages, but only as an “index” page with one long list of randomly sorted links to all my notes; exporting wipes out my carefully crafted notebook and tag hierarchy.
  • The Evernote Web Clipper is unreliable, especially on mobile devices.  Sometimes it can’t save a page at all, other times it gets the text and images jumbled so they are unreadable on my phone.  I have found I prefer “cleaning up” web articles using the Mercury Reader plugin for Chrome and printing to PDF.  Yes, this requires more steps, but it also acts as a natural filter of trivial content, because I wouldn’t go through those steps unless it’s an article I feel is worth the effort.  I have also found it acceptable to just copy and paste the URL into Dropbox, if I’m using my phone, then print the article to PDF when I get home to my laptop.

In a previous blog post I demonstrated another attempt to store what I learn. First, I would take notes in Evernote on what I learned from projects such as Bites of Py. Next, I would add what learned to a “knowledge base” in ZimWiki. This worked well at first, then I realized it was cumbersome to copy and paste between two different applications.

Next, I experimented with writing notes in ReStructuredText and exporting to HTML using Sphinx, whose main advantage is easy, free hosting of content at ReadTheDocs. I soon discovered the downside of using Sphinx like this is that I have to export frequently in order to keep that nice web version of my notes up to date.

Now, at last, I have concluded that ZimWiki is an acceptable replacement for Evernote. Here’s why…

  • Zim stores notes in plain text, in a logical file and folder system, that does not require Zim to read and edit. Any text editor will do, including Dropbox’s built in text editor.
  • I can store my Zim notebook in Dropbox and read, edit and even search my notes using the Dropbox Android app. They are not attractive looking when viewed outside Zim but they are 100% usable on my mobile phone despite the fact that there’s no Android ap for Zim (maybe I’ll be able to write one someday!)
  • As I shared in my post, I used Zim for my Python “knowledgebase” because provides the browseable, ‘birds eye’ view of my notes and topics that I’ve been looking for, kind of like a table of contents.
  • Zim is open source and cross platform. It works on Windows, Linux and Mac.
  • Like Evernote, Zim uses tags, but unlike Evernote, tags can be used to tag specific content within a note, not just the note as a whole.
  • Zim keeps things simple. It’s basically a text file creator, organizer and viewer. It does not try to be a photo editor, a PDF reader, a workplace collaboration app or a presentation tool. For non text documents such as PDFs, it makes it easy to create linked notes for these documents. Clicking on the link within the note opens the documents in my operating system’s default applications. Zim lets other applications do their jobs, while it focuses on doing its job. This simplicity makes Zim faster and more responsive than Evernote.

Therefore, as of now, I am transitioning my notetaking and knowledge organizing from Evernote to ZimWiki.

Oh, and one more thing….ZimWiki is written in Python!

Here is an application that is not only incredibly useful to me on a daily basis, but is also an open source project I hope to one day contribute to. In other words, the tool I use to store knowledge can itself help me improve my knowledge of Python! (is that an example of recursion?).

My number one goal now, naturally, is to writing Python scripts to get all my Evernote content (almost 3000 notes!) into ZimWiki. I have already written code that grabs saved PDF documents from an “inbox” and moves them into Zim’s folder structure, automatically creating a note for each document that links to the document and can serve as a place to write notes on the document’s content. For example, here’s the note I created for Jason Wattier’s inspiring article about PyCon, as viewed in Zim Wiki:

Clicking on the link brings up the article in a PDF reader:

Here’s an example of how Zim organizes PDFs. There is the note ABOUT the article (Crack_the_Code_of_PyCon-Wattier.txt) which sits in the “Python” folder. This note contains my comments on the article, and a link to the article, which is stored in a folder of its own. Note the folder title is the same as the filename of the note (without the *.txt extension):

Here’s the actual article PDF in its folder:

This arrangement of the note, the folder and the target PDF is the way Zim organizes PDFs and links to them. The code I wrote (see above) took the file “Crack the Code of PyCon-Wattier” and moved it into its own folder, and created the note that links to it. ZimWiki’s indexing system updated, making the new note visible and the link clickable. I added my comments on the article to the note afterward.

My current project is to write a Python app to Evernote export files, which look like this in a text editor:

?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE en-export SYSTEM "http://xml.evernote.com/pub/evernote-export2.dtd">
<en-export export-date="20181007T111836Z" application="Evernote/Windows" version="6.x">
<note><title>2018-08-02-2043 Bite 006 PyBites Die Hard</title><content><![CDATA[<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE en-note SYSTEM "http://xml.evernote.com/pub/enml2.dtd">
<en-note>
<div><span style="font-weight: bold;">Requirements</span></div>
<div><span style="font-size: 15.4px; letter-spacing: normal;
orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px;
.
(and so on)
.
</content><created>20180803T004348Z</created><updated>20181007T111318Z</updated><note-attributes><author>Rob Fowler</author>
<source>desktop.win</source><source-url>https://codechalleng.es/bites/6/</source-url><source-application>evernote.win32</source-application></note-attributes></note></en-export>

to Zim Wiki notes, which look like this in a text editor:

Content-Type: text/x-zim-wiki
Wiki-Format: zim 0.4
Creation-Date: 2018-10-07T07:13:45-04:00

====== 2018-08-02 Bite 06 PyBites Die Hard ======
2018-08-02

===== Requirements =====
Given a listing of files of our community branch determine who PR'd (= submitted pull request) the most (excluding PyBites) and what challenge is
the most popular (PR'd) as per snapshot of today (8th of Dec 2017). See preparation done in the code template below. Replace ''pass'' with your code
to make the test pass. Good luck and have fun!

===== My Code =====
"""Checks community branch dir structure to see who submitted most
and what challenge is more popular by number of PRs"""
from collections import Counter, namedtuple
import csv
import os
import urllib.request
.
.
(and so on)
.
.
What I Learned
'''
* Each line is split on the comma: line.split(','). Does not need the csv module.
* endswith() method

'''''''

Now, which one would YOU rather read and edit in your text editor?

Of course, it looks best when viewed in ZimWiki. Here’s a screenshot..also note Zim’s similarity to Evernote in its layout…

The Plan

Here’s where I hope to go, moving forward:

  • Write and share my code for helping Evernote users move to ZimWiki.
  • Study Zim Wiki’s source code in depth to understand how it works, taking careful notes (in ZimWiki!) on what I learn.
  • Write some plugins for ZimWiki, perhaps a built-in “Import from Evernote” feature.
  • Contribute to ZimWiki….which would be my first participation in an open source project.