Gthnk

Gthnk is a personal knowledge management system. Capture your ideas using plain old text files. Make a journal that lasts 100 years.

Python Package Documentation Status Github Project

Overview

  • Gthnk presents a journal consisting of many entries.
  • Entries are created using plain old text files, which Gthnk imports once per day.
  • Any text editor can be used to add information to Gthnk.
  • Entries are searchable using the embedded Gthnk server, which can be accessed with a browser.
  • Plain-text enables backup/restore via hardcopy (e.g. paper) for long-term archival.

The easiest way to run Gthnk is with Docker. Gthnk also installs on Windows, Linux, and MacOS systems with Python 3.5+. See the Installation document for more details.

Quick Start

Use Docker to run Gthnk with all files stored locally in ~/.gthnk.

docker run -d --rm \
    --name gthnk \
    -p 1620:1620 \
    -e TZ=America/Toronto \
    -v ~/.gthnk:/opt/gthnk/var \
    iandennismiller/gthnk:0.8

The default text file where you will record journal entries is ~/.gthnk/journal.txt.

Open journal.txt with a text editor to add new journal entries.

Open http://localhost:1620 to access the user interface.

Journal Entries

Use the journal by editing journal.txt with a text editor. First, insert a date marker YYYY-MM-DD and a blank line to start a new journal day. Then, insert a time marker HHMM and a blank line to start a journal entry.

2020-07-04

0804

This is a simple journal entry.

0805

And this is a separate entry, a minute later.

Those two delimiters - date and time followed by a blank line - are all there is to the Gthnk journal file format. The rest is Markdown.

You can add multiple entries per day - and multiple days in a single journal - by inserting date and time markers as you work.

Command Line Interface

While the Docker container is running, the Gthnk command line interface is available using docker exec. Create a shell alias to simplify access.

alias gthnk="docker exec -it gthnk /opt/gthnk/.venv/bin/gthnk"
gthnk --help

To view the current journal buffer, use gthnk buffer.

To search for a keyword, use gthnk search.

Web Interface

To interact with the Gthnk journal, connect to http://localhost:1620.

Click the fast-forward icon to view the live journal buffer. As you edit journal.txt, this live buffer will be updated.

Once the journals have been rotated, the history of previous days becomes available within the Gthnk Interface.

Journal Rotation

When the journal rotates, all the entries are imported from journal.txt into the database. After import, the journal.txt file is wiped.

The preferred rotation method method is to use an automatic process like cron, systemd, or launchd. The journal can be manually rotated with the command line interface: gthnk rotate.

Before journal.txt is wiped, its contents are backed up. Information is never lost even if there is a problem with rotation.

Integration with Text Editors

Text editor integrations make it easier to insert journal entries.

After installing the plugin for your editor, the following key combinations are available:

  • Ctrl-Alt-Cmd-N: Insert date marker YYYYMMDD
  • Ctrl-Alt-Cmd-M: Insert time marker HHMM

Cloud Sync

You can sync Gthnk to multiple devices using a cloud file system like Dropbox or Syncthing. Use the docker run -v flag to point to your cloud storage: -v ${PATH_TO_CLOUD}/gthnk:/opt/gthnk/var

A complete example using Dropbox could look like:

docker run -d --rm \
    --name gthnk-server \
    -p 1620:1620 \
    -e TZ=America/Toronto \
    -v ~/Dropbox/gthnk:/opt/gthnk/var \
    iandennismiller/gthnk:0.8

This configuration supports running Gthnk on a dedicated server, like a local Linux machine, while editing the journal files on devices that are synced via the cloud.

To support a laptop and phone, use a custom configuration file.

docker run -d --rm \
    --name gthnk-server \
    -p 1620:1620 \
    -e TZ=America/Toronto \
    -v ~/.gthnk/gthnk.conf:/opt/gthnk/.config/gthnk/gthnk.conf \
    -v ~/Dropbox/gthnk:/opt/gthnk/var
    iandennismiller/gthnk:0.8

Then edit ~/.gthnk/gthnk.conf to specify multiple INPUT_FILES.

INPUT_FILES = "/opt/gthnk/var/journal-laptop.txt,/opt/gthnk/var/journal-phone.txt"

Documentation

http://docs.gthnk.com

Introduction

Quick Start

The simplest installation method is to use Docker.

docker run -d --rm \
    --name gthnk \
    -p 1620:1620 \
    -e TZ=America/Toronto \
    -v ~/.gthnk:/opt/gthnk/var \
    iandennismiller/gthnk:0.8

However, advanced methods are also available to install Gthnk as a Python package. Read Installation for more installation options.

Use the Journal

Gthnk uses a basic Text File Format because anything more complicated isn’t reliable enough for something as important as your thoughts.

Create a file called ~/.gthnk/journal.txt and use a text editor to paste the following into it:

2016-01-08

0840

Hello world!  I am making a note in my work journal.

1322

The Gthnk website is http://gthnk.com

This is a basic journal file for January 8, 2016. It has two entries, one at 8:40am and another at 1:22pm. Based in this example, you can probably imagine how to make journal entries for any other day and time.

Accessing the Journal

Access Gthnk in your browser with this URL: http://localhost:1620/

You will see that the journal entries have been added to the database and the text file has been reset to a blank file.

Next Steps

Read Using the Journal to understand how to get the most out of Gthnk.

System Requirements

If you plan to install Gthnk as a python package, you will need to install Python 3.5+ first.

If you want to run the Docker image, then you must install Docker that first.

Next Steps

Now that you have the system requirements installed, read Installation to put Gthnk on your computer.

Installation

There are currently two ways to install Gthnk:

  1. Run it inside a Docker image
  2. Install as a python package

Docker Image

Use Docker to run Gthnk with all files stored locally in ~/.gthnk.

docker run -d --rm \
    --name gthnk \
    -p 1620:1620 \
    -e TZ=America/Toronto \
    -v ~/.gthnk:/opt/gthnk/var \
    iandennismiller/gthnk:0.8

The default text file where you will record journal entries is ~/.gthnk/journal.txt.

If you want to store your files somewhere other than ~/.gthnk then update the -v argument:

-v /opt/somewhere-else:/opt/gthnk/var \

If you want to use a port other than 1620, update the -p argument:

-p 5000:1620 \

Python Installation

Gthnk also installs as a python package - but the Docker method is strongly recommended.

Every system is different but python installation looks like this:

git clone https://github.com/iandennismiller/gthnk.git
cd gthnk
pip3 install --user ./src
gthnk config

By default, gthnk uses `/tmp/gthnk for data storage. Configure gthnk with ~/.config/gthnk/gthnk.conf.

Next Steps

Now check out Using the Journal to learn techniques for actually getting stuff done with Gthnk.

User Guide

Using the Journal

An entry is denoted by a timestamp, which is written using hours and minutes, like 0815 for 8:15am and 1425 for 2:25pm. A day is denoted by a datestamp, which is written using year-month-day, like 2016-01-08 for January 8, 2016. Usually, Text Editor Macros will do this for you, so you really don’t need to worry about it. Read more about the Text File Format to learn other tricks.

The default Gthnk installation will create a file on your desktop called journal.txt. This file is called a “journal buffer” and anything you write in it will be added to the journal once per day. You can change the location of this file by editing ~/.gthnk/gthnk.conf and changing the INPUT_FILES entry.

Example

A simple journal consisting of 1 day and 3 entries looks like this:

2016-01-08

0840

Hello world!  I am making a note in my work journal.

1825

The gthnk website is http://gthnk.com

2210

I had an interesting thought.  What if...

There is no limit to the number of days you can have in a journal buffer. It is also possible to use multiple journal buffers in order to gather entries from Mobile Devices.

Accessing the Journal

Access Gthnk in your browser with this URL: http://localhost:1620/day/live

Browser Integration makes it easy to search Gthnk by keyword, so you can easily find old entries. A variety of GUI methods enable navigation between days.

Daily Journal Maintenance

Every day, Gthnk automatically does two maintenance tasks:

  • Just after midnight, Gthnk will collect any new journal entries for the day and store them in the database.
  • At 9:00am, Gthnk will open a browser window with the previous day’s entries.
Next Steps

Read the following to learn more about Gthnk:

Text Editor Macros

It would be pretty annoying to manually type out a timestamp every time you wanted to make a journal entry. Luckily, many text editors can be extended with macros to make this automatic.

Sublime Text

I personally use Sublime Text 3 as my primary method for editing my work journal. The following steps will help you extend Sublime Text 3 to add datestamps and timestamps using a key combination.

  1. Download journal_date.py
  2. Copy journal_date.py to the Sublime Text 3 Packages directory
cp journal_date.py "$HOME/Library/Application Support/Sublime Text 3/Packages/User"
  1. Add the following to Default (OSX).sublime-keymap
[
    { "keys": ["ctrl+super+alt+n"], "command": "insert_date" },
    { "keys": ["ctrl+super+alt+m"], "command": "insert_time" }
]

This creates two hotkeys that will automatically insert datestamps and timestamps. You can change the keymap however you want to make the hotkeys convenient for you.

Jota+

Jota+ for Android enables me to add notes to my journal using a phone or tablet. I create a file in dropbox called journal-phone.txt and add regular entries to it, just like always.

Jota+ provides macros, but it calls them fixed phrases. You can create two fixed phrases for adding datestamps and timestamps:

  • datestamp: %yyyy%-%MM%-%dd%
  • timestamp: %HH%%mm%

Be sure that the fixed phrases include blank lines around the timestamp.

Other text editors

Any text editor that provides macros for the date and time is likely to support Gthnk. If you successfully set up another text editor, please create an issue that describes your process and we can add it to this document.

Browser Integration

Gthnk runs a private web server that only you can connect to. There are a few browser tricks that make it easier to get extra use out of Gthnk.

Google Chrome

App Launcher
  1. Enable the necessary Chrome flags
    1. chrome://flags
    2. chrome://flags/#bookmark-apps - Enable
    3. chrome://flags/#disable-hosted-apps-in-windows - Enable
  2. Add Gthnk as an App
    1. Navigate to http://localhost:1620/journal/live
    2. Click the 3-dot “more” tool button
    3. Click “More Tools”
    4. Click “Add to Applications”
    5. Call it “Gthnk”
  3. Done
Chrome Extension

This method has been replaced by the App Launcher, which is better…

The Gthnk Google Chrome extension enables Gthnk to be launched from the menu bar using the Chrome Launcher.

  1. Download the extension code from github
git clone https://github.com/iandennismiller/chrome-app-gthnk.git
  1. Open chrome://extensions/ in your browser
  2. Check Developer mode
  3. Click Load unpacked extension…
  4. Navigate to gthnk/integrations/chrome-app
  5. Click OK

Now, you have an icon that will launch Gthnk.

Hotkey

I use Spark.app to create a hotkey pattern for launching the Gthnk app.

Many launchers will work, but the important part is that you must create an “App Launcher” for Gthnk before the hotkey can be created.

Steps for adding a Gthnk hotkey:

  1. Click the gear icon to drop down the list of things to create
  2. Choose Application
  3. Set a unique shortcut key combination
  4. name it Gthnk
  5. Action: Launch
  6. Choose: locate the gthnk chrome app
    1. Navigate to wherever your apps are and looks for Chrome Apps
    2. On OS X, this is probably /Applications/Chrome Apps
    3. Select Gthnk in the Chrome Apps folder
  7. Done

Other Browsers

Since Gthnk is provided as a web server, you can use any browser to connect to it. If you craft integrations for other browsers, please create an issue that describes your process and we can add it to this document.

Mobile Devices

One of the key features of Gthnk is that it runs on your computer - not in the cloud - so you can keep your thoughts private. However, sometimes you’re on the road and you really want to write something down.

This scenario is no problem for Gthnk - just use a service like Dropbox or Seafile to sync your journal buffers back to your computer. This permits you to take non-sensitive notes even when you’re not by your computer.

Journal buffers via Dropbox

Gthnk can import any number of journal buffers every day. Edit INPUT_FILES in the configuration file ~/Library/Gthnk/gthnk.conf and add any Dropbox files there.

Since Dropbox is usually ~/Dropbox, you might create a file called ~/Dropbox/journal-phone.txt for capturing notes that come from your phone. To accomplish this, you would edit the configuration like this:

INPUT_FILES = "/Users/me/Dropbox/journal-phone.txt,/Users/me/Desktop/journal.txt"

Now Gthnk will import entries from the file on your Desktop and from your phone. It doesn’t matter if these files are empty most of the time; that doesn’t bother Gthnk.

Mobile Text Editors

Have you had success with other mobile text editors? Please create an issue that describes your experience and we can add it to this document.

Text File Format

Gthnk uses a basic text file format because anything more complicated isn’t reliable enough for something as important as your thoughts. Even though journal entries are stored in a sqlite3 database for fast indexing, they are also exported as plain text files every night.

Text Formatting

There are a few simple rules that serve to partition a text file into a sequence of journal entries.

Timestamp and Datestamp

An entry is denoted by a timestamp, which is written using hours and minutes, like 0815 for 8:15am and 1425 for 2:25pm. This must have a blank line before it and a blank line after it.

A day is denoted by a datestamp, which is written using year-month-day, like 2016-01-08 for January 8, 2016. This must have a blank line before it and a blank line after it.

Markdown

Any journal entry may contain Markdown. This makes it easy to add lists and URLs. You don’t have to do anything special to enable Markdown.

Custom Markdown Extension

To support rendering Gthnk journals in a more aesthetically pleasing way, mdx_journal was written to provide a minor extension to basic markdown. This extension will cause datestamps to become large headings and timestamps to become slightly smaller headings. It makes a day’s entries much more readable.

Filesystem Exports

One of the goals for using text files is to ensure the journal has a higher likelihood of being readable to future generations. One thing we’ve learned is that file formats can become obsolete in a matter of years. However, ASCII text has been with us since 1960, so let’s stick with what works.

Uninstallation

If installed with pip, do the following

pip uninstall gthnk gthnk_web

Community

Contributing

Gthnk welcomes contributions from the community. The most important theme for Gthnk contributions is communication. An open source project thrives when communication is healthy.

Here are examples of some ways you can help:

  • Extend and clarify the documentation
  • Internationalization; help with translations
  • Software Development

Help with the documentation

Project documentation is one of the most important aspects of an open source project. You can help immensely by editing the current documentation for clarity. You can find the documentation in the Gthnk repository, which you can modify by submitting a pull request.

Internationalization

Can you help localize Gthnk for a new language? We can use help translating the software, websites, and documentation.

Software Development

Gthnk is an open source project. If you know Python, please see the Developer document for more information.

Developer

The first step is to clone the project repository using git. Then, give the rest of this document a read for some specific ideas about contributing.

Submitting a pull request

Gthnk development occurs in the develop branch of the git project. We welcome thoughtful pull requests against the develop branch with code that includes comments and tests. When you create a pull request, be sure you have the latest version of the develop branch.

Implement your favorite feature

If you know a feature you’d like to code in order to help with Gthnk, then make your own local branch and write it until you are happy. If you end up writing a feature that is really useful to you, then maybe you will consider contributing it to the community. In that case, be sure to write tests for your feature and add code comments where appropriate. The code should pass a Python linter that checks for PEP8 compliance.

Help with an existing issue

You can contribute by working on issues that have been submitted by the community using the Issue Tracker. This is a little more advanced than simply implementing a new feature since the issue may imply acceptance criteria. It is recommended that you coordinate with project members before working on an issue. The easiest way to contact the team is through the Issue Tracker itself; each issue has a comment thread associated with it.

Development

Release Procedure

The project advances as a progression of project releases in which each release is a new iteration of the project. Project planning consists of managing which items in docs/Todo.md will be included in each Milestone. In general, each Milestone corresponds to a 0.0.1 increment in the release number.

The project source code repository contains two main branches: master and develop. All of the active work happens in the develop branch. When a milestone is reached, all of the work from the develop branch is merged into the master branch in order to prepare for a release.

In order to actually perform the release, there are several steps that must be performed each time. This document discusses the “Release Procedure” that causes a new iteration of the project to be publicly released.

Overview

  • start in the develop branch.
  • update version in __meta__.py.
  • update docs/changelog.rst with the todo items that were completed for this release.
  • git commit these final changes to the develop branch and git push them to the remote repo.
  • git checkout master and git merge develop to bring in all changes from the develop branch.
  • git push the master branch to kick off Travis CI and Coveralls.
  • make release.
  • verify the release on pypi at https://pypi.python.org/pypi/gthnk

Finalize develop branch

Start in the develop branch.

git checkout develop

Update version in __meta__.py.

Update docs/changelog.rst with the todo items that were completed for this release.

Commit final changes to the develop branch and push them to the remote repo.

git commit -am "prepare develop for release"
git push

Release Master

Checkout the master branch and merge from develop to bring in all changes from the develop branch.

git checkout master
git merge develop

Push to the master branch to kick off Travis CI and Coveralls.

git push

Use the Makefile release target, which delegates to the setuptools upload support for pypi.

make release

Verify the release on pypi at https://pypi.python.org/pypi/gthnk

Release Candidates

In anticipation of a release, there are many reasons why it might be useful to provide a preview release called a “release candidate”. The procedure for a release candidate is the same as for a release. The only notable change for a release candidate is the version scheme. The version uses the number of the upcoming release with rcX appended, where rc1 is the first release candidate, rc2 is the second, and so on. So in anticipation of 0.9.0, the first release candidate might be called 0.9.0rc1.

About

Vision

The vision for Gthnk has been stable for many years.

Gthnk is a system that will enable users to:

  • keep a chronological journal of writings
  • support archival for 100 years
    • import/export via paper
  • review recent entries effortlessly
  • find specific ideas or entries
    • precision search through entire history
  • construct an inter-connected network of knowledge
  • create an entry using any technology
    • computer/phone, ink+paper, other?
  • run privately or publicly
    • (i.e. server or desktop)

This vision describes whta Gthnk is going to become. However, this vision does not specify how that will happen.

Gthnk seeks to enable people to do specific things they could not before. These specific new capabilities are called features. A feature can be spearated from the technology that implements the feature. Therefore, this vision is constructed without reference to the technical details pertaining to the implementation.

This vision represents the end-state; it is not the roadmap.

History of gthnk

I began documenting my work-related thoughts in a work journal circa 2006. Initially, I used Open Office templates. I chose to separate journal entries with a timestamp because I could type it pretty quickly on a numeric keypad. I iterated through quite a few methods for keeping a chronological sequence of entries until I ended up with Gthnk.

This work was inspired by DevonThink, which I really tried to use. For me, I needed a less opaque container for my information. I wanted my text files to stay text files, because my concern was that years or decades down the road, I would not be able to access text files stored in a DevonThink filesystem container.

This work was also inspired by Vernor Vinge’s Marooned in Realtime, which featured a companion-like computer called “GreenInc”. This computer presented an interface that was highly customized to the person who trained and used their GreenInc system. Ironically enough, I misremembered the name of Vinge’s system as “GreenThink” to the extent that I had used it as a name for an earlier prototype of the Gthnk system.

Changelog

0.8.0

2023-06-27

  • complete rewrite of gthnk backend
  • deprecating sqlite support; only text files now
  • focused on essential features, trimmed low-use/high-maintenance features
  • removed: tags, todo, attachments/pages, web UI notes, user accounts, sqlite
  • new command line interface
  • refreshed web interface
  • decent test coverage
  • good code linting
  • tons of bugfixes
  • web search results are paginated

0.7.0

2020-08-09

  • dark mode UI
  • Docker workflow
  • live view of current buffer
  • improvements to mobile UI: swipe left/right, small screen
  • add notes via web UI
  • simple support for tags
  • simple agenda renderer
  • NB: removed attachments features (temporary)
  • major refactoring to support future development
  • fix tons of bugs

0.6.2

2020-06-28

  • Fix bug with journal export of attachments

0.6.0

2020-01-02

  • Update for Python 3

0.5.0

2017-11-29

  • integration subsystem for Windows and OS X
  • python 3 compatibility
  • refresh documentation with new installation procedures
  • unit testing
  • code coverage
  • bug: fix date picker
  • bug: image attachments not centering
  • updated to Flask-Diamond 0.5.0

0.4.0

2016-01-05

  • revise documentation
  • clean unnecessary dependencies
  • automated install process on OS X
  • major refactor of code
  • updated to Flask-Diamond 0.2.14

0.3.0

2015-01-31

  • revised user interface
  • image and PDF attachments
  • PDF export
  • calendar date navigation
  • robust filesystem export

0.2.0

2014-11-15

  • new architecture with app server and database
  • revised user interface
  • search results link to day
  • migrating last parts of old “greenthink” project
  • better project distribution with Makefiles

0.1.0

2013-12-09

  • renaming “greenthink” as Gthnk
  • feature: review yesterday’s notes
  • markdown in entries
  • project review subsystem
  • OS X service launches review every day
  • support mobile; import multiple text sources

License

The MIT License (MIT)

gthnk

Copyright (c) 2013-2020 Ian Dennis Miller

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.