07.02.06

The GTDAlt bundle workflow

Posted in TextMate, GTD at 12:14 am by Haris

This entry has a dual purpose. First, it demonstrates the typical workflow of the GTDAlt bundle through a (hopefully) easy to follow example. Second, it gives an example, with explanations, of how to create new languages/snippets/commands etc.

The sections that address these “developer” issues will be in blockquotes

like this

to distinguish them from the workflow explanations. Think of them as “how did they do that?” pieces. You can safely skip them if you just want to learn about the workflow, but you can take a look at them when you get curious. I have further provided a couple of exercises for those of you who want to learn some more about hacking TextMate to do their bidding. These are marked at a second level of quoting like so:

EXERCISE: This is an exercise. It is not mandatory, but if you complete it you get the bonus of your name being added to the contributors to the bundle.

Okay, ready?

Go!

First of all, you will need to install the GTDAlt bundle. Since this bundle is in the repository, you would want to follow the instructions here to get it. Also, you would probably need to also check out in a similar manner the global support directory whose repository address is . A few words about where these things are situated in your computer. The instructions tell you to put them under /Library/Application Support/TextMate/, and that is indeed where you should put them. Once you start working with a bundle and make some modifications, those modifications are saved under /Users/xxx/Library/Application Support/TextMate/, where xxx is your account’s nickname. This is also more commonly known as ~/Library/Application Support/TextMate/. Those modifications are, if you are running one of the cutting edge releases, saves as “delta files”. What this means is that if you change the shortcut for a command, only this information is saved in ~/Library/Application Support/TextMate/. So if you update the command from the svn repository, then you will get the new command, but still with the shortcut you chose.

If you are developing a language, you will want to keep the entire bundle under ~/Library/Application Support/TextMate/, otherwise any edits you make to it will be in a different place, and you will then need to merge them somehow.

Ok, enough of this general non-sense, let’s get down to business. In order to use the GTDAlt bundle you need to do some groundwork. First, you need to create a nice directory somewhere, that will keep all your GTD files. You can place it wherever you want and name it whatever you want, but I will assume that it is named /Users/haris/Documents/GTD. The next thing is to define an environment variable with that value. Open the preferences and go to the “advanced” tab. Then choose the “shell variables” tab, and in there create a new shell variable with name TM_GTD_DIRECTORY and value the complete path to your directory. One way to do that is to select the value field, clear its contexts, and then drag the folder from the Finder (or the top of the textmate scratch project window probably if you have already opened this folder in textmate) into that field.

Don’t leave the preferences window yet, we need two more variables. We need a variable named TM_GTD_CONTEXT, which contains a space-separated list of the words you want for contexts. For instance it could be equal to email office home errands online programming-ruby programming-python writing and so on. I’ll pretend for the rest of this tutorial that you have set it to the above, but of course feel free to modify it.

We also need a variable TM_GTD_INBOX. This should point to a file called anything you like really, though I would suggest something like inbox.txt. You can put it in the same GTD directory defined above, or somewhere else. The path should be absolute. We won’t use this one for a while though.

You can refer to these variables both in snippets and in commands.

Alright, let’s start by creating a new gtd file. Open the GTD folder you created in textmate by dragging and dropping it onto the TextMate icon in the doc (or if you are master of the command line, open the folder using mate). Now you have a nice empty folder, let’s get some files into it. Open a new file in the project by pressing shift-command-N, and create it with extension gtd. You might now ask yourselves: “Why have I not seen this option in the menus?” But fear not, kind reader. It’s because it is hidden. Open the File menu, and press and hold the shift button. Lo and behold, a whole new set of commands!

Ok, now that we got a new gtd file, time to fill it with projects. But first, make sure the language is the correct one! The bottom of the window should read GTDAlt. Otherwise you need to select it by pressing shift-ctrl-opt-G. Do not proceed until you’ve successfully managed to set the language properly!

We start a new project by pressing the exclamation point ! (that’s shift-1 in most keyboards). This should generate three lines, one saying project project_name, with the project_name part selected, one empty with just a bit of indentation, and the third one saying end. Type in whatever title you want for the project, and then press tab to move to the next line to add methods.

What we just executed was a snippet. The interesting thing about it is that we set its scope in such a way so as to make it trigger only when it makes sense. In particular, if we are at an existing project line or an action line. In order to achieve that, we had to give names to the whole begin and end captures of the meta.project scope, because we do want the snippet to work within a project, to enable creation of subprojects.

As a primer to regular expressions, let’s see precisely how to create a language grammar that matches a project properly. First of all, we want to catch the entire project as a group. This means we have to use a “begin” and “end” rule keys. The end rule key is simple, so let’s talk about the begin rule key. (Btw, you are supposed to be looking at the language grammar in the Bundle Editor right now, or even better have opened the same data in a textmate window with “Language Grammar” as the selected language for the document). The begin rule key has the following regular expression: ^\s*(project)\s+(.*)(\n).

1. The ^ means that whatever this thing matches should be at the beginning of a line.
2. \s* means that the beginning of line could be followed by any number of whitespace characters. This of course is to allow our projects to be matched when they are indented, like when they are sub-projects etc. The \s here means: Match any whitespace character, and the * means match any number of occurrences of whatever preceded it.
3. Then we match the word project. Notice that we put it in parenthesis. That is because we want to give it a scope name in the captures section. The parentheses mean: Match what’s inside, and store it to be used later.
4. The \s+ that follows matches any non-zero number of whitespaces. We basically want to separate the word project from the rest, i.e. we don’t want to match projecthere this way.
5. the (.*) matches any sequence of characters whatsoever. The dot stands for any character. Again we group this with the parentheses for further naming use. This is the name of the project. Finally we match the newline character. The purpose of this is that if the caret is at the end of a project line, then the scope there is special, so we could use this to make the program do things exactly at this location. (We might come back to this a lot later.)

At this point you might wonder, if you are new to regular expressions, why the .* part doesn’t just match the entire document. The reason is simply that the regular expression engine tries to match the regular expression as early as possible, unless instructed otherwise. But let’s not go there yet.

EXERCISE: Learn about character classes here, and change this regular expression so that part 5 instead says: match any number of characters that are not newline characters. Then match a newline character. Also, think about why this would be more efficient. It is worth here to note the part in the definition of this scope that says patterns = ( { include = '$self'; } );. This guarantees that inside any project we match exactly the same kind of stuff as we do outside of the project. The entire grammar is allowed to show up in the part inside the project. This is the typical way of creating “containers” like that.

Alright, now the cool stuff begins. Let’s get some actions in! The simplest way to do that is, while on this new line, pressing the at-symbol, @ (that’s shift-2 in most keyboards). What this does is print an at-sign, followed by the word email selected. This was a rather arbitrary choice, and perhaps would become more customizable in the future, for instance using the first word that shows up in TM_GTD_CONTEXT.

EXERCISE: change the snippet generating this line to make it use this first word. You will need the information provided here and here.

At this point, the next step is to write the first couple of characters from the desired context and press space. This then automatically completes the context name with the first context that matches it.

How did we make this work? This is where scope selectors come in. We targeted the command by assigning it an extremely special scope, so that in every other place the spacebar just inserts a regular space. We’ll talk more about it when we talk about the regular expression that matches an action line.

At this point you will perhaps be wondering how we’ll be able to create our programming-python context this way. The answer is we won’t unfortunately, but there is another command, “Context->Next” in the bundle menu, that changes the context to the one following it, so after pressing space and seeing programming-python, you would press ctrl-C to completed it. Don’t get too attached to this shortcut though, it is bound to change, because it conflicts with the shortcut for stopping the execution of a command, so pressing ctrl-C in rapid succession has undesired results.

Ok, how did this magic happen? We created a command that takes as input the selected text, or the current line in the absence of selected text. We didn’t want to use current word for two reasons: The first one was that in the presence of characters like -, what we perceive as current word is not actually the current word. Instead, we do our own processing of the line in the command. The second reason is more subtle. We want the output of the command to place the caret at the location after one space after the context, so that the user is ready to start typing their action without having to type an extra space, after all, they did just press the spacebar once already. In order to do that, we need to capture this extra space somehow, to process it. Note that we have set the output of the command to be Insert as Snippet. This is exactly in order to be able to place the caret at the desired location in the line, by using a tab stop of $0 where we want it to appear. Note that this might require us to escape some characters in the line, because they have special meaning. This is what the e_sn function in the code does.

EXERCISE: Understand in broad terms what the actual code in the command does.

Alright, now you should be in a position to type in your actual action. Go ahead and do that now. Then you might feel the need to create a new action. You do that in two ways: You can either press enter or shift-enter. The difference is that shift-enter uses the context from the current line automatically, while plain ol’ enter behaves exactly like the “new action” command we used earlier. Actually, enter does one thing slightly better: It works no matter where you are in the current line. It will just create a new line and start afresh there. (The technical term for that is “EOL + newline”.)

Before we move a way from the topic of the context, let’s examine how you can change the context of a given action. So let’s assume that the caret is somewhere on the line with the action whose context you want to change. We’ve already seen what the “Context->Next” commands does. Another one you can use is the “Context->Choose” command, triggered right now by ctrl-opt-C. This offers a drop-down of all the contexts for you to choose from. Yet another is “Context->Type”, which selects the context, and then you can do the same “abbreviation+space” trick we talked about before.

Alright, enough about contexts. They might be extremely important in the GTD system, but they still are just just a single word each. What we need is more words. These go under the name of “notes”, and they follow a Markdown-like convention. While on an action line, run the command “Note->For action”, bound to ctrl-{ (ctrl-shift-[ in most keyboards). This should make the number [1] appear next to the action, and a similar number [1] appear at the end of the document, with the caret next to that [1]. This is where you type a note. Pressing the same keycombo (but actually executing a different command due to the miracle of scope selectors) takes you back to the action. Pressing it again moves you to the note again. In other words, the command either creates a note or takes you to it.

Ok, now how did we do that? The idea is to read the entire document into the command, and have the output set to insert as snippet. Then we place the $0 tab stop at the location where we want the caret to appear. This is not extremely efficient for large documents, but it should work like a charm in reasonably sized ones. And of course, you must never forget to escape the rest of the document, to avoid having parts of it be interpreted by the snippet processing routine.

Stay at the note for a second. The one key thing with notes is that each note for an action needs to be in a line of its own. It just makes life a lot easier in processing the documents, and this restriction might be there for some time. So what if you want to keep a lot of information somewhere? You can create another text file, something like notes on this or that.txt, and then have a link to it stored in the note. To accomplish that, just drag and drop the text file in the exact location in the bundle where you want the link to appear, and you should see something like: /Users/xxx/...../notes%20on%20this%20or%20that.txt show up. This is a url link to your file. Pressing ctrl-L while the caret is in the note line will open all such links with the default application for handling them.

This of course was done via a drag command. Part of the data that a drag command is given is the relative path to the file, given the location of the current file. Then a little work with paths and escaping things creates the right link.

Alright, let’s go back to one of our actions. What we want to do now is add a due date for the action. We do that via the “Due Date->Add New” command. This command asks you to specify the date you want, and it is smart enough to understand a lot of things like “today”, “tomorrow”, “2 months”, “17 days”, “next Sunday” and so on, as well as direct dates, like “7/23”, “2007-04-06” etc.

The only thing worth mentioning for developers here, is that there are some nice libraries if you use Ruby as your language. They live in the global support directory, under the lib subdirectory, and you can “require” them in your code and use them. Of particular interest are the exit_codes, escape and dialog libraries. The first allows you to change on the fly the output of the command, by using a call like exit_discard. The second has a couple of useful commands for escaping string, like for instance the one for escaping a string to be used in a snippet and so on. The third allows you to accept user input via either a Dialog.request_string call (as this date command does) or a Dialog.request_item call for the pull-down menu (as one of the context commands did.) See the code for details.

You will notice that the date shows up in a particular format, with the year first, followed by the month and then the date. One of the reasons is that it archives a bit better this way, with alphabetical ordering doing the right thing.

Now, suppose you realized that the date you entered was wrong or something, and you want to change it. There are two ways to do that. The one is via the same command that you used for creating it in the first place. Just run it again and it will overwrite the existing date. The other method is via set of six commands that automatically add or subtract a day/week/month to the current date on the line. They are triggered appropriately via the “less than” and “greater than” signs, combined with modifier keys.

EXERCISE: Write a command that removes the due date from the current line. Similarly, write a command that removes the note of the current line, both the number [1] or whatever appearing in the action line as well as the actual note. The caret should then be placed at to what you would consider the appropriate location.

Ok, you have just seen the basics of creating new projects and actions. Let me further point out that you could have “single actions”, that are just lying around and are not part of any project. Now go ahead and create a couple of projects and actions, and a couple more .gtd files, so that we’ve got something to play with for the rest. Don’t forget to save.

Alright, the next thing to talk about is how to quickly navigate among the various projects. Of course you should already be familiar with the general methods that TextMate provides for this. One is the “Go to Symbol” command, triggered by shift-cmd-T. Similarly, the “Go to File” command, triggered by cmd-T, which allows you to quickly move to a different file in the project. If you’ve never played with these things, now is the time to do so.

In order to add symbols for use both in the pop-up in the bottom right of the document window and in the “Go to Symbol” command, you have to create a preference item that has the following three properties: First, you determine what will be shown by the scope selector, which in this case is set to line.project.begin.gtdalt. This will make sure to match the line that says project…. Then, it needs to contain the key showInSymbolList = 1;. This makes sure that the line will show up in the project list. Third, we need a line like: symbolTransformation = 's/project//;';. This determines what changes should be made to the line before it shows up, using a sed style. In particular, we are telling it to search for the word “project” and replace it with nothing. Notice that we don’t remove any of the spaces. They are used for indenting the items in the pop-up.

The GTDAlt bundle adds two more commands to help you navigate around. First of all, there is a “Go to Project” command, which offers a pop-up of all projects from all files. It is bound to ctrl-shift-G. The other is the “Find Project” command, bound to ctrl-shift-F. It asks you to type a series of consecutive letters from various words in the project name, and then searches among all projects in all files to find matching projects. For instance, if you want to find the project called “World domination”, then you could type in the search string “wor dom”. You could also try just “wor”. If there are more matchings, then the command will offer you a pull-down choice among all the matches.

Just to keep you on your tows, let’s talk now about the regular expression catching the action lines. It looks like that: ^\s*(@\S+\s++)(?:([^\[]+)(?:\s+)(?:(\[\d+\]))?(?:\s*)(?:((?:due|at|from):)(\[\d{4}-\d{2}-\d{2}\]))?\s*+)?. Wow, that’s monstrous, right? Well, let’s take it one step at a time. First we match the beginning of a line followed by any number of whitespaces. Then we match a group containing the at-sign, followed by at least one non-whitespace character (the \S+ part) along with any whitespaces following it (the \s++ part). Two things are of note here. First, we are matching the whitespaces. Why is that? because we want to have the spacebar activate our command that tried to complete the word behind it to a context. We want this command to only trigger in this setting. However, the scope at the caret’s location is determined by the character to its right, so it is necessary to match at least one of these spaces. Instead, we match as many as we can. The other point is the two pluses following the \s. This makes the match non-backtracking. Normally the way regexps work is that the regexp would first try to match as many of the \s as it can, and then match the rest of the expression. If it can’t, it will take a step back and try again to match the rest of the expression, and will keep on doing that. But we know we don’t want this. The \s++ tells the engine that it should match at this point as many spaces as it can, and not try to backtrack if it can’t match the rest of the regexp. It will just fail in this case. This makes the regexp a tad faster.

ok, let’s proceed with the rest of the regexp. Then we want to match the name of the action. This will match as much as it can until it finds a bracket, which indicates either a note or a date. Here we can’t use backtracking, because we should not match the due: of the date as part of the name, but instead as part of the date. Note the (?: part. This means that this group (ending with the corresponding parenthesis at the other end), should not be stored for later use. It is used just to separate what happens in there from the rest. The ([^\[]+) part matches a sequence of characters that are not a left bracket and stores this. Then, we match a number of spaces. The next three parts are (?:(\[\d+\]))?, (?:\s*) and (?:((?:due|at|from):)(\[\d{4}-\d{2}-\d{2}\]))?\s*+)?. The first matches a bracket (\[) followed by any nonzero number of digits (\d+) and then a close bracket. The question mark at the end says that it is fine if the whole thing does not show up at all. i.e. it makes the whole thing optional. This corresponds to a note, which may or may not be there at all. The then take away any number of whitespaces, and finally match the date format.

EXERCISE: Make sure you understand that last expression.

The next thing one wants to do is mark actions as completed. This happens by pressing the standard comment key-combo, cmd-/. The line is then marked as complete and timestamped. You can also comment a number of lines all at once by selecting them before running the command. This is a “toggle” command, pressing it again “de-completes” the line.

As usual, to be continued soon right here…

Permalink