monotone documentation

Top

Monotone Documentation

Monotone is a distributed version control tool. It can help automate many tedious and error-prone tasks in group software development.

• Store multiple versions of files you are working on efficiently.
• Transmit changes to files between you and your colleagues.
• Merge changes you make with those your colleagues make.
• Make notes about your opinion of the quality of versions of files.
• Make decisions about using or ignoring versions, depending on the notes you receive from others.

Please be aware that monotone is a slightly unorthodox version control tool, and many of its concepts are slightly similar -- but significantly different -- from concepts with similar names in other version control tools.

Complete table of contents

Concepts

This chapter should familiarize you with the concepts, terminology, and behavior described in the remainder of the user manual. Please take a moment to read it, as later sections will assume familiarity with these terms.

Versions of files

Suppose you wish to modify a file file.txt on your computer. You begin with one version of the file, load it into an editor, make some changes, and save the file again. Doing so produces a new version of the file. We will say that the older version of the file was a parent, and the new version is a child, and that you have performed an edit between the parent and the child. We may draw the relationship between parent and child using a graph, where the arrow in the graph indicates the direction of the edit, from parent to child.

We may want to identify the parent and the child precisely, for sake of reference. To do so, we will compute a cryptographic hash function, called SHA1, of each version. The details of this function are beyond the scope of this document; in summary, the SHA1 function takes a version of a file and produces a short string of 20 bytes, which we will use to uniquely identify the version¹. Now our graph does not refer to some "abstract" parent and child, but rather to the exact edit we performed between a specific parent and a specific child.

When dealing with versions of files, we will dispense with writing out "file names", and identify versions purely by their SHA1 value, which we will also refer to as their file ID. Using IDs alone will often help us accommodate the fact that people often wish to call files by different names. So now our graph of parent and child is just a relationship between two versions, only identified by ID.

Version control systems, such as monotone, are principally concerned with the storage and management of multiple versions of some files. One way to store multiple versions of a file is, literally, to save a separate complete copy of the file, every time you make a change. When necessary, monotone will save complete copies of your files in their, compressed with the zlib compression format.

Often we find that successive versions of a file are very similar to one another, so storing multiple complete copies is a waste of space. In these cases, rather than store complete copies of each version of a file, we store a compact description of only the changes which are made between versions. Such a description of changes is called a delta.

Storing deltas between files is, practically speaking, as good as storing complete versions of files. It lets you undo changes from a new version, by applying the delta backwards, and lets your friends change their old version of the file into the new version, by applying the delta forwards. Deltas are usually smaller than full files, so when possible monotone stores deltas, using a modified xdelta format. The details of this format are beyond the scope of this document.

Versions of trees

After you have made many different files, you may wish to capture a "snapshot" of the versions of all the files in a particular collection. Since files are typically collected into trees in a file system, we say that you want to capture a version of your tree. Doing so will permit you to undo changes to multiple files at once, or send your friend a set of changes to many files at once.

To make a snapshot of a tree, we begin by writing a special file called a manifest. In fact, monotone will write this file for us, but we could write it ourselves too. It is just a plain text file. Each line of a manifest file contains two columns. In the first column we write the ID of a file in your tree, and in the second column we write the path to the file, from the root of our tree to the filename.

Now we note that a manifest is itself a file. Therefore a manifest can serve as input to the SHA1 function, and thus every manifest has an ID of its own. By calculating the SHA1 value of a manifest, we capture the state of our tree in a single manifest ID. In other words, the ID of the manifest essentially captures all the IDs and file names of every file in our tree, combined. So we may treat manifests and their IDs as snapshots of a tree of files, though lacking the actual contents of the files themselves.

As with versions of files, we may decide to store manifests in their entirety, or else we may store only a compact description of changes which occur between different versions of manifests. As with files, when possible monotone stores compact descriptions of changes between manifests; when necessary it stores complete versions of manifests.

Historical records

Suppose you sit down to edit some files. Before you start working, you may record a manifest of the files, for reference sake. When you finish working, you may record another manifest. These "before and after" snapshots of the tree of files you worked on can serve as historical records of the set of changes, or changeset, that you made. In order to capture a "complete" view of history - both the changes made and the state of your file tree on either side of those changes - monotone builds a special composite file called a revision each time you make changes. Like manifests, revisions are ordinary text files which can be passed through the SHA1 function and thus assigned a revision ID.

The content of a revision makes reference to file IDs, in describing a changeset, and manifest IDs, in describing tree states "before and after" the changeset. Crucially, revisions also make reference to other revision IDs. This fact - that revisions include the IDs of other revisions - causes the set of revisions to join together into a historical chain of events, somewhat like a "linked list". Each revision in the chain has a unique ID, which includes by reference all the revisions preceeding it. Even if you undo a changeset, and return to a previously-visited manifest ID during the course of your edits, each revision will incorporate the ID of its predecessor, thus forming a new unique ID for each point in history.

Certificates

Often, you will wish to make a statement about a revision, such as stating the reason that you made some changes, or stating the fact that you approve of the changes which were made, or stating the time at which you made the changes. Statements such as these can be thought of, generally, as a bundle of information with three parts:

• an ID, indicating which revision you are making a statement about
• a name indicating the type of statement you are making, such as "changelog", "date" or "disapproval"
• a value indicating the remaining detail of the statement, such as "fixed a bug", "March 9th" or "approved for shipping"

For example, if you want to say that a particular revision was composed on April 4, 2003, you might make a statement like this:

In an ideal world, these are all the parts of a statement we would need in order to go about our work. In the real world, however, there are sometimes malicious people who would make false or misleading statements; so we need a way to verify that a particular person made a particular statement about a revision. We therefore will add two more pieces of information to our bundle:

• a key which identifies the person making a statement
• a signature -- just a large number with particular properties -- certifying the fact that the person made the statement

When these 2 items accompany a statement, we call the total bundle of 5 items a certificate, or cert. A cert makes a statement in a secure fashion. The security of the signature in a cert is derived from the RSA cryptography system, the details of which are beyond the scope of this document.

Monotone uses certs extensively. Any "extra" information which needs to be stored, transmitted or retrieved -- above and beyond files, manifests, and revisions -- is kept in the form of certs. This includes change logs, time and date records, branch membership, authorship, test results, and more. When monotone makes a decision about storing, transmitting, or extracting files, manifests, or revisions, the decision is often based on certs it has seen, and the trustworthiness you assign to those certs.

The RSA cryptography system -- and therefore monotone itself -- requires that you exchange special "public" numbers with your friends, before they will trust certificates signed by you. These numbers are called public keys. Giving someone your public key does not give them the power to impersonate you, only to verify signatures made by you. Exchanging public keys should be done over a trusted medium, in person, or via a trusted third party. Advanced secure key exchange techniques are beyond the scope of this document.

Most of monotone's certs refer to revision IDs. Some certs may refer to file IDs or manifest IDs, depending on context. This capability may also be removed in the future, at which point certs will only refer to revisions.

Storage and workflow

Monotone moves information in and out of three different types of storage:

• a working copy in the local file system
• a local database in the local file system
• a remote database elsewhere on the internet

All information passes through your local database, en route to some other destination. For example, when changes are made in a working copy, you may save those changes to your database, and later you may synchronize your database with someone else's. Monotone will not move information directly between a working copy and a remote database, or between working copies. Your local database is always the "switching point" for communication.

A working copy is a tree of files in your file system, arranged according to the list of file paths and IDs in a particular manifest. A special directory called MT exists in the root of any working copy. Monotone keeps some special files in the MT directory, in order to track changes you make to your working copy.

Aside from the special MT directory, a working copy is just a normal tree of files. You can directly edit the files in a working copy using a plain text editor or other program; monotone will automatically notice when you make any changes. If you wish to add files, remove files, or move files within your working copy, you must tell monotone explicitly what you are doing, as these actions cannot be deduced.

If you do not yet have a working copy, you can check out a working copy from a database, or construct one from scratch and add it into a database. As you work, you will occasionally commit changes you have made in a working copy to a database, and update a working copy to receive changes that have arrived in a database. Committing and updating take place purely between a database and a working copy; the network is not involved.

A database is a single, regular file. You can copy or back it up using standard methods. Typically you keep a database in your home directory. Databases are portable between different machine types. If a database grows too big, you may choose to remove information from it. You can have multiple databases and divide your work between them, or keep everything in a single database if you prefer. You can dump portions of your database out as text, and read them back into other databases, or send them to your friends.

A database contains many files, manifests, revisions, and certificates, some of which are not immediately of interest, some of which may be unwanted or even false. It is a collection of information received from network servers, working copies, and other databases. You can inspect and modify your databases without affecting your working copies, and vice-versa.

Monotone knows how to exchange information in your database with other remote databases, using an interactive protocol called netsync. It supports three modes of exchange: pushing, pulling, and synchronizing. A pull operation copies data from a remote database to your local database. A push operation copies data from your local database to a remote database. A sync operation copies data both directions. In each case, only the data missing from the destination is copied. The netsync protocol calculates the data to send "on the fly" by exchanging partial hash values of each database.

In general, work flow with monotone involves 3 distinct stages:

• When you commit changes from your working copy to your database, your database stores the changes but does not communicate with the network. Your commits happen immediately, without consulting any other party, and do not require network connectivity.
• When you are ready to exchange work with someone else, you can push, pull, or sync with other databases on the network. When you talk to other servers on the network, your database may change, but your working copy will not. In fact, you do not need a working copy at all when exchanging work.
• When you update your working copy, some (but not all) of the changes which your database received from the network are applied to your working copy. The network is not consulted during updates.

The last stage of workflow is worth clarifying: monotone does not blindly apply all changes it receives from a remote database to your working copy. Doing so would be very dangerous, because remote databases are not always trustworthy systems. Rather, monotone evaluates the certificates it has received along with the changes, and decides which particular changes are safe and desirable to apply to your working copy.

You can always adjust the criteria monotone uses to judge the trustworthiness and desirability of changes in your database. But keep in mind that it always uses some criteria; receiving changes from a remote server is a different activity than applying changes to a working copy. Sometimes you may receive changes which monotone judges to be untrusted or bad; such changes may stay in your database but will not be applied to your working copy.

Remote databases, in other words, are just untrusted "buckets" of data, which you can trade with promiscuously. There is no trust implied in communication.

Forks and merges

So far we have been talking about revisions as though each logically follows exactly one revision before it, in a simple sequence of revisions.

This is a rosy picture, but sometimes it does not work out this way. Sometimes when you make new revisions, other people are simultaneously making new revisions as well, and their revisions might be derived from the same parent as yours, or contain different changesets. Without loss of generality, we will assume simultaneous edits only happen two-at-a-time; in fact many more edits may happen at once but our reasoning will be the same.

We call this situation of simultaneous edits a fork, and will refer to the two children of a fork as the left child and right child. In a large collection of revisions with many people editing files, especially on many different computers spread all around the world, forks are a common occurrence.

If we analyze the changes in each child revision, we will often find that the changeset between the parent and the left child are unrelated to the changeset between the parent and the right child. When this happens, we can usually merge the fork, producing a common grandchild revision which contains both changesets.

Branches

Sometimes, people intentionally produce forks which are not supposed to be merged; perhaps they have agreed to work independently for a time, or wish to change their files in ways which are not logically compatible with each other. When someone produces a fork which is supposed to last for a while (or perhaps permanently) we say that the fork has produced a new branch. Branches tell monotone which revisions you would like to merge, and which you would like to keep separate.

You can see all the available branches using monotone list branches.

Branches are indicated with certs. The cert name branch is reserved for use by monotone, for the purpose of identifying the revisions which are members of a branch. A branch cert has a symbolic "branch name" as its value. When we refer to "a branch", we mean all revisions with a common branch name in their branch certs.

For example, suppose you are working on a program called "wobbler". You might develop many revisions of wobbler and then decide to split your revisions into a "stable branch" and an "unstable branch", to help organize your work. In this case, you might call the new branches "wobbler-stable" and "wobbler-unstable". From then on, all revisions in the stable branch would get a cert with name branch and value wobbler-stable; all revisions in the unstable branch would get a cert with name branch and value wobbler-unstable. When a wobbler-stable revision forks, the children of the fork will be merged. When a wobbler-unstable revision forks, the children of the fork will be merged. However, the wobbler-stable and wobbler-unstable branches will not be merged together, despite having a common ancestor.

For each branch, the set of revisions with no children is called the heads of the branch. Monotone can automatically locate, and attempt to merge, the heads of a branch. If it fails to automatically merge the heads, it may ask you for assistance or else fail cleanly, leaving the branch alone.

For example, if a fork's left child has a child of its own (a "left grandchild"), monotone will merge the fork's right child with the left grandchild, since those revisions are the heads of the branch. It will not merge the left child with the right child, because the left child is not a member of the heads.

When there is only one revision in the heads of a branch, we say that the heads are merged, or more generally that the branch is merged, since the heads is the logical set of candidates for any merging activity. If there are two or more revisions in the heads of a branch, and you ask to merge the branch, monotone will merge them two-at-a-time until there is only one.

Tutorial

This chapter illustrates the basic uses of monotone by means of an example, fictional software project. Before we walk through the tutorial, there are two minor issues to address: standard options and revision selectors.

Standard Options

Before operating monotone, two inportant command-line options should be explained.

• Most commands operate on a database, which is selected with the --db option.
• Many commands operate on a subset of the database, called a branch, which is selected with the --branch option.

Monotone will cache the settings for these options in your working copy, so ordinarily once you have checked out a project, you will not need to specify them again. We will therefore only mention these arguments in the first example.

Revision Selectors

Many commands require you to supply 40-character SHA1 values as arguments, which identify revisions. These "revision IDs" are tedious to type, so monotone permits you to supply "revision selectors" rather than complete revision IDs. Selectors are a more "human friendly" way of specifying revisions by combining certificate values into unique identifiers. This "selector" mechanism can be used anywhere a revision ID would normally be used. For details on selector syntax, see Selectors.

We are now ready to explore our fictional project.

The Fictional Project

Our fictional project involves 3 programmers cooperating to write firmware for a robot, the JuiceBot 7, which dispenses fruit juice. The programmers are named Jim, Abe and Beth.

• Jim lives in Japan, and owns JuiceBot Inc. You will know when we're talking about Jim, because everything he does involves the letter "j".
• Abe lives in Australia and writes code related to apple juice. You will know when we're talking about Abe, because everything he does involves the letter "a".
• Beth lives in Brazil and writes code related to banana juice. You will know when we're talking about Beth, because everything she does involves the letter "b".

In our example the programmers work privately on laptops, and are usually disconnected from the network. They share no storage system. Thus when each programmer enters a command, it affects only his or her own computer, unless otherwise stated.

Our fictional project will work through the following version control tasks. Some tasks must be done by each member of our example team, other tasks involve only one member.

Creating a Database

The first step Jim, Abe and Beth each need to perform is to create a new database. This is done with the monotone db init command, providing a --db option to speficy the location of the new database. Each programmer creates their own database, which will reside in their home directory and store all the revisions, files and manifests they work on. Monotone requires this step as an explicit command, to prevent spurious creation of databases when an invalid --db option is given.

Thus Jim issues the command:

     $ monotone db init --db=~/jim.db
     

Abe issues the command:

     $ monotone db init --db=~/abe.db
     

And Beth issues the command:

     $ monotone db init --db=~/beth.db
     

Generating Keys

Now Jim, Abe and Beth must each generate an RSA key pair for themselves. This step requires choosing a key identifier. Typical key identifiers are similar to email addresses, possibly modified with some prefix or suffix to distinguish multiple keys held by the same owner. Our example programmers will use their email addresses at the fictional "juicebot.co.jp" domain name. When we ask for a key to be generated, monotone will ask us for a passphrase. This phrase is used to encrypt the key when storing it on disk, as a security measure.

Jim does the following:

     $ monotone --db=~/jim.db genkey jim@juicebot.co.jp
     enter passphrase for key ID [jim@juicebot.co.jp] : <Jim enters his passphrase>
     monotone: generating key-pair 'jim@juicebot.co.jp'
     monotone: storing key-pair 'jim@juicebot.co.jp' in database
     

Abe does something similar:

     $ monotone --db=~/abe.db genkey abe@juicebot.co.jp
     enter passphrase for key ID [abe@juicebot.co.jp] : <Abe enters his passphrase>
     monotone: generating key-pair 'abe@juicebot.co.jp'
     monotone: storing key-pair 'abe@juicebot.co.jp' in database
     

as does Beth:

     $ monotone --db=~/beth.db genkey beth@juicebot.co.jp
     enter passphrase for key ID [beth@juicebot.co.jp] : <Beth enters her passphrase>
     monotone: generating key-pair 'beth@juicebot.co.jp'
     monotone: storing key-pair 'beth@juicebot.co.jp' in database
     

Each programmer has now generated a key pair and placed it in their local database. Each can list the keys in their database, to ensure the correct key was generated. For example, Jim might see this:

     $ monotone --db=~/jim.db list keys
     
     [public keys]
     9e9e9ef1d515ad58bfaa5cf282b4a872d8fda00c jim@juicebot.co.jp
     
     
     [private keys]
     771ace046c27770a99e5fddfa99c9247260b5401 jim@juicebot.co.jp
     

The hexidecimal string printed out before each key name is a fingerprint of the key, and can be used to verify that the key you have stored under a given name is the one you intended to store. Monotone will never permit one database to store two keys with the same name or the same fingerprint.

This output shows one private and one public key stored under the name jim@juicebot.co.jp, so it indicates that Jim's key-pair has been successfully generated and stored. On subsequent commands, Jim will need to re-enter our passphrase in order to perform security-sensitive tasks. To simplify matters, Jim decides to store his security passphrase in his .monotonerc file, by writing a hook function which returns the passphrase, so that he does not need to repeatedly be prompted for it:

     $ cat >~/.monotonerc
     function get_passphrase(keypair_id)
       return "jimsekret"
     end
     ^D
     

Abe and Beth do the same, with their secret passphrases.

Exchanging Keys

Jim, Abe and Beth all wish to work with one another, and trust one another. For monotone to accept this situation, the team members will need to exchange the public parts of their RSA key with each other.

First, Jim exports his public key:

     $ monotone --db=~/jim.db pubkey jim@juicebot.co.jp >~/jim.pubkey
     

His public key is just a plain block of ASCII text:

     $ cat ~/jim.pubkey
     [pubkey jim@juicebot.co.jp]
     MIGdMA0GCSqGSIb3DQEBAQUAA4GLADCBhwKBgQCbaVff9SF78FiB/1nUdmjbU/TtPyQqe/fW
     CDg7hSg1yY/hWgClXE9FI0bHtjPMIx1kBOig09AkCT7tBXM9z6iGWxTBhSR7D/qsJQGPorOD
     DO7xovIHthMbZZ9FnvyB/BCyiibdWgGT0Gtq94OKdvCRNuT59e5v9L4pBkvajb+IzQIBEQ==
     [end]
     

Abe also exports his public key:

     $ monotone --db=~/abe.db pubkey abe@juicebot.co.jp >~/abe.pubkey
     

As does Beth:

     $ monotone --db=~/beth.db pubkey beth@juicebot.co.jp >~/beth.pubkey
     

Then all three team members exchange keys. The keys are not secret, but the team members must be relatively certain that they are communicating with the person they intend to trust, when exchanging keys, and not some malicious person pretending to be a team member. Key exchange may involve sending keys over an encrypted medium, or meeting in person to exchange physical copies, or any number of techniques. All that matters, ultimately, is for each team member to receive the keys of the others.

So eventually, after key exchange, Jim has Beth's and Abe's public key files in his home directory, along with his own. He tells monotone to read the associated key packets into his database:

     $ monotone --db=~/jim.db read <~/abe.pubkey
     monotone: read 1 packet
     $ monotone --db=~/jim.db read <~/beth.pubkey
     monotone: read 1 packet
     

Beth and Abe similarly tell monotone to read read the two new public keys they received into their respective databases.

Adding Files

Next Jim decides to add some files to the project. He writes up a file containing the prototypes for the JuiceBot 7:

     $ mkdir include
     $ cat >include/jb.h
     /* Standard JuiceBot hw interface */
     
     #define FLOW_JUICE 0x1
     #define POLL_JUICE 0x2
     int spoutctl(int port, int cmd, void *x);
     
     /* JuiceBot 7 API */
     
     #define APPLE_SPOUT 0x7e
     #define BANANA_SPOUT 0x7f
     void dispense_apple_juice ();
     void dispense_banana_juice ();
     ^D
     

Then adds a couple skeleton source files which he wants Abe and Beth to fill in:

     $ mkdir src
     $ cat >src/apple.c
     #include "jb.h"
     
     void
     dispense_apple_juice()
     {
       /* Fill this in please, Abe. */
     }
     ^D
     $ cat >src/banana.c
     #include "jb.h"
     
     void
     dispense_banana_juice()
     {
       /* Fill this in please, Beth. */
     }
     ^D
     

Now Jim tells monotone to add these files to its record of his working copy. He specifies one filename and one directory; monotone recursively scans the directory and adds all its files.

     $ monotone --db=~/jim.db add include/jb.h src
     monotone: adding include/jb.h to working copy add set
     monotone: adding src/apple.c to working copy add set
     monotone: adding src/banana.c to working copy add set
     

This command produces a record of Jim's intentions in a special file called MT/work, stored in the working copy. The file is plain text:

     $ cat MT/work
     add_file "include/jb.h"
     
     add_file "src/apple.c"
     
     add_file "src/banana.c"
     

Jim then gets up from his machine to get a coffee. When he returns he has forgotten what he was doing. He asks monotone:

     $ monotone --db=jim.db status
     
     new_manifest [2098eddbe833046174de28172a813150a6cbda7b]
     
     old_revision []
     old_manifest []
     
     add_file "include/jb.h"
     
     add_file "src/apple.c"
     
     add_file "src/banana.c"
     
     patch "include/jb.h"
      from []
        to [3b12b2d0b31439bd50976633db1895cff8b19da0]
     
     patch "src/apple.c"
      from []
        to [2650ffc660dd00a08b659b883b65a060cac7e560]
     
     patch "src/banana.c"
      from []
        to [e8f147e5b4d5667f3228b7bba1c5c1e639f5db9f]
     

The output of this command tells Jim that his edits, so far, constitute only the addition of some files. In the output we can see one pecularity of monotone's changeset format. The pecularity is that when monotone records a "new file", it actually records two separate events: the addition of an empty file to the working copy, and a patch of that file from empty to its intended contents.

Jim wants to see the actual details of the files he added, however, so he runs a command which prints out the status and a GNU "unified diff" of the patches involved in the changeset:

     $ monotone --db=jim.db diff
     #
     # add_file "include/jb.h"
     #
     # add_file "src/apple.c"
     #
     # add_file "src/banana.c"
     #
     # patch "include/jb.h"
     #  from []
     #    to [3b12b2d0b31439bd50976633db1895cff8b19da0]
     #
     # patch "src/apple.c"
     #  from []
     #    to [2650ffc660dd00a08b659b883b65a060cac7e560]
     #
     # patch "src/banana.c"
     #  from []
     #    to [e8f147e5b4d5667f3228b7bba1c5c1e639f5db9f]
     #
     --- include/jb.h
     +++ include/jb.h
     @ -0,0 +1,13 @
     +/* Standard JuiceBot hw interface */
     +
     +#define FLOW_JUICE 0x1
     +#define POLL_JUICE 0x2
     +#define SET_INTR 0x3
     +int spoutctl(int port, int cmd, void *x);
     +
     +/* JuiceBot 7 API */
     +
     +#define APPLE_SPOUT 0x7e
     +#define BANANA_SPOUT 0x7f
     +void dispense_apple_juice ();
     +void dispense_banana_juice ();
     --- src/apple.c
     +++ src/apple.c
     @ -0,0 +1,7 @
     +#include "jb.h"
     +
     +void
     +dispense_apple_juice()
     +{
     +  /* Fill this in please, Abe. */
     +}
     --- src/banana.c
     +++ src/banana.c
     @ -0,0 +1,7 @
     +#include "jb.h"
     +
     +void
     +dispense_banana_juice()
     +{
     +  /* Fill this in please, Beth. */
     +}
     

Committing Work

Satisfied with the work he's done, Jim then commits his working copy, which causes monotone to process the MT/work file and record the file contents, manifest, and revision into the database.

     $ monotone --db=jim.db --branch=co.jp.juicebot.jb7 commit 'initial checkin of project'
     monotone: beginning commit
     monotone: manifest 2098eddbe833046174de28172a813150a6cbda7b
     monotone: revision 2e24d49a48adf9acf3a1b6391a080008cbef9c21
     monotone: branch co.jp.juicebot.jb7
     monotone: committed revision 2e24d49a48adf9acf3a1b6391a080008cbef9c21
     

Monotone did a number of things when committing the new revision. First, we can see from the output that monotone generated a manifest of the tree Jim committed. The manifest is stored inside the database, but Jim can print it out if he wants to see the exact state of all the files referenced by the revision he committed:

     $ monotone cat manifest
     3b12b2d0b31439bd50976633db1895cff8b19da0  include/jb.h
     2650ffc660dd00a08b659b883b65a060cac7e560  src/apple.c
     e8f147e5b4d5667f3228b7bba1c5c1e639f5db9f  src/banana.c
     

The column on the left contains cryptographic hashes of the files listed in the column on the right. Such a hash is also called the "file ID" of the file. The file ID identifies the state of each file stored in Jim's tree. The manifest is just a plain text file, identical to the output from the popular sha1sum unix command.

When monotone committed Jim's revision, it also erased the MT/work file, and wrote a new file called MT/revision, which contains the working copy's new base revision ID. Jim can use this revision ID in the future, as an argument to the checkout command, if he wishes to return to this revision:

     $ cat MT/revision
     2e24d49a48adf9acf3a1b6391a080008cbef9c21
     

Finally, monotone also generated a number of certificates, attached to the new revision. These certs store metadata about the commit. Jim can ask monotone for a list of certs on this revision.

     $ monotone ls certs 2e24d49a48adf9acf3a1b6391a080008cbef9c21
     -----------------------------------------------------------------
     Key   : jim@juicebot.co.jp
     Sig   : ok
     Name  : branch
     Value : co.jp.juicebot.jb7
     -----------------------------------------------------------------
     Key   : jim@juicebot.co.jp
     Sig   : ok
     Name  : date
     Value : 2004-10-26T02:53:08
     -----------------------------------------------------------------
     Key   : jim@juicebot.co.jp
     Sig   : ok
     Name  : author
     Value : jim@juicebot.co.jp
     -----------------------------------------------------------------
     Key   : jim@juicebot.co.jp
     Sig   : ok
     Name  : changelog
     Value : initial checkin of project
     

The output of this command has a block for each cert found. Each block has 4 significant pieces of information. The first indicates the signer of the cert, in this case jim@juicebot.co.jp. The second indicates whether this cert is "ok", meaning whether the RSA signature provided is correct for the cert data. The third is the cert name, and the fourth is the cert value. This list shows us that monotone has confirmed that, according to jim@juicebot.co.jp, the revision 2e24d49a48adf9acf3a1b6391a080008cbef9c21 is a member of the branch jp.co.juicebot.jb7, written by jim@juicebot.co.jp, with the given date and changelog.

It is important to keep in mind that revisions are not "in" or "out" of a branch in any global sense, nor are any of these cert values true or false in any global sense. Each cert indicates that some person - in this case Jim - would like to associate a revision with some value; it is up to you to decide if you want to accept that association.

Jim can now check the status of his branch using the "heads" command, which lists all the head revisions in the branch:

     $ monotone heads
     branch 'co.jp.juicebot.jb7' is currently merged:
     2e24d49a48adf9acf3a1b6391a080008cbef9c21 jim@juicebot.co.jp 2004-10-26T02:53:08
     

The output of this command tells us that there is only one current "head" revision in the branch co.jp.juicebot.jb7, and it is the revision Jim just committed. A head revision is one without any descendents. Since Jim has not committed any changes to this revision yet, it has no descendents.

Network Service

Jim now decides he will make his base revision available to his employees. To do this first adds a small amount of extra information to his .monotonerc file, permitting Abe and Beth to access his database:

     $ cat >>~/.monotonerc
     function get_netsync_read_permitted (collection, identity)
       if (identity == "abe@juicebot.co.jp") then return true end
       if (identity == "beth@juicebot.co.jp") then return true end
       return false
     end
     
     function get_netsync_write_permitted (collection, identity)
       if (identity == "abe@juicebot.co.jp") then return true end
       if (identity == "beth@juicebot.co.jp") then return true end
       return false
     end
     
     function get_netsync_anonymous_read_permitted (collection)
       return false
     end
     ^D
     

He then makes sure that his TCP port 5253 is open to incoming connections, adjusting his firewall settings as necessary, and runs the monotone serve command:

     $ monotone --db=jim.db serve jim-laptop.juicebot.co.jp co.jp.juicebot.jb7
     

This command sets up a single listener loop on the host jim-laptop.juicebot.co.jp, serving the co.jp.juicebot.jb7 collection. This collection will naturally include the co.jp.juicebot.jb7 branch, and any sub-branches.

Now Abe decides he wishes to fetch Jim's code. To do this he issues the monotone sync command:

     monotone --db=abe.db sync jim-laptop.juicebot.co.jp co.jp.juicebot.jb7
     monotone: rebuilding merkle trees for collection co.jp.juicebot.jb7
     monotone: connecting to jim-laptop.juicebot.co.jp
     monotone: [bytes in: 3200] [bytes out: 673]
     monotone: successful exchange with jim-laptop.juicebot.co.jp
     

Abe now has, in his database, a copy of everything Jim put in the branch. Therefore Abe can disconnect from the expensive network connection he's on and work locally for a while. When Abe wants to send work back to Jim, or get new work Jim has added, all he needs to do is run the sync command again and work will flow both ways, bringing each party up to date with the work of the other.

At this point Jim is operating as a sort of "central server" for the company. If Jim wants to, he can leave his server running forever, or even put his server on a dedicated computer with better network connectivity. But if Jim is ever unable to play this role of "central server", perhaps due to a network failure, either Beth or Abe can run the serve command and provide access for the other to sync with. In fact, each employee can run a server if they like, concurrently, to help minimize the risk of service disruption from hardware failures. Changes will flow between servers automatically as clients access them and trade with one another.

Making Changes

Abe decides to do some work on his part of the code. He has a copy of Jim's database contents, but cannot edit any of that data yet. He begins his editing by checking out the head of the co.jp.juicebot.jb7 branch into a working copy, so he can edit it:

     $ monotone --db=abe.db --branch=co.jp.juicebot.jb7 checkout .
     

Monotone unpacks the set of files in the head revision's manifest into Abe's current directory. Abe then opens up one of the files, src/apple.c, and edits it:

     $ vi src/apple.c
     <Abe writes some apple-juice dispensing code>
     

The file src/apple.c has now been changed. Abe gets up to answer a phone call, and when he returns to his work he has forgotten what he changed. He can ask monotone for details:

     $ monotone diff
     #
     # patch "src/apple.c"
     #  from [2650ffc660dd00a08b659b883b65a060cac7e560]
     #    to [e2c418703c863eabe70f9bde988765406f885fd0]
     #
     --- src/apple.c
     +++ src/apple.c
     @ -1,7 +1,10 @
      #include "jb.h"
     
      void
      dispense_apple_juice()
      {
     -  /* Fill this in please, Abe. */
     +  spoutctl(APPLE_SPOUT, FLOW_JUICE, 1);
     +  while (spoutctl(APPLE_SPOUT, POLL_JUICE, 1) == 0)
     +    usleep (1000);
     +  spoutctl(APPLE_SPOUT, FLOW_JUICE, 0);
      }
     

Satisfied with his day's work, Abe decides to commit.

     $ monotone commit
     monotone: beginning commit
     monotone: manifest b33cb337dccf21d6673f462d677a6010b60699d1
     monotone: revision 70decb4b31a8227a629c0e364495286c5c75f979
     monotone: branch co.jp.juicebot.jb7
     

Abe neglected to provide a change log on the command line. Monotone therefore invokes an external "log message editor" -- typically an editor like vi -- with an explanation of the changes being committed and the opportunity to enter a log message.

     polling implementation of src/apple.c
     MT:
     MT: ----------------------------------------------------------------------
     MT: Enter Log.  Lines beginning with `MT:' are removed automatically
     MT:
     MT: new_manifest [b33cb337dccf21d6673f462d677a6010b60699d1]
     MT:
     MT: old_revision [2e24d49a48adf9acf3a1b6391a080008cbef9c21]
     MT: old_manifest [2098eddbe833046174de28172a813150a6cbda7b]
     MT:
     MT: patch "src/apple.c"
     MT: from [2650ffc660dd00a08b659b883b65a060cac7e560]
     MT:   to [e2c418703c863eabe70f9bde988765406f885fd0]
     MT:
     MT: ----------------------------------------------------------------------
     MT:
     

Abe enters a single line above the explanatory message, saying "polling implementation of src/apple.c". He then saves the file and quits the editor. Monotone deletes all the lines beginning with "MT:" and leaves only Abe's short message. Returning to the shell, Abe's commit completes:

     monotone: committed revision 70decb4b31a8227a629c0e364495286c5c75f979
     

Abe then sens his new revision back to Jim:

     $ monotone sync jim-laptop.juicebot.co.jp jp.co.juicebot.jb7
     monotone: rebuilding merkle trees for collection co.jp.juicebot.jb7
     monotone: including branch co.jp.juicebot.jb7
     monotone: [keys: 2] [rcerts: 8]
     monotone: connecting to jim-laptop.juicebot.co.jp
     monotone: [bytes in: 630] [bytes out: 2844]
     monotone: successful exchange with jim-laptop.juicebot.co.jp
     

Beth does a similar sequence, First she syncs her database with Jim's:

     monotone --db=beth.db sync jim-laptop.juicebot.co.jp co.jp.juicebot.jb7
     monotone: rebuilding merkle trees for collection co.jp.juicebot.jb7
     monotone: connecting to jim-laptop.juicebot.co.jp
     monotone: [bytes in: 3200] [bytes out: 673]
     monotone: successful exchange with jim-laptop.juicebot.co.jp
     

Checks out a copy of the tree from her database:

     $ monotone --db=beth.db --branch=co.jp.juicebot.jb7 checkout .
     

Edits the file src/banana.c:

     $ vi src/banana.c
     <Beth writes some banana-juice dispensing code>
     

Commits her work:

     $ monotone commit 'polling implementation of src/banana.c'
     monotone: beginning commit
     monotone: manifest eaebc3c558d9e30db6616ef543595a5a64cc6d5f
     monotone: revision 80ef9c9d251d39074d37e72abf4897e0bbae1cfb
     monotone: branch co.jp.juicebot.jb7
     monotone: committed revision 80ef9c9d251d39074d37e72abf4897e0bbae1cfb
     

And syncs with Jim again:

     $ monotone sync jim-laptop.juicebot.co.jp co.jp.juicebot.jb7
     monotone: rebuilding merkle trees for collection co.jp.juicebot.jb7
     monotone: including branch co.jp.juicebot.jb7
     monotone: [keys: 3] [rcerts: 12]
     monotone: connecting to jim-laptop.juicebot.co.jp
     monotone: [bytes in: 630] [bytes out: 2844]
     monotone: successful exchange with jim-laptop.juicebot.co.jp
     

Dealing with a Fork

Careful readers will note that, in the previous section, the JuiceBot company's work was perfectly serialized:

1. Jim did some work
2. Abe synced with Jim
3. Abe did some work
4. Abe synced with Jim
5. Beth synced with Jim
6. Beth did some work
7. Beth synced with Jim

The result of this ordering is that Jim's work entirely preceeded Abe's work, which entirely preceeded Beth's work. Moreover, each worker was fully informed of the "up-stream" worker's actions, and produced purely derivative, "down-stream" work:

1. Jim made revision 2e24d...
2. Abe changed revision 2e24d... into revision 70dec...
3. Beth derived revision 70dec... into revision 80ef9...

This is a simple, but sadly unrealistic, ordering of events. In real companies or work groups, people often work in parallel, diverging from commonly known revisions and merging their work together, sometime after each unit of work is complete.

Monotone supports this diverge/merge style of operation naturally; any time two revisions diverge from a common parent revision, we say that the revision graph has a fork in it. Forks can happen at any time, and require no coordination between workers. In fact any interleaving of the previous events would work equally well; with one exception: if forks were produced, someone would eventually have to run the merge command, and possibly resolve any conflicts in the fork.

To illustrate this, we return to our workers Beth and Abe. Suppose Jim sends out an email saying that the current polling juice dispensers use too much CPU time, and must be rewritten to use the JuiceBot's interrupt system. Beth wakes up first and begins working immediately, basing her work off the revision 80ef9... which is currently in her working copy:

     $ vi src/banana.c
     <Beth changes her banana-juice dispenser to use interrupts>
     

Beth finishes and examines her changes:

     $ monotone diff
     #
     # patch "src/banana.c"
     #  from [7381d6b3adfddaf16dc0fdb05e0f2d1873e3132a]
     #    to [5e6622cf5c8805bcbd50921ce7db86dad40f2ec6]
     #
     --- src/banana.c
     +++ src/banana.c
     @ -1,10 +1,15 @
      #include "jb.h"
     
     +static void
     +shut_off_banana()
     +{
     +  spoutctl(BANANA_SPOUT, SET_INTR, 0);
     +  spoutctl(BANANA_SPOUT, FLOW_JUICE, 0);
     +}
     +
      void
     -dispense_banana_juice()
     +dispense_banana_juice()
      {
     +  spoutctl(BANANA_SPOUT, SET_INTR, &shut_off_banana);
        spoutctl(BANANA_SPOUT, FLOW_JUICE, 1);
     -  while (spoutctl(BANANA_SPOUT, POLL_JUICE, 1) == 0)
     -    usleep (1000);
     -  spoutctl(BANANA_SPOUT, FLOW_JUICE, 0);
      }
     

Commits her work:

     $ monotone commit 'interrupt implementation of src/banana.c'
     monotone: beginning commit
     monotone: manifest de81e46acb24b2950effb18572d5166f83af3881
     monotone: revision 8b41b5399a564494993063287a737d26ede3dee4
     monotone: branch co.jp.juicebot.jb7
     monotone: committed revision 8b41b5399a564494993063287a737d26ede3dee4
     

And syncs with Jim:

     $ monotone sync jim-laptop.juicebot.co.jp co.jp.juicebot.jb7
     

Unfortunately, before Beth managed to sync with Jim, Abe had woken up and implemented a similar interrupt-based apple juice dispenser, but his working copy is 70dec..., which is still "upstream" of Beth's.

     $ vi apple.c
     <Abe changes his apple-juice dispenser to use interrupts>
     

Thus when Abe commits, he unknowingly creates a fork:

     $ monotone commit 'interrupt implementation of src/apple.c'
     

Abe does not see the fork yet; Abe has not actually seen any of Beth's work yet, because he has not not synchronized with Jim. Since he has new work to contribute, however, he now syncs:

     $ monotone sync jim-laptop.juicebot.co.jp co.jp.juicebot.jb7
     

Now Jim and Abe will be aware of the fork. Jim sees it when he sits down at his desk and asks monotone for the current set of heads of the branch:

     $ monotone heads
     monotone: branch 'co.jp.juicebot.jb7' is currently unmerged:
     39969614e5a14316c7ffefc588771f491c709152 abe@juicebot.co.jp 2004-10-26T02:53:16
     8b41b5399a564494993063287a737d26ede3dee4 beth@juicebot.co.jp 2004-10-26T02:53:15
     

Clearly there are two heads to the branch: it contains an un-merged fork. Beth will not yet know about the fork, but in this case it doesn't matter: anyone can merge the fork, and since there are no conflicts Jim does so himself:

     $ monotone merge
     monotone: merging with revision 1 / 2
     monotone: [source] 39969614e5a14316c7ffefc588771f491c709152
     monotone: [source] 8b41b5399a564494993063287a737d26ede3dee4
     monotone: common ancestor 70decb4b31a8227a629c0e364495286c5c75f979 found
     monotone: trying 3-way merge
     monotone: [merged] da499b9d9465a0e003a4c6b2909102ef98bf4e6d
     

The output of this command shows Jim that two heads were found, combined via a 3-way merge with their ancestor, and saved to a new revision. This happened automatically, because the changes between the common ancestor and heads did not conflict. If there had been a conflict, monotone would have invoked an external merging tool to help resolve it.

After merging, the branch has a single head again, and Jim updates his working copy.

     $ monotone update
     monotone: selected update target da499b9d9465a0e003a4c6b2909102ef98bf4e6d
     monotone: updating src/apple.c to f088e24beb43ab1468d7243e36ce214a559bdc96
     monotone: updating src/banana.c to 5e6622cf5c8805bcbd50921ce7db86dad40f2ec6
     monotone: updated to base revision da499b9d9465a0e003a4c6b2909102ef98bf4e6d
     

The update command selected an update target -- in this case the newly merged head -- and performed an in-memory merge between Jim's working copy and the chosen target. The result was then written to Jim's working copy. If Jim's working copy had any uncommitted changes in it, they would have been merged with the update in exactly the same manner as the merge of multiple committed heads.

Monotone makes very little distinction between a "pre-commit" merge (an update) and a "post-commit" merge. Both sorts of merge use the exact same algorithm. The major difference concerns the recoverability of the pre-merge state: if you commit your work first, and merge after committing, the merge can fail (due to difficulty in a manual merge step) and your committed state is still safe. It is therefore recommended that you commit your work first, before merging.

Advanced Uses

This chapter covers slightly less common aspects of using monotone. Some users of monotone will find these helpful, though possibly not all. We assume that you have read through the taxonomy and tutorial, and possibly spent some time playing with the program to familiarize yourself with its operation.

Selectors

Revisions can be specified on the monotone command line, precisely, by entering the entire 40-character hexidecimal SHA1 code. This can be cumbersome, so monotone also a more general syntax called "selectors" which is less precise but more "human friendly". Any command which expects a precise revision ID can also accept a selector in its place; in fact a revision ID is just a special type of selector which is very precise.

Simple examples

Some selector examples are helpful in clarifying the idea:

a432

Revision IDs beginning with the string a432

graydon@pobox.com/2004-04

Revisions written by graydon@pobox.com in April 2004.

'jrh@nodomain.org/2 weeks ago'

Revisions written by jrh@nodomain.org 2 weeks ago.

graydon/net.venge.monotone.win32/yesterday

Revisions in the net.venge.monotone.win32 branch, written by graydon, yesterday.

A moment's examination reveals that these specifications are "fuzzy" and indeed may return multiple values, or may be ambiguous. When ambiguity arises, monotone will inform you that more detail is required, and list various possibilities. The precise specification of selectors follows.

Selectors in detail

A selector is a combination of a selector type, which is a single ASCII character, followed by a : character and a selector value. The value is matched against identifiers or certs, depending on its type, in an attempt to match a single revision. Selectors are matched as prefixes. The current set of selection types are:

Author selection

Uses selector type a. For example, a:graydon matches author certs where the cert value begins with graydon.

Branch selection

Uses selector type b. For example, b:net.venge matches branch certs where the cert value begins with net.venge.

Date selection

Uses selector type d. For example, d:2004-04 matches date certs where the cert value begins with 2004-04.

Identifier selection

Uses selector type i. For example, i:0f3a matches revision IDs which begin with 0f3a.

Tag selection

Uses selector type t. For example, t:monotone-0.11 matches tag certs where the cert value begins with monotone-0.11.

Further selector types may be added in the future.

Composite selectors

Selectors may be combined with the / character. The combination acts as database intersection (or logical and). For example, the selector a:graydon/d:2004-04 can be used to select a revision which has an author cert beginning with graydon as well as a date cert beginning with 2004-04.

Selector expansion

Before selectors are passed to the database, they are expanded using a lua hook: expand_selector. The default definition of this hook attempts to guess a number of common forms for selection, allowing you to omit selector types in many cases. For example, the hook guesses that the typeless selector jrh@nodomain.org is an author selector, due to its syntactic form, so modifies it to read a:jrh@nodomain.org. This hook will generally assign a selector type to values which "look like" partial hex strings, email addresses, branch names, or date specifications. For the complete source code of the hook, see Hook Reference.

Typeless selection

If, after expansion, a selector still has no type, it is matched as a special "unknown" selector type, which will match either a tag, an author, or a branch. This costs slightly more database access, but often permits simple selection using an author's login name and a date. For example, the selector graydon/net.venge.monotone.win32/yesterday would pass through the selector graydon as an unknown selector; so long as there are no branches or tags beginning with the string graydon this is just as effective as specifying a:graydon.

Quality Assurance

Monotone was constructed to serve both as a version control tool and as a quality assurance tool. The quality assurance features permit users to ignore, or "filter out", versions which do not meet their criteria for quality. This section describes the way monotone represents and reasons about quality information.

Monotone often views the collection of revisions as a directed graph, in which revisions are the nodes and changes between revisions are the edges. We call this the revision graph. The revision graph has a number of important subgraphs, many of which overlap. For example, each branch is a subgraph of the revision graph, containing only the nodes carrying a particular branch cert.

Many of monotone's operations involve searching the revision graph for the ancestors or descendants of a particular revision, or extracting the "heads" of a subgraph, which is the subgraph's set of nodes with no descendants. For example, when you run the update command, monotone searches the subgraph consisting of descendants of the base revision of the current working copy, trying to locate a unique head to update the base revision to.

Monotone's quality assurance mechanisms are mostly based on restricting the subgraph each command operates on. There are two methods used to restrict the subgraph:

• By restricting the set of trusted branch certificates, you can require that specific code reviewers have approved of each edge in the subgraph you focus on.
• By restricting the set of trusted testresult certificates, you can require that the endpoints of an update operation have a certificate asserting that the revision in question passed a certain test, or testsuite.

The evaluation of trust is done on a cert-by-cert basis by calling a set of lua hooks: get_revision_cert_trust, get_manifest_cert_trust and get_file_cert_trust. These hooks are only called when a cert has at least one good signature from a known key, and are passed all the keys which have signed the cert, as well as the cert's id, name and value. The hook can then evaluate the set of signers, as a group, and decide whether to grant or deny trust to the assertion made by the cert.

The evaluation of testresults is controlled by the accept_testresult_change hook. This hook is called when selecting update candidates, and is passed a pair of tables describing the testresult certs present on the source and proposed destination of an update. Only if the change in test results are deemed "acceptable" does monotone actually select an update target to merge into your working copy.

For details on these hooks, see the Hook Reference.

Reserved Files

A monotone working copy consists of control files and non-control files. Each type of file can be versioned or non-versioned. These classifications lead to four groups of files:

• versioned control files
• non-versioned control files
• versioned non-control files
• non-versioned non-control files

Control files contain special content formatted for use by monotone. Versioned files are recorded in a monotone database and have their state tracked as they are modified.

If a control file is versioned, it is considered part of the state of the working copy, and will be recorded as a manifest entry. If a control file is not versioned, it is used to manage the state of the working copy, but it not considered an intrinsic part of it.

Most files you manage with monotone will be versioned non-control files. For example, if you keep source code or documents in a monotone database, they are versioned non-control files. Non-versioned, non-control files in your working copy are generally temporary or junk files, such as backups made by editors or object files made by compilers. Such files are ignored by monotone.

Identifying control files

Control files are identified by their names. Non-control files can have any name except the names reserved for control files. The names of control files follow a regular pattern:

Versioned control files

Any file name beginning with .mt-

Non-versioned control files

Any file in the directory MT/

Existing control files

The following control files are currently used. More control files may be added in the future, but they will follow the patterns given above.

.mt-attrs

Contains versioned attributes of files, associated with the files' pathnames.

MT/revision

Contains the identity of the "base" revision of the working copy. Each working copy has a base revision. When the working copy is committed, the base revision is considered to be the ancestor of the committed revision.

MT/options

Contains "sticky" command-line options such as --db or --branch, such that you do not need to enter them repeatedly after checking out a particular working copy.

MT/work

Contains a list of additions, deletions, and renames which have occurred in the current working copy, relative to the base version.

Reserved Certs

Every certificate has a name. Some names have meaning which is built in to monotone, others may be used for customization by a particular user, site, or community. If you wish to define custom certificates, you should prefix such certificate names with x-. For example, if you want to make a certificate describing the existence of security vulnerabilities in a revision, you might wish to create a certificate called x-vulnerability. Monotone reserves all names which do not begin with x- for possible internal use. If an x- certificate becomes widely used, montotone will likey adopt it as a reserved cert name and standardize its semantics.

Most reserved certificate names have no meaning yet; some do. Usually monotone is also responsible for generating these certificates, so you should generally have no cause to make them yourself. They are described here to help you understand monotone's operation.

The pre-defined, reserved certificate names are:

author

This cert's value is the name of a person who committed the revision the cert is attached to. The cert is generated when you commit a revision. It is displayed by the log command.

branch

This cert's value is the name of a branch. A branch cert associates a revision with a branch. The revision is said to be "in the branch" named by the cert. The cert is generated when you commit a revision, either directly with the commit command or indirectly with the merge or propagage commands. The branch certs are read and directly interpreted by many monotone commands, and play a fundamental role in organizing work in any monotone database.

changelog

This cert's value is the change log message you provide when you commit a revision. It is displayed by the log command.

comment

This cert's value is an additional comment, usually provided after committing, about a revision. Certs with the name comment can be applied to files as well, and will be shown by the log command.

date

This cert's value is an ISO date string indicating the time at which a revision was committed. It is displayed by the log command, and may be used as an additional heuristic or selection criterion in other commands in the future.

tag

This cert's value is a symbolic name given to a revision, which may be used in the future as a way of selecting versions for checkout.

testresult

This cert's value is interpreted as a boolean string, either 0 or 1. It is generated by the testresult command and represents the results of running a particular test on the underlying revision. Typically you will make a separate signing key for each test you intend to run on revisions. This cert influences the update algorithm.

vcheck

This cert contains a seed value and a special message authentication code (MAC) calculated with that seed value over the files of the manifest the cert applies to. It can be used to reduce the chance of accidental collision in SHA1.

Naming Conventions

Some names in monotone are private to your work, such as filenames. Other names are potentially visible outside your project, such as RSA key identifiers or branch names. It is possible that if you choose such names carelessly, you will choose a name which someone else in the world is using, and subsequently you may cause confusion when your work and their is received simultaneously by some third party.

We therefore recommend two naming conventions:

• For RSA keys, use the name of an active email address you own. This will minimize conflicts, and also serves as a mnemonic to associate your personal identity with signatures made with your key. For example, monotone's primary author uses the key identifier graydon@pobox.com.
• For branch names, select any name you like but prefix it with the "inverted domain name" of a DNS domain you control or are otherwise authorized to use. This behavior mimics the package naming convention in the java programming language. For example, monotone itself is developed within the net.venge.monotone branch, because the author owns the DNS domain venge.net.

File Attributes

Monotone contains a mechanism for storing persistent file attributes. These differ from file certificates in an important way: attributes are associated with a path name in your working copy, rather than a particular version of a file. Otherwise they are similar: a file attribute associates a simple name/value pair with a file in your working copy.

The attribute mechanism is motivated by the fact that some people like to store executable programs in version control systems, and would like the programs to remain executable when they check out a working copy. For example, the configure shell script commonly shipped with many programs should be executable.

Similarly, some people would like to store devices, symbolic links, read-only files, and all manner of extra attributes of a file, not directly related to a file's data content.

Rather than try to extend the manifest file format to accommodate attributes, monotone requires that you place your attributes in a specially named file in the root of your working copy. The file is called .mt-attrs, and it has a simple 3-column format, for example:

     execute   true   src/configure
     execute   true   autogen.sh
     

Each line of an .mt-attrs file assigns an attribute to a file in your working copy. The first column of each line contains the name of the attribute, the second column is the value of the attribute, and the third column is the file name within the working copy to which the attribute will be applied.

Every time your working copy is written to, monotone will look for an .mt-attrs file, and if it finds one, run the corresponding hooks registered for each attribute found in the file. This way, you can extend the vocabulary of attributes understood by monotone simply by writing new hooks.

For more information on defining hooks for new attributes, see the attr_functions entry in Hook Reference.

Aside from its special interpretation, the .mt-attrs file is a normal text file. If you want other people to see your attributes, you should add and commit the .mt-attrs file in your working copy. If you make changes to it which conflict with changes other people make, you will have to resolve those conflicts, as plain text, just as with any other text file in your working copy.

Migrating and Dumping

While the state of your database is logically captured in terms of a packet stream, it is sometimes necessary or desirable (especially while monotone is still in active development) to modify the SQL table layout or storage parameters of your version database, or to make backup copies of your database in plain text. These issues are not properly addressed by generating packet streams: instead, you must use migration or dumping commands.

The monotone db migrate command is used to alter the SQL schema of a database. The schema of a monotone database is identified by a special hash of its generating SQL, which is stored in the database's auxiliary tables. Each version of monotone knows which schema version it is able to work with, and it will refuse to operate on databases with different schemas. When you run the migrate command, monotone looks in an internal list of SQL logic which can be used to perform in-place upgrades. It applies entries from this list, in order, attempting to change the database it has into the database it wants. Each step of this migration is checked to ensure no errors occurred and the resulting schema hashes to the intended value. The migration is attempted inside a transaction, so if it fails -- for example if the result of migration hashes to an unexpected value -- the migration is aborted.

If more drastic changes to the underlying database are made, such as changing the page size of sqlite, or if you simply want to keep a plain text version of your database on hand, the monotone db dump command can produce a plain ASCII SQL statement which generates the state of your database. This dump can later be reloaded using the monotone db load command.

Note that when reloading a dumped database, the schema of the dumped database is included in the dump, so you should not try to init your database before a load.

Importing from CVS

Monotone is capable of reading CVS files directly and importing them into a database. This feature is still somewhat immature, but moderately large "real world" CVS trees on the order of 1GB have successfully been imported.

Note however that the machine requirements for CVS trees of this size are not trivial: it can take several hours on a modern system to reconstruct the history of such a tree and calculate the millions of cryptographic certificates involved. We recommend experimenting with smaller trees first, to get a feel for the import process.

We will assume certain values for this example which will differ in your case:

• Your domain name, test.net in this example.
• Your key name, import@test.net in this example.
• Your project name, wobbler in this example.
• Your database name, test.db in this example.
• Your CVS repository path, /usr/local/cvsroot in this example.

Accounting for these differences at your site, the following is an example procedure for importing a CVS repository "from scratch", and checking the resulting head version of the import out into a working copy:

     $ monotone --db=test.db db init
     $ monotone --db=test.db genkey import@test.net
     $ monotone --db=test.db --branch=net.test.wobbler cvs_import /usr/local/cvsroot
     $ monotone --db=test.db --branch=net.test.wobbler checkout wobber-checkout
     

CVS Phrasebook

This chapter translates common CVS commands into monotone commands. It is an easy alternative to reading through the complete command reference.

Checking Out a Tree

$ CVSROOT=:pserver:cvs.foo.com/wobbler $ cvs -d $CVSROOT checkout -r 1.2

$ monotone pull www.foo.com com.foo.wobbler $ monotone checkout fe37 wobbler

The CVS command contacts a network server, retrieves a revision, and stores it in your working copy. There are two cosmetic differences with the monotone command: remote databases are specified by hostnames and collections, and revisions are denoted by SHA1 values (or selectors).

There is also one deep difference: pulling revisions into your database is a separate step from checking out a single revision; after you have pulled from a network server, your database will contain several revisions, possibly the entire history of a project. Checking out is a separate step, after communication, which only copies a particular revision out of your database and into a named directory.

Committing Changes

$ cvs commit -m 'log message'

$ monotone commit 'log message' $ monotone push www.foo.com com.foo.wobbler

As with other networking commands, the communication step with monotone is explicit: committing changes only saves them to the local database. A separate command, push, sends the changes to a remote database.

Incorporating New Changes

$ cvs update -d

$ monotone pull www.foo.com com.foo.wobbler $ monotone merge $ monotone update

This command, like other networking commands, involves a separate communication step with monotone. The extra command, merge, ensures that the branch your are working on has a unique head. You can omit the merge step if you only want update to examine descendants of your base revision, and ignore other heads on your branch.

Viewing Differences

$ cvs diff	$ monotone diff
$ cvs diff -r 1.2 -r 1.4	$ monotone diff 3e7db 278df

Monotone's diff command is modeled on that of CVS, so the main features are the same: diff alone prints the differences between your working copy and its base revision, whereas diff accompanied by two revision numbers prints the difference between those two revisions. The major difference between CVS and monotone here is that monotone's revision numbers are revision IDs, so the diff command prints the difference between the two entire trees.

Showing Working Copy Status

$ cvs status

$ monotone status

This command operates similarly in monotone and CVS. The only major difference is that monotone's status command always gives a status of the whole tree, and outputs a more compact summary than CVS.

Adding Directories and Files to Working Copy

$ cvs add dir $ cvs add dir/subdir $ cvs add dir/subdir/file.txt

$ monotone add dir/subdir/file.txt

Monotone does not explicitly store directories, so adding a file only involves adding the file's complete path, including any directories. Directories are created as needed, and empty directories are ignored.

Removing Directories and Files from Working Copy

$ rm file.txt $ cvs remove file.txt

$ monotone drop file.txt

Monotone does not require that you erase a file from the working copy before you drop it. Dropping a file merely removes its entry in the manifest of the current revision.

Initializing a Repository

$ cvs init -d /path/to/repository

$ monotone db init --db=/path/to/database.db

Monotone's "repository" is a single-file database, which is created and initialized by this command. This file is only ever used by you, and does not need to be in any special location, or readable by other users.

Command Reference

Monotone has a large number of commands. To help navigate through them all, commands are grouped into logical categories.

Tree

monotone cat file id

monotone cat manifest id

These commands write the contents of a specific file or manifest version id to standard output. They are useful if you wish to inspect a version without checking it out into your working copy.

monotone checkout id directory

monotone co id directory

monotone --branch=branchname checkout directory

monotone --branch=branchname co directory

These commands copy a revision id out of your database, writing the string id into the file directory/MT/revision. These commands then copy every file version listed in the revision's manifest to paths under directory. For example, if the revision's manifest contains these entries:

          84e2c30a2571bd627918deee1e6613d34e64a29e  Makefile
          c61af2e67eb9b81e46357bb3c409a9a53a7cdfc6  include/hello.h
          97dfc6fd4f486df95868d85b4b81197014ae2a84  src/hello.c
          

Then the following files are created:

          directory/Makefile
          directory/include/hello.h
          directory/src/hello.c
          

If you wish to checkout in the current directory, you can supply the special name . (a single period) for directory.

If no id is provided, as in the latter two commands, you must provide a branchname; monotone will attempt to infer id as the unique head of branchname if it exists.

monotone heads --branch=branchname

This command lists the "heads" of branchname.

The "heads" of a branch is the set of revisions which are members of the branch, but which have no descendants. These revisions are generally the "newest" revisions committed by you or your colleagues, at least in terms of ancestry. The heads of a branch may not be the newest revisions, in terms of time, but synchronization of computer clocks is not reliable, so monotone usually ignores time.

monotone merge --branch=branchname

This command merges the "heads" of branchname, if there are multiple heads, and commits the results to the database, marking the resulting merged revision as a member of branchname. The merged revision will contain each of the head revision IDs as ancestors.

Merging is performed by repeated pairwise merges: two heads are selected, then their least common ancestor is located in the ancestry graph and these 3 revisions are provided to the built-in 3-way merge algorithm. The process then repeats for each additional head, using the result of each previous merge as an input to the next.

monotone propagate sourcebranch destbranch

This command takes a unique head from sourcebranch and merges it with a unique head of destbranch, using the least common ancestor of the two heads for a 3-way merge. The resulting revision is committed to destbranch. If either sourcebranch or destbranch has multiple heads, propagate aborts, doing nothing.

The purpose of propagate is to copy all the changes on sourcebranch, since the last propagate, to destbranch. This command supports the idea of making separate branches for medium-length development activities, such as maintenance branches for stable software releases, trivial bug fix branches, public contribution branches, or branches devoted to the development of a single module within a larger project.

Working Copy

monotone add filename...

This command places "add" entries for the paths specified in filename... in the working copy's "work list". The work list of your working copy is located at MT/work, and is a list of explicit pathname changes you wish to commit at some future time, such as addition, removal or renaming of files.

While this command places an "add" entry on your work list, it does not immediately affect your database. When you commit your working copy, monotone will use the work list to build a new revision, which it will then commit to the database. The new revision will have any added entries inserted in its manifest.

monotone drop filename...

This command places "drop" entries for the paths specified in filename... in the working copy's "work list". The work list of your working copy is located at MT/work, and is a list of explicit pathname changes you wish to commit at some future time, such as addition, removal, or renaming of files.

While this command places a "drop" entry on your work list, it does not immediately affect your database. When you commit your working copy, monotone will use the work list to build a new revision, which it will then commit to the database. The new revision will have any dropped entries removed from its manifest.

monotone rename src dst

This command places "rename" entries for the paths specified in src and dst in the working copy's "work list". The work list of your working copy is located at MT/work, and is a list of explicit pathname changes you wish to commit at some future time, such as addition, removal, or renaming of files.

While this command places a "rename" entry on your work list, it does not immediately affect your database. When you commit your working copy, monotone will use the work list to build a new revision, which it will then commit to the database. The new revision will have any renamed entries in its manifest adjusted to their new names.

monotone commit

monotone commit logmsg

This command looks at your working copy, decides which files have changed, and saves the changes to your database. It does this by loading the revision named in the MT/revision file, locating the base manifest for your working copy, applying any changes described in the MT/work file, and then comparing the updated base manifest to the files it finds in your working copy, to determine which files have been edited.

For each edited file, a delta is copied into the database. Then the newly constructed manifest is recorded (as a delta) and finally the new revision. Once all these objects are recorded in you database, commit overwrites the MT/revision file with the new revision ID, and deletes the MT/work file.

The commit command also synthesizes a number of certificates, which it attaches to the new manifest version and copies into your database:

• An author cert, indicating the person responsible for the changes leading to the new revision.
• A branch cert, indicating the branch the committed revision belongs to.
• A date cert, indicating when the new revision was created.
• A changelog cert, containing the "log message" for these changes. If you provided logmsg on the command line, this text will be used, otherwise commit will run the Lua hook edit_comment (commentary), which typically invokes an external editor program, in which you can compose a log message for the change.

monotone revert

monotone revert filename...

With no files given, this command changes your working copy, so that changes you have made since the last checkout or update are discarded. It does this by changing every file listed in the working copy's base manifest to have contents equal to the SHA1 value listed in the manifest, and by erasing the MT/work file.

If files are given as arguments, only those files are affected instead of the entirety of your working copy.

monotone update

This command incorporates "recent" changes found in your database into your working copy. It does this by performing 3 separate stages. If any of these stages fails, the update aborts, doing nothing. The stages are:

• Examine the ancestry graph of revisions in your database, and (subject to trust evaluation) select the set of all descendents of your working copy's base revision. Call this set the "candidates" of the update.
• Remove any candidates which lack acceptable testresult certificates. From the remaining candidates, select the deepest child by ancestry and call it the "target" of the update.
• Merge the target of the update with the working copy, in memory, and if the merge is successful, write the result over top of the working copy.

Network

monotone serve address collection

monotone pull address collection

monotone push address collection

monotone sync address collection

These commands operate the "netsync" protocol built into monotone. This is a custom protocol for rapidly synchronizing two monotone databases using a hash tree index. The protocol is "peer to peer", but requires one peer to listen for incoming connections (the server) and the other peer (the client) to connect to the server.

The network address specified in each case should be the same: a host name to listen on, or connect to, optionally followed by a colon and a port number. The collection parameter indicates a set of branches to exchange; every branch for which collection is a prefix will be indexed and made available for synchronization.

For example, supposing bob and alice wish to synchronize their net.venge.monotone.win32 and net.venge.monotone.i18n branches. Supposing alice's computer has hostname alice.someisp.com, then alice might run:

          $ monotone serve alice.someisp.com net.venge.monotone
          

And bob might run

          $ monotone sync alice.someisp.com net.venge.monotone
          

When the operation completes, all branches beginning with net.venge.monotone will be synchronized between alice and bob's databases.

Informative

monotone status

This command prints a description of the "status" of your working copy. In particular, it prints:

• The "base revision ID" and "base manifest ID", which are referenced by the file MT/revision, and which your working copy is an in-progress descendant of.
• The "current manifest ID", which is the ID of the manifest which results from applying MT/work to the base manifest, and updating any SHA1 values of files to reflect changes you have made to the working copy. In other words, the current manifest ID is the ID which would accompany any revision you would commit, if you ran monotone commit.
• A list of logical changes between the base and current manifest versions, such as adds, drops, renames, and patches.

monotone log

monotone log id

This command prints out a log, in reverse-ancestry order, of small history summaries. Each summary contains author, date, changelog and comment information associated with a revision. If id is given, the command starts tracing back through history from the revision id, otherwise it starts from the base revision of your working copy.

monotone complete file partial-id

monotone complete manifest partial-id

monotone complete revision partial-id

These commands print out all known completions of a partial SHA1 value, listing completions which are file, manifest or revision IDs depending on which variant is used. For example, suppose you enter this command and get this result:

          $ monotone complete manifest fa36
          fa36deead87811b0e15208da2853c39d2f6ebe90
          fa36b76dd0139177b28b379fe1d56b22342e5306
          fa36965ec190bee14c5afcac235f1b8e2239bb2a
          

Then monotone is telling you that there are 3 manifests it knows about, in its database, which begin with the 4 hex digits fa36. This command is intended to be used by programmable completion systems, such as those in bash and zsh.

monotone diff

monotone diff id

monotone diff id1 id2

These commands print out GNU "unified diff format" textual difference listings between various manifest versions. If no extra arguments are given, monotone diff will print the differences between the base revision and the current revision in your working copy. If one argument id is given, monotone will print the differences between the revision id and the current revision in your working copy. If two arguments id1 and id2 are given, monotone will print the differences between revisions id1 and id2, ignoring your working copy.

In all cases, monotone will print a textual summary - identical to the summary presented by monotone status - of the logical differences between revisions in lines proceeding the diff. These lines begin with a single hash mark #, and should be ignored by a program processing the diff, such as patch.

monotone list certs id

These commands will print out a list of certificates associated with a particular revision id. Each line of the print out will indicate:

• Whether the signature on the certificate is ok or bad
• The key ID of the signer of the certificate
• The name of the certificate
• The value of the certificate

For example, this command lists the certificates associated with a particular version of monotone itself, in the monotone development branch:

          $ ./monotone list certs 4a96
          monotone: expandeding partial id '4a96'
          monotone: expanded to '4a96a230293456baa9c6e7167cafb3c5b52a8e7f'
          -----------------------------------------------------------------
          Key   : graydon@pobox.com
          Sig   : ok
          Name  : author
          Value : graydon@dub.venge.net
          -----------------------------------------------------------------
          Key   : graydon@pobox.com
          Sig   : ok
          Name  : branch
          Value : monotone
          -----------------------------------------------------------------
          Key   : graydon@pobox.com
          Sig   : ok
          Name  : date
          Value : 2003-10-17T03:20:27
          -----------------------------------------------------------------
          Key   : graydon@pobox.com
          Sig   : ok
          Name  : changelog
          Value : 2003-10-16  graydon hoare  <graydon@pobox.com>
                :
                :         * sanity.hh: Add a const version of idx().
                :         * diff_patch.cc: Change to using idx() everywhere.
                :         * cert.cc (find_common_ancestor): Rewrite to recursive
                :         form, stepping over historic merges.
                :         * tests/t_cross.at: New test for merging merges.
                :         * testsuite.at: Call t_cross.at.
                :
          

monotone list keys

monotone list keys pattern

These commands list RSA keys held in your current database. They do not print out any cryptographic information; they simply list the names of public and private keys you have on hand.

If pattern is provided, it is used as a glob to limit the keys listed. Otherwise all keys in your database are listed.

monotone list branches

This command lists all known branches in your database.

monotone list unknown

This command lists all files in your working copy that monotone is either ignoring or knows nothing about.

monotone list ignored

This command lists all files in your working copy that monotone is intentionally ignoring, due to the results of the ignore_file (filename) hook.

monotone list missing

This command lists all files in your working copy's base manifest, which are not present in the working copy.

Key and Cert

monotone genkey keyid

This command generates an RSA public/private key pair, using a system random number generator, and stores it in your database under the key name keyid. If the the hook non_blocking_rng_ok() returns true, the key generation will use an unlimited random number generator (such as /dev/urandom), otherwise it will use a higher quality random number generator (such as /dev/random) but might run slightly slower.

The private half of the key is stored in an encrypted form, using the symmetric cipher ARC4, so that anyone accidentally reading your database cannot extract your private key and use it. You must provide a passphrase for your key when it is generated, which is used to key the ARC4 cipher. In the future you will need to enter this passphrase again each time you sign a certificate, which happens every time you commit to your database. You can tell monotone to automatically use a certain passphrase for a given key using the get_passphrase(keypair_id), but this significantly increases the risk of a key compromise on your local computer. Be careful using this hook.

monotone cert id certname

monotone cert id certname certval

These commands create a new certificate with name certname, for a revision with version id. If certval is provided, it is the value of the certificate. Otherwise the certificate value is read from stdin.

monotone --branch=branchname vcheck create

monotone vcheck create id

monotone --branch=branchname vcheck check

monotone vcheck check id

These commands produce or check vcheck certs for the given manifest id. A vcheck cert on a manifest simply reduces the probability that you and your colleague have different manifests or files with the same (colliding) SHA1 value. You can produce as many vcheck certs as you like.

If you omit the id, these commands operate on the heads of branchname, which you must provide.

Certificate

monotone approve id1 id2

This command is a synonym for monotone cert id2 ancestor id1

monotone disapprove id1 id2

This command is a synonym for monotone cert id2 disapproval id1

monotone comment id

monotone comment id comment

These commands are synonyms for monotone cert id comment comment. If comment is not provided, it is read from stdin.

monotone tag id tagname

This command is a synonym for monotone cert id tag tagname.

monotone testresult id 0

monotone testresult id 1

These commands are synonyms for monotone cert id testresult 0 or monotone cert id testresult 1.

Packet I/O

Monotone can produce and consume data in a convenient, portable form called packets. A packet is a sequence of ASCII text, wrapped at 70-columns and easily sent through email or other transports. If you wish to manually transmit a piece of information - for example a public key - from one monotone database to another, it is often convenient to read and write packets.

Note: earlier versions of monotone queued and replayed packet streams for their networking system. This older networking system is deprecated and will be removed in a future version, as the netsync protocol has several properties which make it advantageous as a communication system. However, the packet i/o facility will remain in monotone as a utility for moving individual data items around manually.

monotone fcerts id

monotone mcerts id

monotone rcerts id

These commands print out an fcert, mcert or rcert packet for each file, manifest, or revision cert in your database associated with id. These can be used to transport certificates safely between monotone databases.

monotone fdata id

monotone mdata id

monotone rdata id

These commands print out an fdata, mdata or rdata packet for the file, manifest or revision id in your database. These can be used to transport files, manifests or revisions, in their entirety, safely between monotone databases.

monotone fdelta id1 id2

monotone mdelta id1 id2

These commands print out an fdelta or mdelta packet for the differences between file or manifest versions id1 and id2, in your database. These can be used to transport file or manifest differences safely between monotone databases.

monotone privkey keyid

monotone pubkey keyid

These commands print out an privkey or pubkey packet for the RSA key keyid. These can be used to transport public or private keys safely between monotone databases.

monotone read

This command reads packets from stdin and applies them to your database.

Database

monotone db init --db=dbfile

This command initializes a new monotone database at dbfile.

monotone db info --db=dbfile

This command prints information about the monotone database dbfile, including its schema version and various table size statistics.

monotone db version --db=dbfile

This command prints out just the schema version of the monotone database dbfile.

monotone db dump --db=dbfile

This command dumps an SQL statement representing the entire state of dbfile to the standard output stream. It is a very low-level command, and produces the most "recoverable" dumps of your database possible. It is sometimes also useful when migrating databases between variants of the underlying sqlite database format.

monotone db load --db=dbfile

This command applies a raw SQL statement, read from the standard input stream, to the database dbfile. It is most useful when loading a database dumped with the dump command.

Note that when reloading a dumped database, the schema of the dumped database is included in the dump, so you should not try to init your database before a load.

monotone db migrate --db=dbfile

This command attempts to migrate the database dbfile to the newest schema known by the version of monotone you are currently running. If the migration fails, no changes should be made to the database.

If you have important information in your database, you should back up a copy of it before migrating, in case there is an untrapped error during migration.

monotone db execute sql-statement

This is a debugging command which executes sql-statement against your database, and prints any results of the expression in a tabular form. It can be used to investigate the state of your database, or help diagnose failures.

RCS

monotone rcs_import filename...

This command imports all the file versions in each RCS file listed in filename.... These files should be raw RCS files, ending in ,v. Monotone parses them directly and inserts them into your database.

monotone cvs_import pathname...

This command imports all the file versions in each RCS file found in the tree of files starting at pathname..., then reconstructs the tree-wide history of logical changes by comparing RCS time stamps and change log entries. For each logical tree-wide change, monotone synthesizes a manifest and revision, and commits them (along with all associated file deltas) to your database. It also copies all change log entries, author identifiers, and date stamps to manifest certificates.

Hook Reference

Monotone's behavior can be customized and extended by writing hook functions, which are written in the Lua programming language. At certain points in time, when monotone is running, it will call a hook function to help it make a decision or perform some action. If you provide a hook function definition which suits your preferences, monotone will execute it. This way you can modify how monotone behaves.

You can put new definitions for any of these hook functions in a file $HOME/.monotonerc, or in your working copy in MT/monotonerc, both of which will be read every time monotone runs. Definitions in MT/monotonerc shadow (override) definitions made in your $HOME/.monotonerc. You can also tell monotone to interpret extra hook functions from any other file using the --rcfile=file option; hooks defined in files specified on the command-line will shadow hooks from the the automatic files.

The remainder of this section documents the existing hook functions and their default definitions.

note_commit (new_id, certs)

Called by monotone after the version new_id is committed. The second parameter, certs, is a lua table containing the set of certificate names and values committed along with this version. There is no default definition for this hook.

Note that since the certs table does not contain cryptographic or trust information, and only contains one entry per cert name, it is an incomplete source of information about the committed version. This hook is only intended as an aid for integrating monotone with informal commit-notification systems such as mailing lists or news services. It should not perform any security-critical operations.

get_branch_key (branchname)

Returns a string which is the name of an RSA private key used to sign certificates in a particular branch branchname. There is no default definition for this hook. The command-line option --key=keyname overrides any value returned from this hook function. If you have only one private key in your database, you do not need to define this function or provide a --key=keyname option; monotone will guess that you want to use the unique private key.

get_passphrase (keypair_id)

Returns a string which is the passphrase used to encrypt the private half of keypair_id in your database, using the ARC4 symmetric cipher. This hook has no default definition. If this hook is not defined or returns false, monotone will prompt you for a passphrase each time it needs to use a private key.

get_author (branchname)

Returns a string which is used as a value for automatically generated author certificates when you commit changes to branchname. Generally this hook remains undefined, and monotone selects your signing key name for the author certificate. You can use this hook to override that choice, if you like.

This hook has no default definition, but a possible definition might be:

          function get_author(branchname)
                  local user = os.getenv("USER")
                  local host = os.getenv("HOSTNAME")
                  if ((user == nil) or (host == nil)) then return nil end
                  return string.format("%s@%s", user, host)
          end
          

edit_comment (commentary)

Returns a log entry for a given set of changes, described in commentary. The commentary is identical to the output of monotone status. This hook is intended to interface with some sort of editor, so that you can interactively document each change you make. The result is used as the value for a changelog certificate, automatically generated when you commit changes.

The default definition of this hook is:

          function edit_comment(commentary)
                  local exe = "vi"
                  local visual = os.getenv("VISUAL")
                  if (visual ~= nil) then exe = visual end
                  local editor = os.getenv("EDITOR")
                  if (editor ~= nil) then exe = editor end
          
                  local tmp, tname = temp_file()
                  if (tmp == nil) then return nil end
                  commentary = "MT: " .. string.gsub(commentary, "\n", "\nMT: ")
                  tmp:write(commentary)
                  io.close(tmp)
          
                  if (os.execute(string.format("%s %s", exe, tname)) ~= 0) then
                          os.remove(tname)
                          return nil
                  end
          
                  tmp = io.open(tname, "r")
                  if (tmp == nil) then os.remove(tname); return nil end
                  local res = ""
                  local line = tmp:read()
                  while(line ~= nil) do
                          if (not string.find(line, "^MT:")) then
                                  res = res .. line .. "\n"
                          end
                          line = tmp:read()
                  end
                  io.close(tmp)
                  os.remove(tname)
                  return res
          end
          

persist_phrase_ok ()

Returns true if you want monotone to remember the passphrase of a private key for the duration of a single command, or false if you want monotone to prompt you for a passphrase for each certificate it generates. Since monotone often generates several certificates in quick succession, unless you are very concerned about security you probably want this hook to return true.

The default definition of this hook is:

          function persist_phrase_ok()
                  return true
          end
          

non_blocking_rng_ok ()

Returns true if you are willing to let monotone use the system's non-blocking random number generator, such as /dev/urandom, for generating random values during cryptographic operations. This diminishes the cryptographic strength of such operations, but speeds them up. Returns false if you want to force monotone to always use higher quality random numbers, such as those from /dev/random.

The default definition of this hook is:

          function non_blocking_rng_ok()
                  return true
          end
          

get_netsync_read_permitted (collection, identity)

Returns true if a peer authenticated as key identity should be allowed to read from your database certs, revisions, manifests, and files associated with the netsync index collection; otherwise false. This hook has no default definition, therefore the default behavior is to deny all reads.

Note that the identity value is a key id (such as "graydon@pobox.com") but will correspond to a unique key fingerprint (hash) in your database. Monotone will not permit two keys in your database to have the same id. Make sure you confirm the key fingerprints of each key in your database, as key id strings are "convenience names", not security tokens.

get_netsync_anonymous_read_permitted (collection)

This hook has identical semantics to get_netsync_read_permitted except that it is called when a connecting client requests anonymous read access to a collection. There is no corresponding anonymous write access hook. This hook has no default definition, therefore the default behavior is to deny all anonymous reads.

get_netsync_write_permitted (collection, identity)

Returns true if a peer authenticated as key identity should be allowed to write into your database certs, revisions, manifests, and files associated with the netsync index collection; otherwise false. This hook has no default definition, therefore the default behavior is to deny all writes.

Note that the identity value is a key id (such as "graydon@pobox.com") but will correspond to a unique key fingerprint (hash) in your database. Monotone will not permit two keys in your database to have the same id. Make sure you confirm the key fingerprints of each key in your database, as key id strings are "convenience names", not security tokens.

ignore_file (filename)

Returns true if filename should be ignored while adding, dropping, or moving files. Otherwise returns false. This is most important when performing recursive actions on directories, which may affect multiple files simultaneously. The default definition of this hook is:

          function ignore_file(name)
             if (string.find(name, "%.a$")) then return true end
             if (string.find(name, "%.so$")) then return true end
             if (string.find(name, "%.o$")) then return true end
             if (string.find(name, "%.la$")) then return true end
             if (string.find(name, "%.lo$")) then return true end
             if (string.find(name, "%.aux$")) then return true end
             if (string.find(name, "%.bak$")) then return true end
             if (string.find(name, "%.orig$")) then return true end
             if (string.find(name, "%.rej$")) then return true end
             if (string.find(name, "%~$")) then return true end
             if (string.find(name, "/core$")) then return true end
             if (string.find(name, "^CVS/")) then return true end
             if (string.find(name, "/CVS/")) then return true end
             if (string.find(name, "^%.svn/")) then return true end
             if (string.find(name, "/%.svn/")) then return true end
             if (string.find(name, "^SCCS/")) then return true end
             if (string.find(name, "/SCCS/")) then return true end
             return false;
          end
          

ignore_branch (branchname)

Returns true if branchname should be ignored while listing branches. Otherwise returns false. This hook has no default definition, therefore the default behavior is to list all branches.

get_revision_cert_trust (signers, id, name, val)

Returns whether or not you trust the assertion name=value on a given revision id, given a valid signature from all the keys in signers. The signers parameter is a table containing all the key names which signed this cert, the other three parameters are strings.

The default definition of this hook simply returns true, which corresponds to a form of trust where every key which is defined in your database is trusted. This is a weak trust setting; you should change it to something stronger. A possible example of a stronger trust function (along with a utility function for computing the intersection of tables) is the following:

          function intersection(a,b)
             local s={}
             local t={}
             for k,v in pairs(a) do s[v] = 1 end
             for k,v in pairs(b) do if s[v] ~= nil then table.insert(t,v) end end
             return t
          end
          
          function get_revision_cert_trust(signers, id, name, val)
             local trusted_signers = { "bob@happyplace.com",
                                       "friend@trustedplace.com",
                                       "myself@home.com" }
             local t = intersection(signers, trusted_signers)
          
             if t == nil then return false end
          
             if    (name ~= "ancestor" and table.getn(t) >= 1)
                or (name == "ancestor" and table.getn(t) >= 2)
             then
                return true
             else
                return false
             end
          end
          

In this example, any revision certificate is trusted if it is signed by at least one of three "trusted" keys, unless it is an ancestor certificate, in which case it must be signed by two or more trusted keys. This is one way of requiring that ancestry assertions go through an extra "reviewer" before they are accepted.

get_manifest_cert_trust (signers, id, name, val)

get_file_cert_trust (signers, id, name, val)

These hooks are identical to the hook get_revision_cert_trust, except they is called for manifest or file certificates rather than revision certificates. See the documentation for that hook for more details.

accept_testresult_change (old_results, new_results)

This hook is used by the update algorithm to determine whether a change in test results between update source and update target is acceptable. The hook is called with two tables, each of which maps a signing key - representing a particular testsuite - to a boolean value indicating whether or not the test run was successful. The function should return true if you consider an update from the version carrying the old_results to the version carrying the new_results to be acceptable.

The default definition of this hook follows:

          function accept_testresult_change(old_results, new_results)
             for test,res in pairs(old_results)
             do
                if res == true and new_results[test] ~= true
                then
          	 return false
                end
             end
             return true
          end
          

This definition accepts only those updates which preserve the set of true test results from update source to target. If no rest results exist, this hook has no affect; but once a true test result is present, future updates will require it. If you want a more lenient behavior you must redefine this hook.

merge2 (left, right)

Returns a string, which should be the merger of the 2 provided strings, which are the contents of the left and right nodes of a file fork which monotone was unable to automatically merge. The merge should either call an intelligent merge program or interact with the user. The default definition of this hook is:

          function merge2(left, right)
             local lfile = nil
             local rfile = nil
             local outfile = nil
             local data = nil
          
             lfile = write_to_temporary_file(left)
             rfile = write_to_temporary_file(right)
             outfile = write_to_temporary_file("")
          
             if lfile ~= nil and
                rfile ~= nil and
                outfile ~= nil
             then
                local cmd = nil
                if program_exists_in_path("xxdiff") then
                   cmd = merge2_xxdiff_cmd(lfile, rfile, outfile)
                elseif program_exists_in_path("emacs") then
                   cmd = merge2_emacs_cmd("emacs", lfile, rfile, outfile)
                elseif program_exists_in_path("xemacs") then
                   cmd = merge2_emacs_cmd("xemacs", lfile, rfile, outfile)
                end
          
                if cmd ~= nil
                then
                   io.write(
                      string.format("executing external 2-way merge command: %s\n",
                                    cmd))
                   os.execute(cmd)
                   data = read_contents_of_file(outfile)
                else
                   io.write("no external 2-way merge command found")
                end
             end
          
             os.remove(lfile)
             os.remove(rfile)
             os.remove(outfile)
          
             return data
          end
          

merge3 (ancestor, left, right)

Returns a string, which should be the merger of the 3 provided strings, which are the contents of left and right nodes, and least common ancestor, of a file fork which monotone was unable to automatically merge. The merge should either call an intelligent merge program or interact with the user. The default definition of this hook is:

          function merge3(ancestor, left, right)
             local afile = nil
             local lfile = nil
             local rfile = nil
             local outfile = nil
             local data = nil
          
             lfile = write_to_temporary_file(left)
             afile = write_to_temporary_file(ancestor)
             rfile = write_to_temporary_file(right)
             outfile = write_to_temporary_file("")
          
             if lfile ~= nil and
                rfile ~= nil and
                afile ~= nil and
                outfile ~= nil
             then
                local cmd = nil
                if program_exists_in_path("xxdiff") then
                   cmd = merge3_xxdiff_cmd(lfile, afile, rfile, outfile)
                elseif program_exists_in_path("emacs") then
                   cmd = merge3_emacs_cmd("emacs", lfile, afile, rfile, outfile)
                elseif program_exists_in_path("xemacs") then
                   cmd = merge3_emacs_cmd("xemacs", lfile, afile, rfile, outfile)
                end
          
                if cmd ~= nil
                then
                   io.write(
                      string.format("executing external 3-way merge command: %s\n",
                                    cmd))
                   os.execute(cmd)
                   data = read_contents_of_file(outfile)
                else
                   io.write("no external 3-way merge command found")
                end
             end
          
             os.remove(lfile)
             os.remove(rfile)
             os.remove(afile)
             os.remove(outfile)
          
             return data
          end
          

expand_selector (str)

Attempts to expand str as a selector. Expansion generally means providing a type prefix for the selector, such as a: for authors or d: for dates. Expansion may also mean recognizing and interpreting special words such as yesterday or 6 months ago and converting them into well formed selectors. For more detail on the use of selectors, see Selectors. The default definition of this hook is:

          function expand_selector(str)
          
             -- simple date patterns
             if string.find(str, "^19%d%d%-%d%d")
                or string.find(str, "^20%d%d%-%d%d")
             then
                return ("d:" .. str)
             end
          
             -- something which looks like an email address
             if string.find(str, "[%w%-_]+@[%w%-_]+")
             then
                return ("a:" .. str)
             end
          
             -- something which looks like a branch name
             if string.find(str, "[%w%-]+%.[%w%-]+")
             then
                return ("b:" .. str)
             end
          
             -- a sequence of nothing but hex digits
             if string.find(str, "^%x+$")
             then
                return ("i:" .. str)
             end
          
             -- "yesterday", the source of all hangovers
             if str == "yesterday"
             then
                local t = os.time(os.date('!*t'))
                return os.date("d:%F", t - 86400)
             end
          
             -- "CVS style" relative dates such as "3 weeks ago"
             local trans = {
                minute = 60;
                hour = 3600;
                day = 86400;
                week = 604800;
                month = 2678400;
                year = 31536000
             }
             local pos, len, n, type = string.find(str, "(%d+)
                                                   ([minutehordaywk]+)s? ago")
             if trans[type] ~= nil
             then
                local t = os.time(os.date('!*t'))
                return os.date("d:%F", t - (n * trans[type]))
             end
          
             return nil
          end
          

get_system_linesep ()

Returns a string which defines the default system line separator. This should be one of the strings CR, LF, or CRLF. The system line separator may be used when reading or writing data to the terminal, or otherwise interfacing with the user. The system line separator is not used to convert files in the working copy; use get_linesep_conv for converting line endings in the working copy.

This hook has no default definition. For more information on line ending conversion, see the section on Internationalization.

get_linesep_conv (filename)

Returns a table which contains two strings. The first string in the return value is the name of a line ending convention to use for the "internal" representation of filename. The second string in the return value is the name of a line ending convention to use for the "external" representation of filename. Line ending conventions should be one of the strings CR, LF, or CRLF.

When filename is read from the working copy, it is run through line ending conversion from the external form to the internal form. When filename is written to the working copy, it is run through line ending conversion from the internal form to the external form. SHA1 values are calculated from the internal form of filename. It is your responsibility to decide which line ending conversions your work will use.

This hook has no default definition; monotone's default behavior is to keep external and internal forms byte-for-byte identical. For more information on line ending conversion, see the section on Internationalization.

get_charset_conv (filename)

Returns a table which contains two strings. The first string in the return value is the name of a character set to use for the "internal" representation of filename. The second string in the return value is the name of a character set to use for the "external" representation of filename.

When filename is read from the working copy, it is run through character set conversion from the external form to the internal form. When filename is written to the working copy, it is run through character set conversion from the internal form to the external form. SHA1 values are calculated from the internal form of filename. It is your responsibility to decide which character set conversions your work will use.

This hook has no default definition; monotone's default behavior is to keep external and internal forms byte-for-byte identical. For more information on character set conversion, see the section on Internationalization.

attr_functions [arrtibute] (filename, value)

This is not a hook function, but a table of hook functions. Each entry in the table attr_functions, at table entry attribute, is a function taking a file name filename and a attribute value value. The function should "apply" the attribute to the file, possibly in a platform-specific way.

Persistent attributes are stored in the .mt-attrs, in your working copy and manifest. If such a file exists, hook functions from this table are called for each triple found in the file, after any command which modifies the working copy. This facility can be used to extend monotone's understanding of files with platform-specific attributes, such as permission bits, access control lists, or special file types.

By default, there is only one entry in this table, for the execute attribute. Its definition is:

          attr_functions["execute"] =
            function(filename, value)
                  if (value == "true") then
                          os.execute(string.format("chmod +x %s", filename))
                  end
            end
          

Special Topics

This chapter describes some "special" issues which are not directly related to monotone's use, but which are occasionally of interest to people researching monotone or trying to learn the specifics of how it works. Most users can ignore these sections.

Internationalization

Monotone initially dealt with only ASCII characters, in file path names, certificate names, key names, and packets. Some conservative extensions are provided to permit internationalized use. These extensions can be summarized as follows:

• Monotone uses GNU gettext to provide localized progress and error messages. Translations may or may not exist for your locale, but the infrastructure is present to add them.
• All command-line arguments are mapped from your local character set to UTF-8 before processing. This means that monotone can only handle key names, file names and certificate names which map cleanly into UTF-8.
• Monotone's control files are stored in UTF-8. This includes: revisions and manifests, both inside the database and when written to the MT/ directory of the working copy; the MT/options and MT/work files; and the .mt-attrs file. Converting these files to any other character set will cause monotone to break; do not do so.
• File path names in the working copy are converted to the locale's character set (determined via the LANG or CHARSET environment variables) before monotone interacts with the file system. If you are accustomed to being able to use file names in your locale's character set, this should "just work" with monotone.
• Key and cert names, and similar "name-like" entities are subject to some cleaning and normalization, and conversion into network-safe subsets of ASCII (typically ACE). Generally, you should be able to use "sensible" strings in your locale's character set as names, but they may appear mangled or escaped in certain contexts such as network transmission.
• Monotone's transmission and storage forms are otherwise unchanged. Packets and database contents are 7-bit clean ASCII.

The remainder of this section is a precise specification of monotone's internationalization behavior.

General Terms

Character set conversion

The process of mapping a string of bytes representing wide characters from one encoding to another. Per-file character set conversions are specified by a Lua hook get_charset_conv which takes a filename and returns a table of two strings: the first represents the "internal" (database) charset, the second represents the "external" (file system) charset.

Line ending conversion

The process of converting platform-dependent end-of-line codes (0x0D, 0x0A, or the pair 0x0D 0x0A) from one convention to another. Per-file line ending conversion is specified by a Lua hook get_linesep_conv which takes a filename and returns a table of two strings: the first represents the "internal" (database) line ending convention, the second represents the "external" (file system) line ending convention. each string should be one of the three strings "CR", "LF", or "CRLF".

Note that Line ending conversion is always performed on the internal character set, when both character set and line ending conversion are enabled; this behavior is meant to encourage the use of the monotone's "normal form" (UTF-8, '\n') as an internal form for your source files, when working with multiple external forms. Also note that line ending conversion only works on character encodings with the specific code bytes described above, such as ASCII, ISO-8859x, and UTF-8.

Normal form conversion

Character set and line ending conversions done between a "system form" and a "normal form". The system character set form is inferred from the environment, using the various locale environment variables. The system line ending form can be additionally specialized using the get_system_linesep hook. No hooks exist for adjusting the system character set, since the system character set must be known during command-line argument processing, before any Lua hooks are loaded.

Monotone's normal form is the UTF-8 character set and the 0x0A (LF) line ending form. This form is used in any files monotone needs to read, write, and interpret itself, such as: MT/revision, MT/work, MT/options, .mt-attrs

LDH

Letters, digits, and hyphen: the set of ASCII bytes 0x2D, 0x30..0x39, 0x41..0x5A, and 0x61..0x7A.

stringprep

RFC 3454, a general framework for mapping, normalizing, prohibiting and bidirectionality checking for international names prior to use in public network protocols.

nameprep

RFC 3491, a specific profile of stringprep, used for preparing international domain names (IDNs)

punycode

RFC 3492, a "bootstring" encoding of unicode into ASCII.

IDNA

RFC 3490, international domain names for applications, a combination of the above technologies (nameprep, punycoding, limiting to LDH characters) to form a specific "ASCII compatible encoding" (ACE) of unicode, signified by the presence of an "unlikely" ACE prefix string "xn-". IDNA is intended to make it possible to use unicode relatively "safely" over legacy ASCII-based applications. the general picture of an IDNA string is this:

                {ACE-prefix}{LDH-sanitized(punycode(nameprep(UTF-8-string)))}
          

It is important to understand that IDNA encoding does not preserve the input string: it both prohibits a wide variety of possible strings and normalizes non-equal strings to supposedly "equivalent" forms.

By default, monotone does not decode IDNA when printing to the console (IDNA names are ASCII, which is a subset of UTF-8, so this normal form conversion can still apply, albeit oddly). this behavior is to protect users against security problems associated with malicious use of "similar-looking" characters. If the hook display_decoded_idna returns true, IDNA names are decoded for display.

Filenames

• Filenames are subject to normal form conversion.
• Filenames are subject to an additional normal form stage which adjusts for platform name semantics, for example changing the Windows 0x5C '\' path separator to 0x2F '/'. This extra processing is performed by boost::filesystem.
• FIXME: Monotone does not properly handle case insensitivity on windows.
• A filename (in normal form) is constrained to be a nonempty sequence of path components, separated by byte 0x2F (ASCII / ), and without a leading or trailing 0x2F.
• A path component is a nonempty sequence of any UTF-8 character codes except the path separator byte 0x2F and any ASCII "control codes" (0x00..0x1F and 0x7F).
• The path components "." and ".." are prohibited.
• Manifests and revisions are constructed from the normal form (UTF-8). The LC_COLLATE locale category is not used to sort manifest or revision entries.

File contents

• Files are subject to character set conversion and line ending conversion.
• File SHA1 values are calculated from the internal form of the conversions. If the external form of a file differs from the internal form, running a 3rd party program such as sha1sum will produce different results than those entries shown in a corresponding manifest.

UI messages

UI messages are displayed via calls to gettext().

Host names

Host names are read on the command-line and subject to normal form conversion. Host names are then split at 0x2E (ASCII '.'), each component is subject to IDNA encoding, and the components are rejoined.

After processing, host names are stored internally as ASCII. The invariant is that a host name inside monotone contains only sequences of LDH separated by 0x2E.

Cert names

Read on the command line and subject to normal form conversion and IDNA encoding as a single component. The invariant is that a cert name inside monotone is a single LDH ASCII string.

Cert values

Cert values may be either text or binary, depending on the return value of the hook cert_is_binary. If binary, the cert value is never printed to the screen (the literal string "<binary>" is displayed, instead), and is never subjected to line ending or character conversion. If text, the cert value is subject to normal form conversion, as well as having all UTF-8 codes corresponding to ASCII control codes (0x0..0x1F and 0x7F) prohibited in the normal form, except 0x0A (ASCII LF).

Key names

Read on the command line and subject to normal form conversion and IDNA encoding as an email address (split and joined at '.' and '@' characters). The invariant is that a key name inside monotone contains only LDH, 0x2E (ASCII '.') and 0x40 (ASCII '@') characters.

Packets

Packets are 7-bit ASCII. The characters permitted in packets are the union of these character sets:

• The 65 characters of base64 encoding (64 coding + "=" pad).
• The 16 characters of hex encoding.
• LDH, '@' and '.' characters, as required for key and cert names.
• '[' and ']', the packet delimiters.
• ASCII codes 0x0D (CR), 0x0A (LF), 0x09 (HT), and 0x20 (SP).

The .mt-attrs file

Now uses 0x0A (ASCII LF) as a delimiter, to permit 0x20 in filenames. This may change in the future.

Hash Integrity

Some proponents of a competing, proprietary version control system have suggested, in a usenix paper, that the use of a cryptographic hash function such as SHA1 as an identifier for a version is unacceptably unsafe. This section addresses the argument presented in that paper and describes monotone's additional precautions.

To summarize our position:

• the analysis in the paper is wrong,
• even if it were right, monotone is sufficiently safe.

The analysis is wrong

The paper displays a fundamental lack of understanding about what a cryptographic hash function is, and how it differs from a normal hash function. Furthermore it confuses accidental collision with attack scenarios, and mixes up its analysis of the risk involved in each. We will try to untangle these issues here.

A cryptographic hash function such as SHA1 is more than just a uniform spread of inputs to an output range. Rather, it must be designed to withstand attempts at:

• reversal: deriving an input value from the output
• collision: finding two different inputs which hash to the same output

Collision is the problem the paper is concerned with. Formally, an n-bit cryptographic hash should cost 2^n work units to collide against a given value, and sqrt(2^n) tries to find a random pair of colliding values. This latter probability is sometimes called the hash's "birthday paradox probability".

Accidental collision

One way of measuring these bounds is by measuring how single-bit changes in the input affect bits in the hash output. The SHA1 hash has a strong avalanche property, which means that flipping any one bit in the input will cause on average half the 160 bits in the output code to change. The fanciful VAL1 hash presented in the paper does not have such a property -- flipping its first bit when all the rest are zero causes no change to any of the 160 output bits -- and is completely unsuited for use as a cryptographic hash, regardless of the general shape of its probability distribution.

The paper also suggests that birthday paradox probability cannot be used to measure the chance of accidental SHA1 collision on "real inputs", because birthday paradox probability assumes a uniformly random sample and "real inputs" are not uniformly random. The paper is wrong: the inputs to SHA1 are not what is being measured (and in any case can be arbitrarily long); the collision probability being measured is of output space. On output space, the hash is designed to produce uniformly random spread, even given nearly identical inputs. In other words, it is a primary design criterion of such a hash that a birthday paradox probability is a valid approximation of its collision probability.

The paper's characterization of risk when hashing "non-random inputs" is similarly deceptive. It presents a fanciful case of a program which is storing every possible 2kb block in a file system addressed by SHA1 (the program is trying to find a SHA1 collision). While this scenario will very likely encounter a collision somewhere in the course of storing all such blocks, the paper neglects to mention that we only expect it to collide after storing about 2^80 of the 2^16384 possible such blocks (not to mention the requirements for compute time to search, or disk space to store 2^80 2kb blocks).

Noting that monotone can only store 2^41 bytes in a database, and thus probably some lower number (say 2^32 or so) active rows, we consider such birthday paradox probability well out of practical sight. Perhaps it will be a serious concern when multi-yottabyte hard disks are common.

Collision attacks

Setting aside accidental collisions, then, the paper's underlying theme of vulnerability rests on the assertion that someone will break SHA1. Breaking a cryptographic hash usually means finding a way to collide it trivially. While we note that SHA1 has in fact resisted attempts at breaking for 8 years already, we cannot say that it will last forever. Someone might break it. We can say, however, that finding a way to trivially collide it only changes the resistance to active attack, rather than the behavior of the hash on benign inputs.

Therefore the vulnerability is not that the hash might suddenly cease to address benign blocks well, but merely that additional security precautions might become a requirement to ensure that blocks are benign, rather than malicious. The paper fails to make this distinction, suggesting that a hash becomes "unusable" when it is broken. This is plainly not true, as a number of systems continue to get useful low collision hashing behavior -- just not good security behavior -- out of "broken" cryptographic hashes such as MD4.

Monotone is probably safe anyways

Perhaps our arguments above are unconvincing, or perhaps you are the sort of person who thinks that practice never lines up with theory. Fair enough. Below we present practical procedures you can follow to compensate for the supposed threats presented in the paper.

Accidental collision

To minimize the chance of accidental collision further, monotone has a special type of certificate called a vcheck, which you can issue at any time and in any quantity you like. A vcheck certificate contains a randomly keyed message authentication code (MAC). This effectively means that a vcheck cert identifies your tree again, with a "different" variant of SHA1.

If you have the SHA1 of your manifest and a vcheck cert, the probability of a collision involving that manifest drops from 2^-160 to 2^-320. If you have two vcheck certs, it drops to 2^-480, and so on. You can issue as many vcheck certs as you like until your personal level of paranoia is met. Each vcheck cert has its own random seed, usually 128 bits, so after you've issued about 2^64 vcheck certs on the same manifest you have a decent chance of repeating one, but that is harmless.

Adding the occasional vcheck cert to your tree is a good idea if you are worried about the "deterministic" nature of accidental collisions in SHA1, outlined in the paper. Since the MAC is randomized, unintentional (non-attack) collisions in vcheck certs are reliably nondeterministic. Furthermore, they minimize the chance that a malicious collision has gone undetected, by requiring the collision to pass an additional (and equally hard) check. A large warning is printed if such a check fails.

Collision attacks

A successful collision attack on SHA1, as mentioned, does not disrupt the probability features of SHA1 on benign blocks. So if, at any time, you believe SHA1 is "broken", it does not mean that you cannot use it for your work with monotone. It means, rather, that you cannot base your trust on SHA1 values anymore. You must trust who you communicate with.

The way around this is reasonably simple: if you do not trust SHA1 to prevent malicious blocks from slipping into your communications, you can always augment it by enclosing your communications in more security, such as tunnels or additional signatures on your email posts. If you choose to do this, you will still have the benefit of self-identifying blocks, you will simply cease to trust such blocks unless they come with additional authentication information.

If in the future SHA1 (or, indeed, RSA) becomes accepted as broken we will naturally upgrade monotone to a newer hash or public key scheme, and provide migration commands to recalculate existing databases based on the new algorithm.

Similarly, if you do not trust our vigilance in keeping up to date with cryptography literature, you can modify monotone to use any stronger hash you like, at the cost of isolating your own communications to a group using the modified version. Monotone is free software, and runs atop crypto++, so it is both legal and relatively simple to change it to use some other algorithm.

Man Page

NAME

monotone - distributed version control system

SYNOPSIS

monotone [options] <command> [parameters]

Options, which affect global behavior or set default values, come first in the argument list. A single command must follow, indicating the operation to perform, followed by parameters which vary depending on the command.

Note

This man page is a summary of some of the features and commands of monotone, but it is not the most detailed source of information available. For a complete discussion of the concepts and a tutorial on its use, please refer to the texinfo manual (via the info monotone command, or online).

Commands

comment <id>

Write a comment cert for a revision.

approve <id1>

Make a "branch" cert approving of a revision's membership in a branch.

disapprove <id1>

Disapprove of a revision, committing the inverse changes as as a descendant of the disapproved revision.

tag <id> <tagname>

Put a symbolic tag cert on a revision.

testresult <id> (0|1)

Indicate a passing or failing test result on a revision.

agraph

Dump revision graph to stdout.

diff

Show diffs between working copy and database.

status

Show status of working copy.

log [id]

Show historical log of revisions, starting from working copy base revision, or [id] if given.

cert <id> <certname> [certval]

Create a custom cert for a revision. Reads cert value from stdin if no value given on command line.

vcheck (create|check) [id]

Produce or check vcheck certs for the given manifest [id].

genkey <keyid>

Generate an RSA key-pair and store it in the database.

list certs <id>

ls certs <id>

List certs associated with revision.

list keys [partial-id]

ls keys [partial-id]

List keys matching glob, or list all keys if no glob given.

list branches

ls branches

List all branches.

list unknown

ls unknown

List files in working directory, but not in revision's manifest or work list.

list ignored

ls ignored

List files intentionally ignored due to the ignore_file hook.

list missing

ls missing

List files in revision's manifest, but not in working directory.

fcerts <id>

Write file cert packets to stdout.

fdata <id>

Write file data packet to stdout.

fdelta <oldid> <newid>

Write file delta packet to stdout.

mcerts <id>

Write manifest cert packets to stdout.

mdata <id>

Write manifest data packet to stdout.

mdelta <oldid> <newid>

Write manifest delta packet to stdout.

rcerts <id>

Write revision cert packets to stdout.

rdata <id>

Write revision data packet to stdout.

privkey <id>

Write private key packet to stdout.

pubkey <id>

Write public key packet to stdout.

read

Read packets from stdin. It is very important to only read keys from trusted sources; all other trust relationships are built out of the trust assigned to keys.

cvs_import <cvsroot>

Import all versions in CVS repository. Reconstructs revisions and converts metadata to certificates. A private signing key must already exist in the database.

rcs_import <rcsfile> ...

Import all file versions in RCS files. Does not reconstruct revisions across the entire tree.

checkout [manifest-id] <directory>

co [manifest-id] <directory>

Check out tree state from database, into directory.

cat (file|manifest|revision) <id>

Write file, manifest or revision from database to stdout.

heads

Show unmerged heads of branch, or report when branch is merged.

merge

Merge unmerged heads of branch.

add <pathname> [...]

Add files to working copy. adding a file does not copy it into the database, merely adds it to the work list. You must commit your changes in order to copy added files to the database.

commit [log message]

Commit working copy to database. If a log message is provided on the command line, it is used; otherwise a log message editor will be invoked.

drop <pathname> [...]

Drop files from working copy. Files are not deleted from working copy, merely noted as removals in the work list.

rename <src> <dst>

Rename files from <src> to <dst> in working copy.

update

Update working copy.

push <host> <collection>

Push contents of <collection> to database on <host>

pull <host> <collection>

Pull contents of <collection> from database on <host>

sync <host> <collection>

Sync contents of <collection> with database on <host>

serve <host> <collection>

Serve contents of <collection> at network address <host>

db (init|info|version|dump|load|migrate|execute <sql>)

Manipulate database state.

DESCRIPTION

Monotone is a version control system, which allows you to keep old versions of files, as well as special revisions and manifests which describe the edit history, location, and content of files in a tree. Unlike other systems, versions in monotone are identified by cryptographic hash, and operations are authenticated by individual users' evaluating cryptographic signatures on meta-data, rather than any central authority.

Monotone keeps a collection of versions in a single-file relational database. It is essentially serverless, using network servers only as untrusted communication facilities. A monotone database is a regular file, which contains all the information needed to extract previous versions of files, verify signatures, merge and modify versions, and communicate with network servers.

OPTIONS

--help

Print help message.

--debug

Turn on debugging log on standard error stream. This is very verbose. Default is to be silent, unless an error occurs, in which case failure log is dumped.

--quiet

Turn off normal progress messages.

--dump=<file>

Dump debugging log to <file> on failure.

--nostd

Do not evaluate "standard" Lua hooks compiled into monotone.

--norc

Do not load Lua hooks from user's ~/.monotonerc file.

--rcfile=<file>

Load extra Lua hooks from file (may be given multiple times).

--db=<file>

Use database in file.

--key=<keyid>

Use keyid for operations which produce RSA signatures. Default is inferred from presence of unique private key in database. Can also be customized on a per-branch basis with hook function get_branch_key(branchname).

--branch=<branchname>

Use branchname for operations on a branch. Default is inferred in operations on existing branches (commit, update, etc).

--ticker=[dot|count]

Use the given method to print tickers. The count method prints the count for each ticker on one line, incrementing the numbers in place, while the dot method prints a continuous string of characters (like some programs provide a progress line of dots). The default is count.

ENVIRONMENT

EDITOR

Used to edit comments, log messages, etc.

VISUAL

Used in preference to EDITOR, if set.

FILES

$HOME/.monotonerc

A Lua script, used as a customization file.

NOTES

Command line options override environment variables and settings in Lua scripts (such as .monotonerc)

SEE ALSO

info monotone

BUGS

see http://savannah.nongnu.org/bugs/?group=monotone

AUTHOR

graydon hoare <graydon@pobox.com>

Index

• accept_testresult_change (old_results, new_results): Hook Reference
• attr_functions [arrtibute] (filename, value): Hook Reference
• edit_comment (commentary): Hook Reference
• expand_selector (str): Hook Reference
• get_author (branchname): Hook Reference
• get_branch_key (branchname): Hook Reference
• get_charset_conv (filename): Hook Reference
• get_file_cert_trust (signers, id, name, val): Hook Reference
• get_linesep_conv (filename): Hook Reference
• get_manifest_cert_trust (signers, id, name, val): Hook Reference
• get_netsync_anonymous_read_permitted (collection): Hook Reference
• get_netsync_read_permitted (collection, identity): Hook Reference
• get_netsync_write_permitted (collection, identity): Hook Reference
• get_passphrase (keypair_id): Hook Reference
• get_revision_cert_trust (signers, id, name, val): Hook Reference
• get_system_linesep (): Hook Reference
• ignore_branch (branchname): Hook Reference
• ignore_file (filename): Hook Reference
• merge2 (left, right): Hook Reference
• merge3 (ancestor, left, right): Hook Reference
• monotone --branch=branchname checkout directory: Tree
• monotone --branch=branchname co directory: Tree
• monotone --branch=branchname vcheck check: Key and Cert
• monotone --branch=branchname vcheck create: Key and Cert
• monotone add filename...: Working Copy
• monotone approve id1 id2: Certificate
• monotone cat file id: Tree
• monotone cat manifest id: Tree
• monotone cert id certname: Key and Cert
• monotone cert id certname certval: Key and Cert
• monotone checkout id directory: Tree
• monotone co id directory: Tree
• monotone comment id: Certificate
• monotone comment id comment: Certificate
• monotone commit: Working Copy
• monotone commit logmsg: Working Copy
• monotone complete file partial-id: Informative
• monotone complete manifest partial-id: Informative
• monotone complete revision partial-id: Informative
• monotone cvs_import pathname...: RCS
• monotone db dump --db=dbfile: Database
• monotone db execute sql-statement: Database
• monotone db info --db=dbfile: Database
• monotone db init --db=dbfile: Database
• monotone db load --db=dbfile: Database
• monotone db migrate --db=dbfile: Database
• monotone db version --db=dbfile: Database
• monotone diff: Informative
• monotone diff id: Informative
• monotone diff id1 id2: Informative
• monotone disapprove id1 id2: Certificate
• monotone drop filename...: Working Copy
• monotone fcerts id: Packet I/O
• monotone fdata id: Packet I/O
• monotone fdelta id1 id2: Packet I/O
• monotone genkey keyid: Key and Cert
• monotone heads --branch=branchname: Tree
• monotone list branches: Informative
• monotone list certs id: Informative
• monotone list ignored: Informative
• monotone list keys: Informative
• monotone list keys pattern: Informative
• monotone list missing: Informative
• monotone list unknown: Informative
• monotone log: Informative
• monotone log id: Informative
• monotone mcerts id: Packet I/O
• monotone mdata id: Packet I/O
• monotone mdelta id1 id2: Packet I/O
• monotone merge --branch=branchname: Tree
• monotone privkey keyid: Packet I/O
• monotone propagate sourcebranch destbranch: Tree
• monotone pubkey keyid: Packet I/O
• monotone pull address collection: Network
• monotone push address collection: Network
• monotone rcerts id: Packet I/O
• monotone rcs_import filename...: RCS
• monotone rdata id: Packet I/O
• monotone read: Packet I/O
• monotone rename src dst: Working Copy
• monotone revert: Working Copy
• monotone revert filename...: Working Copy
• monotone serve address collection: Network
• monotone status: Informative
• monotone sync address collection: Network
• monotone tag id tagname: Certificate
• monotone testresult id 0: Certificate
• monotone testresult id 1: Certificate
• monotone update: Working Copy
• monotone vcheck check id: Key and Cert
• monotone vcheck create id: Key and Cert
• non_blocking_rng_ok (): Hook Reference
• note_commit (new_id, certs): Hook Reference
• persist_phrase_ok (): Hook Reference