Monotone Documentation
Monotone is a distributed version control tool. It can help automate many tedious and error-prone tasks in group software development.
Please be aware that monotone is a slightly unorthodox version control tool, and many of its concepts are slightly similar -- but significantly different -- from concepts with similar names in other version control tools.
Complete table of contents
This chapter should familiarize you with the concepts, terminology, and behavior described in the remainder of the user manual. Please take a moment to read it, as later sections will assume familiarity with these terms.
Suppose you wish to modify a file file.txt on your
computer. You begin with one version of the file, load it into
an editor, make some changes, and save the file again. Doing so
produces a new version of the file. We will say that the older
version of the file was a parent, and the new version is a
child, and that you have performed an edit between the
parent and the child. We may draw the relationship between parent and
child using a graph, where the arrow in the graph indicates the
direction of the edit, from parent to child.
+----------------+
| |
| parent version |
| of file.txt |
| |
+----------------+
|
|
\|/
V
+----------------+
| |
| child version |
| of file.txt |
| |
+----------------+
We may want to identify the parent and the child precisely, for sake of reference. To do so, we will compute a cryptographic hash function, called SHA1, of each version. The details of this function are beyond the scope of this document; in summary, the SHA1 function takes a version of a file and produces a short string of 20 bytes, which we will use to uniquely identify the version1. Now our graph does not refer to some "abstract" parent and child, but rather to the exact edit we performed between a specific parent and a specific child.
+---------------------------------------------------+
| parent version |
| of file.txt |
| |
| SHA1 = 65f1bde1f38262034e7c3457301e8f736ba6381b |
+---------------------------------------------------+
|
|
\|/
V
+---------------------------------------------------+
| child version |
| of file.txt |
| |
| SHA1 = a91566316d208dc405795904f8d67ae3a0e765cb |
+---------------------------------------------------+
When dealing with versions of files, we will dispense with writing out "file names", and identify versions purely by their SHA1 value, which we will also refer to as their version code. Using version codes alone will often help us accommodate the fact that people often wish to call files by different names. So now our graph of parent and child is just a relationship between two versions, only identified by version code.
+---------------------------------------------------+
| parent version |
| SHA1 = 65f1bde1f38262034e7c3457301e8f736ba6381b |
+---------------------------------------------------+
|
|
\|/
V
+---------------------------------------------------+
| child version |
| SHA1 = a91566316d208dc405795904f8d67ae3a0e765cb |
+---------------------------------------------------+
Version control systems, such as monotone, are principally concerned
with the storage and management of multiple versions of some
files. One way to store multiple versions of a file is, literally, to
save a separate complete copy of the file, every time you make a
change. When necessary, monotone will save complete copies of your
files in their, compressed with the zlib compression format.
+--------------+ +--------------+ +--------------+
| | | | | |
| Hello | | Hello, | | Why, Hello |
| | | world! | | there world, | . . .
| | | | | how do you |
| | | | | do? |
| | | | | |
+--------------+ +--------------+ +--------------+
\______ ______/ \______ ______/ \______ ______/
\/ \/ \/
1st version 2nd version 3rd version
Often we find that successive versions of a file are very similar to one another, so storing multiple complete copies is a waste of space. In these cases, rather than store complete copies of each version of a file, we store a compact description of only the changes which are made between versions. Such a description of changes is called a delta.
Storing deltas between files is, practically speaking, as good as
storing complete versions of files. It lets you undo changes from a
new version, by applying the delta backwards, and lets your friends
change their old version of the file into the new version, by applying
the delta forwards. Deltas are usually smaller than full files, so
when possible monotone stores deltas, using a modified xdelta
format. The details of this format are beyond the scope of this
document.
difference
between versions
_____/\____
+--------------+ / \ +--------------+
| | | |
| Hello | +[, world!] | Hello, |
| | | world! |
| | | |
| | | |
| | | |
+--------------+ +--------------+
\______ ______/ \______ ______/
\/ \/
1st version 2nd version
After you have made many different files, you may wish to capture a "snapshot" of the versions of all the files in a particular collection. Since files are typically collected into trees in a file system, we say that you want to capture a version of your tree. Doing so will permit you to undo changes to multiple files at once, or send your friend a set of changes to many files at once.
To make a snapshot of a tree, we begin by writing a special file called a manifest. In fact, monotone will write this file for us, but we could write it ourselves too. It is just a plain text file. Each line of a manifest file contains two columns. In the first column we write the version code of a file in your tree, and in the second column we write the path to the file, from the root of our tree to the filename.
+--------------------------------------------------------------+
| |
| f2e5719b975e319c2371c98ed2c7231313fac9b5 fs/readdir.c |
| 81f0c9a0df254bc8d51bb785713a9f6d0b020b22 fs/read_write.c |
| 943851e7da46014cb07473b90d55dd5145f24de0 fs/pipe.c |
| ddc2686e000e97f670180c60a3066989e56a11a3 fs/open.c |
| 295d276e6c9ce64846d309a8e39507bcb0a14248 fs/namespace.c |
| 71e0274f16cd68bdf9a2bf5743b86fcc1e597cdc fs/namei.c |
| 1112c0f8054cebc9978aa77384e3e45c0f3b6472 fs/iobuf.c |
| 8ddcfcc568f33db6205316d072825d2e5c123275 fs/inode.c |
| |
+--------------------------------------------------------------+
\_____________________________ _______________________________/
\/
an example
manifest file
Now we note that a manifest is itself a file. Therefore a manifest can serve as input to the SHA1 function, and thus every manifest has a version code of its own. By calculating the SHA1 value of a manifest, we capture the state of our tree in a single version code. In other words, the version code of the manifest essentially captures all the version codes and file names of every file in our tree, combined. So we may treat manifests and their version codes as snapshots of a tree of files, though lacking the actual contents of the files themselves.
+--------------------------+
| int readdir(...) { |
| ... |
| } |
+--------------------------+
\____________ ___________/
\/
SHA1
||
+--------------||----------------------------------------------+
| \/ |
| f2e5719b975e319c2371c98ed2c7231313fac9b5 fs/readdir.c |
| 81f0c9a0df254bc8d51bb785713a9f6d0b020b22 fs/read_write.c |
| 943851e7da46014cb07473b90d55dd5145f24de0 fs/pipe.c |
| . . . . . . |
+--------------------------------------------------------------+
\_____________________________ _______________________________/
\/
SHA1
||
||
\/
manifest version:
a2eeaa28574141a7d48fa1cc2802070150b93ec4
As with versions of files, we may decide to store manifests in their entirety, or else we may store only a compact description of changes which occur between different versions of manifests. As with files, when possible monotone stores compact descriptions of changes between manifests; when necessary it stores complete versions of manifests.
Often, you will wish to make a statement about a version of a file or manifest, such as stating the reason that you made some changes, or stating the fact that you approve of the changes which were made, or stating the time at which you made the changes. Statements such as these can be thought of, generally, as a bundle of information with three parts:
For example, if you want to say that a particular version of a manifest was written on April 4, 2003, you might make a statement like this:
+------------------------------------------------------+
| version code |
| a2eeaa28574141a7d48fa1cc2802070150b93ec4 |
+--------------------------+---------------------------+
| statement name | statement value |
| "date" | "2003-04-04T07:39:51" |
+--------------------------+---------------------------+
In an ideal world, these are all the parts of a statement we would need in order to go about our work. In the real world, however, there are sometimes malicious people who would make false or misleading statements; so we need a way to verify that a particular person made a particular statement about a version. We therefore will add two more pieces of information to our bundle:
When these 2 items accompany a statement, we call the total bundle of 5 items a certificate, or cert. A cert makes a statement in a secure fashion. The security of the signature in a cert is derived from the RSA cryptography system, the details of which are beyond the scope of this document.
+------------------------------------------------------+
| version code |
| a2eeaa28574141a7d48fa1cc2802070150b93ec4 |
+--------------------------+---------------------------+
| cert name | cert value |
| "date" | "2003-04-04T07:39:51" |
+--------------------------+---------------------------+
| signed by key | signature |
| "jrh@somewhere.com" | "a02380def....0983fe90" |
+--------------------------+---------------------------+
Monotone uses certs extensively. Any extra information which needs to be stored, transmitted or retrieved -- above and beyond versions of files and manifests -- is kept in the form of certs. This includes ancestry information, change logs, time and date records, and more. When monotone makes a decision about storing, transmitting, or recovering some files or manifests, the decision is based on certs it has seen, and the trustworthiness you assign to those certs.
The RSA cryptography system -- and therefore monotone itself -- requires that you exchange special "public" numbers with your friends, before they will trust certificates signed by you. These numbers are called public keys. Giving someone your public key does not give them the power to impersonate you, only to verify signatures made by you. Exchanging public keys should be done over a trusted medium, in person, or via a trusted third party. Advanced secure key exchange techniques are beyond the scope of this document.
Monotone moves information in and out of three different types of storage:
All information passes through your local database, en route to some other destination. For example, when changes are made in a working copy, you may save those changes to your database, and later you may synchronize your database with someone else's. Monotone will not move information directly between a working copy and a remote database, or between working copies. Your local database is always the "switching point" for communication.
pull, push, sync
(untrusted network exchanges)
_________/\________
/ \
+-------------+ +------------+ +--------------+
| | | | | |
| remote db | <==> | local db | <==> | working copy |
| | | | | |
+-------------+ +------------+ +--------------+
\________ _______/
\/
commit, update
(certified local exchanges)
A working copy is a tree of files in your file system, arranged
according to the paths and version codes in a particular manifest.
A special directory called MT exists in the root of any
working copy. Monotone keeps some special files in the MT
directory, in order to track changes you make to your working copy.
Aside from the special MT directory, a working copy is just a
normal tree of files. You can directly edit the files in a working copy
using a plain text editor or other program; monotone will
automatically notice when you make any changes. If you wish to add
files, remove files, or move files within your working copy, you may
need to tell monotone explicitly what you are doing, as these actions
cannot always be deduced.
If you do not yet have a working copy, you can check out a working copy from a database, or construct one from scratch and add it into a database. As you work, you will occasionally commit changes you have made in a working copy to a database, and update a working copy to receive changes that have arrived in a database. Committing and updating take place purely between a database and a working copy; the network is not involved.
----------------- check out, working copy
( ) or update +----------------
----------------- ----------> |
| | | src/func.c
| local | <---------- | src/func.h
| database | add, | src/main.c
| | or commit | Makefile
\_________________/ | MT/
|
A database is a single, regular file. You can copy or back it up using standard methods. Typically you keep a database in your home directory. Databases are portable between different machine types. If a database grows too big, you may choose to remove information from it. You can have multiple databases and divide your work between them, or keep everything in a single database if you prefer. You can dump portions of your database out as text, and read them back into other databases, or send them to your friends.
A database contains many versions and certificates, some of which are not immediately of interest, some of which may be unwanted or even false. It is a collection of information received from network servers, working copies, and other databases. You can inspect and modify your databases without affecting your working copies, and vice-versa.
Monotone knows how to exchange information in your database with other remote databases, using an interactive protocol called netsync. It supports three modes of exchange: pushing, pulling, and synchronizing. A pull operation copies data from a remote database to your local database. A push operation copies data from your local database to a remote database. A sync operation copies data both directions. In each case, only the data missing from the destination is copied. The netsync protocol calculates the data to send "on the fly" by exchanging partial hash values of each database.
----------------- -----------------
( ) ( )
----------------- --- pull ---> -----------------
| | | |
| remote | <--- sync ---> | local |
| database | | database |
| | <--- push --- | |
\_________________/ \_________________/
In general, work flow with monotone involves 3 distinct stages:
The last stage of workflow is worth clarifying: monotone does not blindly apply all changes it receives from a remote database to your working copy. Doing so would be very dangerous, because remote databases are not always trustworthy systems. Rather, monotone evaluates the certificates it has received along with the changes, and decides which particular changes are safe and desirable to apply to your working copy.
You can always adjust the criteria monotone uses to judge the trustworthiness and desirability of changes in your database. But keep in mind that it always uses some criteria; receiving changes from a remote server is a different activity than applying changes to a working copy. Sometimes you may receive changes which monotone judges to be untrusted or bad; such changes may stay in your database but will not be applied to your working copy.
Remote databases, in other words, are just untrusted "buckets" of data, which you can trade with promiscuously. There is no trust implied in communication.
We previously described "old" and "new" versions of a file as "parents" and "children" in a directed graph. The relationship between a child and its parents is called the ancestry of the child.
Since monotone often stores related versions as deltas, it may
sometimes be possible to infer ancestry relationships by analyzing
the deltas. For technical reasons we will not dwell on here, monotone
does not do this. Instead, ancestry relationships are stored as
certs. The cert name ancestor is reserved by monotone for
expressing this relationship; the cert value of an ancestor
cert is the SHA1 of the parent version.
For example, suppose the version 994fe79ae852133ff9a93ba0146d520192977078 is logically a child of the version 53f48d87d6ffbbb5a980f1393159e83a4921841e, as shown:
+---------------------------------------------------+
| parent version |
| SHA1 = 53f48d87d6ffbbb5a980f1393159e83a4921841e |
+---------------------------------------------------+
|
|
\|/
V
+---------------------------------------------------+
| child version |
| SHA1 = 994fe79ae852133ff9a93ba0146d520192977078 |
+---------------------------------------------------+
Then the following cert would express the ancestry relationship:
+----------------------------------------------------------+
| version code |
| 994fe79ae852133ff9a93ba0146d520192977078 |
+------------+---------------------------------------------+
| cert name | cert value |
| "ancestor" | "53f48d87d6ffbbb5a980f1393159e83a4921841e" |
+------------+-------------+-------------------------------+
| signed by key | signature |
| "jrh@somewhere.com" | "ac9c489c8...5d6b1c26c" |
+--------------------------+-------------------------------+
So far we have been talking about versions of files as though each new version logically follows the version before it, in a simple sequence of versions.
+----------------+
| parent version |
+----------------+
|
|
+----------------+
| child version |
+----------------+
|
|
+---------------------+
| grandchild version |
+---------------------+
This is a rosy picture, but sometimes it does not work out this way. Sometimes when you make new versions of files, other people are simultaneously making new versions too, and their new versions might be derived from the same parent as yours. Without loss of generality, we will assume simultaneous edits only happen two-at-a-time; in fact many more edits may happen at once but our reasoning will be the same.
We call this situation of simultaneous edits a fork, and will refer to the two children of a fork as the left child and right child. In a large collection of files with many people editing versions, especially on many different computers spread all around the world, forks are a common occurrence.
+----------------+
| parent version |
+----------------+
| |
+--------+ +---------+
| |
+-------------+ +--------------+
| left child | | right child |
+-------------+ +--------------+
If we analyze the changes in each child version, we will often find that the changes made between the parent and the left child are unrelated to the changes made between the parent and the right child. When this happens, we can usually merge the fork, producing a common grandchild version which contains both sets of changes.
+----------------+
| parent version |
+----------------+
| |
+--------+ +---------+
| |
+-------------+ +--------------+
| left child | | right child |
+-------------+ +--------------+
| |
+--------+ +---------+
| |
+----------------+
| merged version |
+----------------+
Sometimes, people intentionally produce forks which are not supposed to be merged; perhaps they have agreed to work independently for a time, or wish to change their files in ways which are not logically compatible with each other. When someone produces a fork which is supposed to last for a while (or perhaps permanently) we say that the fork has produced a new branch. Branches tell monotone which versions you would like to merge, and which you would like to keep separate.
You can see all the available branches using monotone list branches.
Branches are indicated with certs. The cert name branch is
reserved for use by monotone, for the purpose of identifying the
members of a branch. A branch cert has a symbolic "branch
name" as its value. When we refer to "a branch", we mean all
versions with a common branch name in their branch certs.
For example, suppose you are working on a program called "wobbler".
You might develop many versions of wobbler and then decide to split
your versions into a "stable branch" and an "unstable branch", to
help organize your work. In this case, you might call the new branches
"wobbler-stable" and "wobbler-unstable". From then on, all
versions in the stable branch would get a cert with name branch
and value wobbler-stable; all versions in the unstable branch
would get a cert with name branch and value
wobbler-unstable. When a wobbler-stable version forks,
the children of the fork will be merged. When a
wobbler-unstable version forks, the children of the fork will
be merged. However, the wobbler-stable and
wobbler-unstable branches will not be merged together, despite
having a common ancestor.
+-------------------------+
| common ancestor version |
+-------------------------+
| |
+---------+ +---------+
| |
+----------------+ +------------------+
| stable version | | unstable version |
+----------------+ +------------------+
| | | |
+-------+ +-------+ +-------+ +-------+
| | | |
+-------------+ +--------------+ +---------------+ +----------------+
| left stable | | right stable | | left unstable | | right unstable |
| child | | child | | child | | child |
+-------------+ +--------------+ +---------------+ +----------------+
| | | |
+-------+ +--------+ +-------+ +--------+
| | | |
+----------------+ +------------------+
| merged stable | | merged unstable |
| version | | version |
+----------------+ +------------------+
\_____________ _______________/ \_______________ _________________/
\/ \/
stable branch unstable branch
For each branch, the set of versions with no children is called the heads of the branch. Monotone can automatically locate, and attempt to merge, the heads of a branch. If it fails to automatically merge the heads, it may ask you for assistance or else fail cleanly, leaving the branch alone.
For example, if a fork's left child has a child of its own (a "left grandchild"), monotone will merge the fork's right child with the left grandchild, since those versions are the heads of the branch. It will not merge the left child with the right child, because the left child is not a member of the heads.
+----------------+
| parent version |
+----------------+
| |
+--------+ |
| |
+-------------+ |
| left child | |
+-------------+ |
| |
*************|****************|************
* | | *
* +-----------------+ +-------------+ *
* | left grandchild | | right child | * the heads of the branch
* +-----------------+ +-------------+ * (before the merge)
* | | *
*************|****************|************
| |
+--------+ +---+
| |
+----------------+
| merged version |
+----------------+
When there is only one version in the heads of a branch, we say that the heads are merged, or more generally that the branch is merged, since the heads is the logical set of candidates for any merging activity. If there are two or more version in the heads of a branch, monotone will merge them two-at-a-time until there is only one.
Getting started using monotone is easy. This chapter is a tutorial, which illustrates the basic use of monotone by means of a small example software project: the canonical "hello world" program. We will assume the following files exist:
src/hello.c
include/hello.h
Makefile
Before operating monotone, two options should be explained.
--db option.
--branch option.
Monotone will cache the settings for these options in your working copy, so ordinarily once you have checked out a project, you will not need to specify them again. We will therefore only mention these arguments in the first example.
Many commands require you to supply 40-character SHA1 values as arguments, which we refer to as versions. These are tedious to type, so monotone permits you to supply "version selectors" rather than complete versions. Selectors are a more "human friendly" way of specifying versions by combining certificate values into unique identifiers. This "selector" mechanism can be used anywhere a version code would normally be used. For details on selector syntax, see Selectors.
We are now ready to explore some version control tasks.
The first step is to create a new database. Monotone requires this as
an explicit step to prevent spurious creation of databases when an
invalid --db option is given.
This is easily done using the special monotone db init command:
$ monotone db init --db=$HOME/test.db
Now we must generate an RSA key pair for ourselves. This step requires choosing a key identifier. Typical key identifiers are similar to email addresses, possibly modified with some prefix or suffix to distinguish multiple keys held by the same owner. We will use the key identifier "testkey@nodomain.org". When we ask for a key to be generated, monotone will ask us for a passphrase. This phrase is used to encrypt the key when storing it on disk, as a security measure.
$ monotone --db=$HOME/test.db genkey testkey@nodomain.org
enter passphrase for key ID [testkey@nodomain.org] : <enter passphrase>
monotone: generating key-pair 'testkey@nodomain.org'
monotone: storing key-pair 'testkey@nodomain.org' in database
We have now generated a key pair and placed it in the database
~/test.db. We can list the keys in this database, to ensure the
correct key was generated.
$ monotone --db=$HOME/test.db list keys
[public keys]
9e9e9ef1d515ad58bfaa5cf282b4a872d8fda00c testkey@nodomain.org
[private keys]
771ace046c27770a99e5fddfa99c9247260b5401 testkey@nodomain.org
The hexidecimal string printed out before each key name is a fingerprint of the key, and can be used to verify that the key you have stored under a given name is the one you intended to store. Monotone will never permit one database to store two keys with the same name or the same fingerprint.
This output shows one private and one public key stored under the name
testkey@nodomain.org, so it indicates that our keys have been
successfully generated and stored. On subsequent commands, we will
need to re-enter our passphrase to produce signatures on various tags.
Next we add all our files to our database. We can specify directories or filenames; monotone will recursively scan any directories we mention.
$ monotone --branch=testbranch --db=$HOME/test.db add Makefile include src
monotone: adding Makefile to working copy add set
monotone: adding include/hello.h to working copy add set
monotone: adding src/hello.c to working copy add set
This command has now added our files to the work list, as
"additions". The work list is a special file called work,
stored in a special control directory monotone just created, called
MT. There is only one control directory for the entire working
copy.
If we look at the work list, we see it contains instructions to add these files.
$ cat MT/work
add
Makefile
add
include/hello.h
add
src/hello.c
Now we are ready to commit our additions. We will provide a "log message" for our commit as an argument on the command line.
$ monotone commit 'initial addition of files'
monotone: committing 577c21252290d4b66240efb1f917a9f3c4136fe5
monotone: committing to branch testbranch
enter passphrase for key ID [testkey@nodomain.org] : <enter passphrase>
monotone: committed 577c21252290d4b66240efb1f917a9f3c4136fe5
Now we can check to see that monotone is tracking these files in the
working copy manifest. The working copy manifest is held in the file
MT/manifest.
$ cat MT/manifest
84e2c30a2571bd627918deee1e6613d34e64a29e Makefile
c61af2e67eb9b81e46357bb3c409a9a53a7cdfc6 include/hello.h
97dfc6fd4f486df95868d85b4b81197014ae2a84 src/hello.c
The column on the left contains cryptographic hashes of the files listed in the column on the right. Such a hash is called the "version code" of each file. The manifest is a plain text file, and can be manipulated with standard tools. If a file appears in a manifest, and you do not want to manage the file with monotone, you can remove that line of the manifest.
We can also check to see which certificates monotone generated when we
committed our work, using the list certs command. Note that we
only need to mention the first few digits of the manifest version we
wish to inspect, and monotone completes the version code for us:
$ monotone list certs manifest 577c
monotone: expanding partial id '577c'
monotone: expanded to '577c21252290d4b66240efb1f917a9f3c4136fe5'
-----------------------------------------------------------------
Key : testkey@nodomain.org
Sig : ok
Name : author
Value : jrh@nodomain.org
-----------------------------------------------------------------
Key : testkey@nodomain.org
Sig : ok
Name : branch
Value : testbranch
-----------------------------------------------------------------
Key : testkey@nodomain.org
Sig : ok
Name : date
Value : 2003-04-04T07:39:51
-----------------------------------------------------------------
Key : testkey@nodomain.org
Sig : ok
Name : changelog
Value : initial addition of files
The output of this command has a block for each cert found. Each block
has 4 significant pieces of information. The first indicates the
signer of the cert, in this case testkey@nodomain.org. The
second indicates whether this cert is "ok", meaning whether the
RSA signature provided is correct for the cert data. The third is
the cert name, and the fourth is the cert value. This list shows us
that monotone has confirmed that, according to
testkey@nodomain.org, the manifest
577c21252290d4b66240efb1f917a9f3c4136fe5 is a member of the
branch testbranch, written by jrh@nodomain.org, with
the given date and changelog.
It is important to keep in mind that manifests are not "in" or "out" of a branch in any global sense, nor are any of these cert values true or false in any global sense. Each cert indicates that some person would like to associate a manifest with some value; it is up to you to decide if you want to accept that association.
We can check status of our branch with the following command:
$ monotone heads
branch 'testbranch' is currently merged:
577c21252290d4b66240efb1f917a9f3c4136fe5 jrh@nodomain.org 2003-04-04T07:39:51
The output of this command tells us that there is only one current
"head" version in the branch testbranch, and it is the manifest
we just imported. A head version is one without any descendents. Since
we have not committed any changes to this manifest yet, it has no
descendents.
Finally, if we wanted to print out the manifest and inspect it, we could issue the following command:
$ monotone cat manifest 577c
monotone: expanding partial id '577c'
monotone: expanded to '577c21252290d4b66240efb1f917a9f3c4136fe5'
84e2c30a2571bd627918deee1e6613d34e64a29e Makefile
c61af2e67eb9b81e46357bb3c409a9a53a7cdfc6 include/hello.h
97dfc6fd4f486df95868d85b4b81197014ae2a84 src/hello.c
This command accesses our database directly, printing out the manifest identified as the head of our branch. This command not read the manifest from the working copy. We could safely erase our working copy, as the manifest (and its files) are stored in our database. We will not erase it, however, because there is more left to explore.
Initially, the contents of the file src/hello.c were as follows:
#include "hello.h"
void say_hello()
{
printf("hello, world\n");
}
int main()
{
say_hello();
}
Suppose now that we would like to make a change to this file, say to the message which is printed out. We can edit the file directly, and save the following new version:
#include "hello.h"
void say_hello()
{
printf("hello, smelly world\n");
}
int main()
{
say_hello();
}
If we would like to see a compact listing of the change we have made, relative to the version of the file recorded in our current manifest, we can use the following command:
$ monotone status
Old manifest: 577c21252290d4b66240efb1f917a9f3c4136fe5
New manifest: 86975f57ad571a61eb653371a2ffd12b96a301ed
Summary of changes:
patch src/hello.c
from 97dfc6fd4f486df95868d85b4b81197014ae2a84
to 8574638d5f34493e05e62d9e48b3c338169e7558
The output of this command tells us that our edits, so far, constitute
only a simple edit to a single file. This command accessed our working
copy alone: monotone read the MT/manifest file and compared it
to the SHA1 values of all the files found in the working copy,
then listed the differences it found. If we would like to see those
differences in closer detail, we can also print out a difference
listing in GNU "unified diff" format:
$ monotone diff
# Old manifest: 577c21252290d4b66240efb1f917a9f3c4136fe5
# New manifest: 86975f57ad571a61eb653371a2ffd12b96a301ed
# Summary of changes:
#
# patch src/hello.c
# from 97dfc6fd4f486df95868d85b4b81197014ae2a84
# to 8574638d5f34493e05e62d9e48b3c338169e7558
#
--- src/hello.c
+++ src/hello.c
@@ -1,8 +1,8 @@
#include "hello.h"
void say_hello()
{
- printf("hello, world\n");
+ printf("hello, smelly world\n");
}
int main()
This command accessed our database and our working copy: as in the
previous command, monotone compared the MT/manifest file to the
working copy. Once monotone found a difference, it consulted the
database consulted to construct the previous file version (97dfc6...),
and ran a line-oriented diff algorithm between the old version of the
file, from the database, and the new version of the file, in the
working copy.
Suppose now we would also like to move our say_hello
function to a secondary source file, and modify our Makefile to
build it separately. We can make these changes directly:
$ head -n 6 src/hello.c >src/say-hello.c
$ vi src/hello.c
<remove the function from this source file>
$ vi Makefile
<add a rule to build the new object file>
We must then add the new file to monotone's working record:
$ monotone add src/say-hello.c
This command did not modify the MT/manifest file, but rather
made a note in a persistent log file MT/work, stating that we
wish to add the file src/say-hello.c. The MT/work file
will be processed when we commit our work. Before we commit anything,
we can show the current status of our working copy once more, which
will include any changes between the MT/manifest file and the
files in our working copy, as well as any additions or deletions
written in the MT/work file:
$ monotone status
Old manifest: 86975f57ad571a61eb653371a2ffd12b96a301ed
New manifest: 0b34c61bd098a9c37cece9cbf7b765cc4d7b587e
Summary of changes:
add src/say-hello.c
as 84afbb102eca7e5f802ce381987ad7100ff7a1a9
patch Makefile
from 84e2c30a2571bd627918deee1e6613d34e64a29e
to b506d36ac5a6ea0fd588acacf1ebaa0317601abe
patch src/hello.c
from 97dfc6fd4f486df95868d85b4b81197014ae2a84
to eac552f8ee3d29198884368d592de67db5cb77e2
And, as before, we can view the full difference listing:
$ monotone diff
# Old manifest: 86975f57ad571a61eb653371a2ffd12b96a301ed
# New manifest: 0b34c61bd098a9c37cece9cbf7b765cc4d7b587e
# Summary of changes:
#
# add src/say-hello.c
# as 84afbb102eca7e5f802ce381987ad7100ff7a1a9
#
# patch Makefile
# from 84e2c30a2571bd627918deee1e6613d34e64a29e
# to b506d36ac5a6ea0fd588acacf1ebaa0317601abe
#
# patch src/hello.c
# from 97dfc6fd4f486df95868d85b4b81197014ae2a84
# to eac552f8ee3d29198884368d592de67db5cb77e2
#
--- Makefile
+++ Makefile
@@ -1,2 +1,6 @@
-hello: include/hello.h src/hello.c
- gcc -o hello -I include src/hello.c
+hello: include/hello.h src/say-hello.o src/hello.c
+ gcc -o hello say-hello.o src/hello.c
+
+say-hello.o: include/hello.h src/say-hello.c
+ gcc -c say-hello.o src/say-hello.c
+
--- src/hello.c
+++ src/hello.c
@@ -1,10 +1,5 @@
#include "hello.h"
-void say_hello()
-{
- printf("hello, world\n");
-}
-
int main()
{
say_hello();
Now that we have made changes, we can commit them to the database. Doing so will replace the "head" version of our branch with the new, changed version, and store the older version as a reverse delta from the new version.
$ monotone commit
This will first place us in a "log message editor" -- typically an editor
like vi -- with an explanation of the changes being committed and
the opportunity to enter a log message.
example commit log
MT: ----------------------------------------------------------------------
MT: Enter Log. Lines beginning with `MT:' are removed automatically
MT:
MT: Old manifest: 86975f57ad571a61eb653371a2ffd12b96a301ed
MT: New manifest: 0b34c61bd098a9c37cece9cbf7b765cc4d7b587e
MT: Summary of changes:
MT:
MT: add src/say-hello.c
MT: as 84afbb102eca7e5f802ce381987ad7100ff7a1a9
MT:
MT: patch Makefile
MT: from 84e2c30a2571bd627918deee1e6613d34e64a29e
MT: to b506d36ac5a6ea0fd588acacf1ebaa0317601abe
MT:
MT: patch src/hello.c
MT: from 97dfc6fd4f486df95868d85b4b81197014ae2a84
MT: to eac552f8ee3d29198884368d592de67db5cb77e2
MT:
MT: ----------------------------------------------------------------------
MT:
We have entered a single line above the explanatory message, saying "example commit log". We then save the file and quit the editor. Monotone will delete all the lines beginning with "MT:" and leave only our message. Returning to the shell, we are prompted for our passphrase.
monotone: committing 0b34c61bd098a9c37cece9cbf7b765cc4d7b587e
monotone: committing to branch testbranch
enter passphrase for key ID [testkey@nodomain.org] : <enter passphrase>
monotone: committed 0b34c61bd098a9c37cece9cbf7b765cc4d7b587e
A commit typically involves making several certs, as well as storing
new versions of files and the manifest in the database. We can show the
certs produced, using the list certs command.
$ monotone list certs manifest 0b34
monotone: expanding partial id '0b34'
monotone: expanded to '0b34c61bd098a9c37cece9cbf7b765cc4d7b587e'
-----------------------------------------------------------------
Key : testkey@nodomain.org
Sig : ok
Name : ancestor
Value : 577c21252290d4b66240efb1f917a9f3c4136fe5
-----------------------------------------------------------------
Key : testkey@nodomain.org
Sig : ok
Name : author
Value : jrh@nodomain.org
-----------------------------------------------------------------
Key : testkey@nodomain.org
Sig : ok
Name : branch
Value : testbranch
-----------------------------------------------------------------
Key : testkey@nodomain.org
Sig : ok
Name : date
Value : 2003-04-04T07:44:54
-----------------------------------------------------------------
Key : testkey@nodomain.org
Sig : ok
Name : changelog
Value : example commit log
In this commit, monotone produced a branch cert which relates
the new version to the branch testbranch, an ancestor
cert which relates old version to the new one, and several informative
meta-data certs such as author, date, and
changelog.
If we would like to examine the ancestry graph at any time, we can
produce a representation of it suitable for viewing with the popular
xvcg graph visualization tool, with the following command:
$ monotone agraph
graph:
{
node: { title : "0b34c61bd098a9c37cece9cbf7b765cc4d7b587e"}
node: { title : "577c21252290d4b66240efb1f917a9f3c4136fe5"}
edge: { sourcename : "577c21252290d4b66240efb1f917a9f3c4136fe5"
targetname : "0b34c61bd098a9c37cece9cbf7b765cc4d7b587e" }
}
Now that we have committed one version, the MT/manifest file
has been updated to reflect the current contents of the files we are
working on:
$ cat MT/manifest
b506d36ac5a6ea0fd588acacf1ebaa0317601abe Makefile
c61af2e67eb9b81e46357bb3c409a9a53a7cdfc6 include/hello.h
eac552f8ee3d29198884368d592de67db5cb77e2 src/hello.c
84afbb102eca7e5f802ce381987ad7100ff7a1a9 src/say-hello.c
This version of the tree is safely stored in the database, however, so we can feel free to "checkout" previous versions from the database, without fear of losing our work:
$ monotone checkout 577c .
monotone: expanding partial id '577c'
monotone: expanded to '577c21252290d4b66240efb1f917a9f3c4136fe5'
$ cat MT/manifest
84e2c30a2571bd627918deee1e6613d34e64a29e Makefile
c61af2e67eb9b81e46357bb3c409a9a53a7cdfc6 include/hello.h
97dfc6fd4f486df95868d85b4b81197014ae2a84 src/hello.c
$ cat src/hello.c
#include "hello.h"
void say_hello()
{
printf("hello, world\n");
}
int main()
{
say_hello();
}
Note that monotone has reverted the tree to its original contents. It
has not, however, deleted the file src/say-hello.c which
we added to our second version; it has merely removed it from the
manifest.
Suppose that we (or some other user) happen to produce a different modified version of our example project, derived from the same initial version. We call this situation a fork. Monotone does not prevent you from making forks, even when it can detect them.
$ cat >>src/hello.c
void say_goodbye()
{
printf("goodbye\n");
}
^D
We can view our differences, as before:
$ monotone diff
# Summary of changes:
#
# patch src/hello.c
# from 97dfc6fd4f486df95868d85b4b81197014ae2a84
# to bdcf3d335db02a8a944b8945357c0e080b40f1b4
#
--- src/hello.c
+++ src/hello.c
@@ -9,3 +9,8 @@
{
say_hello();
}
+
+void say_goodbye()
+{
+ printf("goodbye\n");
+}
And commit our change, this time providing the changelog message on
the command line as a third argument to the commit command.
$ monotone commit "a fork"
monotone: committing 57d3afae9e806b930130a674befec2a52bd85cd1
monotone: committing to branch testbranch
enter passphrase for key ID [testkey@nodomain.org] : <enter passphrase>
monotone: committed 57d3afae9e806b930130a674befec2a52bd85cd1
Monotone does not prohibit us from making a fork. However, if we check the heads of our branch, we will now find that it is unmerged. In other words, it has an unresolved fork:
$ monotone heads
branch 'testbranch' is currently unmerged:
0b34c61bd098a9c37cece9cbf7b765cc4d7b587e jrh@nodomain.org 2003-04-04T07:44:54
57d3afae9e806b930130a674befec2a52bd85cd1 jrh@nodomain.org 2003-04-04T07:46:02
This state does not, in itself, constitute a threat to the branch; we can still commit new changes, check out versions, show differences, etc. without resolving this fork. However, we will probably want to resolve it eventually, if for no other reason than to incorporate the positive features of both edges into a common head version.
Merging a fork can be quite simple:
$ monotone merge
enter passphrase for key ID [testkey@nodomain.org] : <enter passphrase>
[source] 0b34c61bd098a9c37cece9cbf7b765cc4d7b587e
[source] 57d3afae9e806b930130a674befec2a52bd85cd1
[merged] ecec7c49374ab9a7706a500339a2a0d6157372f1
This case is quite trivial and was handled automatically by the built
in 3-way merge algorithm. In more complicated merges, a hook is called
and the user may provide an external merge tool. By default, this hook
runs an emacs process with a batch command invoking ediff mode
with the conflicting files, to interactively merge the difference, but
this can be overridden if you prefer some other tool. If we
investigate the resulting manifest, we see the following.
$ monotone list certs manifest ecec
monotone: expanding partial id 'ecec'
monotone: expanded to 'ecec7c49374ab9a7706a500339a2a0d6157372f1'
-----------------------------------------------------------------
Key : testkey@nodomain.org
Sig : ok
Name : ancestor
Value : 57d3afae9e806b930130a674befec2a52bd85cd1
-----------------------------------------------------------------
Key : testkey@nodomain.org
Sig : ok
Name : ancestor
Value : 0b34c61bd098a9c37cece9cbf7b765cc4d7b587e
-----------------------------------------------------------------
Key : testkey@nodomain.org
Sig : ok
Name : author
Value : jrh@nodomain.org
-----------------------------------------------------------------
Key : testkey@nodomain.org
Sig : ok
Name : branch
Value : testbranch
-----------------------------------------------------------------
Key : testkey@nodomain.org
Sig : ok
Name : date
Value : 2003-04-04T08:00:09
-----------------------------------------------------------------
Key : testkey@nodomain.org
Sig : ok
Name : changelog
Value : merge of 0b34c61bd098a9c37cece9cbf7b765cc4d7b587e
and 57d3afae9e806b930130a674befec2a52bd85cd1
In the process of merging, monotone generated a merged copy of our forked file, a merged copy of our forked manifest, and a number of certs relating the fork and merge:
ancestor cert relating the left ancestor version of
src/hello.c to the merged version
ancestor cert relating the right ancestor version of
src/hello.c to the merged version
ancestor cert relating the left ancestor manifest to the merged version
ancestor cert relating the right ancestor manifest to the merged version
branch cert associating the merged manifest with testbranch
author, date, and changelog certs for the merged manifest
Each cert is recorded separately. An interesting affect of this independent record is that once you have performed a merge between two file versions, the merge will automatically be reused in any future tree merges which involve the same two files, even if they appear in different branches.
We can now check the status of our branch's heads, to ensure that only one merged copy remains.
$ monotone heads
branch 'testbranch' is currently merged:
ecec7c49374ab9a7706a500339a2a0d6157372f1 jrh@nodomain.org 2003-04-04T08:00:09
Now that we've merged our fork, we notice that the current version of our working copy is no longer the "newest", in the sense that it has a live descendent version in the database (the merged version).
Updating our working copy is quite straightforward in this simple case.
$ monotone update
monotone: updated to base version ecec7c49374ab9a7706a500339a2a0d6157372f1
If we check our version of src/hello.c, we now see we have a
merge of the two previous files.
$ cat src/hello.c
#include "hello.h"
int main()
{
say_hello();
}
void say_goodbye()
{
printf("goodbye\n");
}
An update actually consists of 3 distinct phases.
merge command, if the built in 3-way merge fails then monotone
will run a user-provided hook. If this merge fails, the update stops.
MT/manifest file, and the merged files are written to your
working copy. This happens after the update is known to
succeed, so it is an all-or-nothing phase.
The update operation essentially "re-parents" your working copy, changing it from being a child of the version it was initially checked out as, to being a child of the new base version. Your changes relative to your previous parent are moved forward during the merge phase.
One interesting feature of the update operation is that you can override or extend the logic monotone uses to filter update candidates, with Lua hook functions. See Quality Assurance, for details.
After doing some work, typically we want to send these changes to other users. Monotone contains a built-in protocol which can directly synchronize your database with another over TCP/IP. Supposing we set up a monotone "server", then synchronizing with it will "send" our work to the server, and "receive" any work the server has which we have not yet seen.
Setting up a remote server is the subject of the next chapter, which includes some examples illustrating the use of a client and server. This concludes the initial tutorial.
Note: Monotone's networking system changed between release 0.9 and 0.10. The old system is deprecated and will be removed. Documentation for it has already been removed. Please familiarize yourself with the new system.
Monotone's networking protocol is called netsync. It is a pipelined, bidirectional protocol which synchronizes monotone databases over a TCP connection. Monotone can behave as either a client or server for netsync. No additional server software is required.
To illustrate netsync's use, we consider the branch containing
monotone development itself: net.venge.monotone. Suppose we
wish to serve this branch via the netsync protocol, on a host called
off.net. We begin by initializing a database, say
server.db, generating a public and private keypair, and storing
the resulting public key packet in a text file:
$ monotone --db=server.db db init
$ monotone --db=server.db genkey monotone@off.net
enter passphrase for key ID [monotone@off.net] : <enter passphrase>
monotone: generating key-pair 'monotone@off.net'
monotone: storing key-pair 'monotone@off.net' in database
$ monotone --db=server.db pubkey montone@off.net >server-key.txt
Now that the database is initialized, we need to write a small lua hook file specifying the anonymous read permission.
$ cat >permissions.lua
function get_netsync_anonymous_read_permitted (collection)
return true
end
^D
We can then start monotone running as a server using the serve
command, providing the hook file as an additional command line
argument:
$ monotone --db=server.db --rcfile=permissions.lua \
serve off.net net.venge.monotone
Monotone will continue to serve incoming netsync requests until it is
terminated, for example with a CTRL-C interrupt from the
operator's keyboard. If you want to run a monotone server permanently,
it is a good idea to run it under a supervision program such as
daemontools.
In order to connect to a monotone server, a client needs to know the server's public key. This involves reading the key packet generated in the previous step.
$ monotone --db=client.db read <server-key.txt
monotone: read 1 packet
Netsync clients can then pull, push, or sync with a given server. Sync simply means "push and pull at the same time", but it is performed in a single connection. The server may accept or refuse connections depending on configuration.
So, returning to our example, suppose we wish to retrieve data as a
client from the server we started on off.net. Then we can use the
pull command, providing an empty key to authenticate
anonymously with the server:
$ monotone --key="" --db=client.db pull off.net net.venge.monotone
monotone: warning: doing anonymous pull
monotone: rebuilding merkle trees for collection net.venge.monotone
monotone: [keys: 1] [mcerts: 1]
monotone: connecting to off.net
monotone: [bytes in: 2387498] [bytes out: 498739]
monotone: read from fd 5 (peer off.net) closed OK after goodbye
Here monotone has warned us that it is doing an anonymous pull, described the indexing process on the local database, connected, exchanged data, and disconnected from the remote database. Most netsync operations look like this from the client side.
Later, we may want to start sending data to our server. To do this, we need to configure the server to accept connections from us. We do this by first exporting the client key into a text file:
$ monotone --db=client.db pubkey client@somewhere.net >client-key.txt
Then loading the client key into the server database:
$ monotone --db=server.db read <client-key.txt
monotone: read 1 packet
And and writing a small lua hook which defines the list of keys which can read and write to the server:
$ cat >>permissions.lua
function get_netsync_read_permitted (collection, identity)
if (identity == "client@somewhere.net") then return true end
return false
end
function get_netsync_write_permitted (collection, identity)
if (identity == "client@somewhere.net") then return true end
return false
end
^D
We can then restart the server with this new set of permissions:
$ monotone --db=server.db --rcfile=permissions.lua \
serve off.net net.venge.monotone
And then the client can push changes to the server:
$ monotone --db=client.db push off.net net.venge.monotone
monotone: rebuilding merkle trees for collection net.venge.monotone
monotone: [keys: 5] [mcerts: 704]
monotone: connecting to off.net
monotone: [bytes in: 221] [bytes out: 782]
monotone: read from fd 5 (peer off.net) closed OK after goodbye
We can combine a push and pull operation against a
single server with a sync. It requires nothing new since we've
configured the server to permit reads and writes from the client.
It is important to point out that there is no coupling between our client and any particular server. If the server on off.net vanished or was taken offline, we could sync with any other database, without having to re-exchange any history or adjust our understanding of branches. Indeed, if there are no firewalls in the way, we can skip the use of servers altogether and synchronize between clients directly, in a "peer to peer" fashion.
This chapter covers special topics which are of interest to some users of monotone, though possibly not all. We assume that you have read through the taxonomy and tutorial, and possibly spent some time playing with the program to familiarize yourself with its operation.
Version codes can be specified on the monotone command line, precisely, by entering the entire 40-character hexidecimal SHA1 code. This can be cumbersome, so monotone also a more general syntax called "selectors" which is less precise but more "human friendly". Any command which expects a precise version identifier can also accept a selector in its place; in fact a version identifier is just a special type of selector which is very precise.
Some selector examples are helpful in clarifying the idea:
a432
a432
graydon@pobox.com/2004-04
graydon@pobox.com in April 2004.
'jrh@nodomain.org/2 weeks ago'
jrh@nodomain.org 2 weeks ago.
graydon/net.venge.monotone.win32/yesterday
net.venge.monotone.win32 branch, written by
graydon, yesterday.
A moment's examination reveals that these specifications are "fuzzy" and indeed may return multiple values, or may be ambiguous. When ambiguity arises, monotone will inform you that more detail is required, and list various possibilities. The precise specification of selectors follows.
A selector is a combination of a selector type, which is a single
ASCII character, followed by a : character and a selector
value. The value is matched against identifiers or certs, depending on
its type, in an attempt to match a single version. Selectors are
matched as prefixes. The current set of selection types are:
a. For example, a:graydon matches
author certs where the cert value begins with graydon.
b. For example, b:net.venge matches
branch certs where the cert value begins with net.venge.
d. For example, d:2004-04 matches
date certs where the cert value begins with 2004-04.
i. For example, i:0f3a matches
version identifiers which begin with 0f3a.
t. For example, t:monotone-0.11
matches tag certs where the cert value begins with
monotone-0.11.
Further selector types may be added in the future.
Selectors may be combined with the / character. The combination
acts as database intersection (or logical and). For example,
the selector a:graydon/d:2004-04 can be used to select a
version which has an author cert beginning with graydon
as well as a date cert beginning with 2004-04.
Before selectors are passed to the database, they are expanded using a
lua hook: expand_selector. The default definition of this hook
attempts to guess a number of common forms for selection, allowing you
to omit selector types in many cases. For example, the hook guesses
that the typeless selector jrh@nodomain.org is an author
selector, due to its syntactic form, so modifies it to read
a:jrh@nodomain.org. This hook will generally assign a selector
type to values which "look like" partial hex strings, email
addresses, branch names, or date specifications. For the complete
source code of the hook, see Hook Reference.
If, after expansion, a selector still has no type, it is matched as a
special "unknown" selector type, which will match either a tag, an
author, or a branch. This costs slightly more database access, but
often permits simple selection using an author's login name and a
date. For example, the selector
graydon/net.venge.monotone.win32/yesterday would pass through
the selector graydon as an unknown selector; so long as there
are no branches or tags beginning with the string graydon this
is just as effective as specifying a:graydon.
Some proponents of a competing, proprietary version control system have suggested, in a usenix paper, that the use of a cryptographic hash function such as SHA1 as an identifier for a version is unacceptably unsafe. This section addresses the argument presented in that paper and describes monotone's additional precautions.
To summarize our position:
The paper displays a fundamental lack of understanding about what a cryptographic hash function is, and how it differs from a normal hash function. Furthermore it confuses accidental collision with attack scenarios, and mixes up its analysis of the risk involved in each. We will try to untangle these issues here.
A cryptographic hash function such as SHA1 is more than just a uniform spread of inputs to an output range. Rather, it must be designed to withstand attempts at:
Collision is the problem the paper is concerned with. Formally, an n-bit cryptographic hash should cost 2^n work units to collide against a given value, and sqrt(2^n) tries to find a random pair of colliding values. This latter probability is sometimes called the hash's "birthday paradox probability".
One way of measuring these bounds is by measuring how single-bit changes in the input affect bits in the hash output. The SHA1 hash has a strong avalanche property, which means that flipping any one bit in the input will cause on average half the 160 bits in the output code to change. The fanciful VAL1 hash presented in the paper does not have such a property -- flipping its first bit when all the rest are zero causes no change to any of the 160 output bits -- and is completely unsuited for use as a cryptographic hash, regardless of the general shape of its probability distribution.
The paper also suggests that birthday paradox probability cannot be used to measure the chance of accidental SHA1 collision on "real inputs", because birthday paradox probability assumes a uniformly random sample and "real inputs" are not uniformly random. The paper is wrong: the inputs to SHA1 are not what is being measured (and in any case can be arbitrarily long); the collision probability being measured is of output space. On output space, the hash is designed to produce uniformly random spread, even given nearly identical inputs. In other words, it is a primary design criterion of such a hash that a birthday paradox probability is a valid approximation of its collision probability.
The paper's characterization of risk when hashing "non-random inputs" is similarly deceptive. It presents a fanciful case of a program which is storing every possible 2kb block in a file system addressed by SHA1 (the program is trying to find a SHA1 collision). While this scenario will very likely encounter a collision somewhere in the course of storing all such blocks, the paper neglects to mention that we only expect it to collide after storing about 2^80 of the 2^16384 possible such blocks (not to mention the requirements for compute time to search, or disk space to store 2^80 2kb blocks).
Noting that monotone can only store 2^41 bytes in a database, and thus probably some lower number (say 2^32 or so) active rows, we consider such birthday paradox probability well out of practical sight. Perhaps it will be a serious concern when multi-yottabyte hard disks are common.
Setting aside accidental collisions, then, the paper's underlying theme of vulnerability rests on the assertion that someone will break SHA1. Breaking a cryptographic hash usually means finding a way to collide it trivially. While we note that SHA1 has in fact resisted attempts at breaking for 8 years already, we cannot say that it will last forever. Someone might break it. We can say, however, that finding a way to trivially collide it only changes the resistance to active attack, rather than the behavior of the hash on benign inputs.
Therefore the vulnerability is not that the hash might suddenly cease to address benign blocks well, but merely that additional security precautions might become a requirement to ensure that blocks are benign, rather than malicious. The paper fails to make this distinction, suggesting that a hash becomes "unusable" when it is broken. This is plainly not true, as a number of systems continue to get useful low collision hashing behavior -- just not good security behavior -- out of "broken" cryptographic hashes such as MD4.
Perhaps our arguments above are unconvincing, or perhaps you are the sort of person who thinks that practice never lines up with theory. Fair enough. Below we present practical procedures you can follow to compensate for the supposed threats presented in the paper.
To minimize the chance of accidental collision further, monotone has a
special type of certificate called a vcheck, which you can
issue at any time and in any quantity you like. A vcheck
certificate contains a randomly keyed message authentication
code (MAC). This effectively means that a vcheck
cert identifies your tree again, with a "different" variant of
SHA1.
If you have the SHA1 of your manifest and a vcheck cert,
the probability of a collision involving that manifest drops from
2^-160 to 2^-320. If you have two vcheck certs,
it drops to 2^-480, and so on. You can issue as many
vcheck certs as you like until your personal level of paranoia
is met. Each vcheck cert has its own random seed, usually 128
bits, so after you've issued about 2^64 vcheck certs on
the same manifest you have a decent chance of repeating one, but that
is harmless.
Adding the occasional vcheck cert to your tree is a good idea
if you are worried about the "deterministic" nature of accidental
collisions in SHA1, outlined in the paper. Since the MAC is
randomized, unintentional (non-attack) collisions in vcheck
certs are reliably nondeterministic. Furthermore, they minimize the
chance that a malicious collision has gone undetected, by requiring
the collision to pass an additional (and equally hard) check. A large
warning is printed if such a check fails.
A successful collision attack on SHA1, as mentioned, does not disrupt the probability features of SHA1 on benign blocks. So if, at any time, you believe SHA1 is "broken", it does not mean that you cannot use it for your work with monotone. It means, rather, that you cannot base your trust on SHA1 values anymore. You must trust who you communicate with.
The way around this is reasonably simple: if you do not trust SHA1 to prevent malicious blocks from slipping into your communications, you can always augment it by enclosing your communications in more security, such as tunnels or additional signatures on your email posts. If you choose to do this, you will still have the benefit of self-identifying blocks, you will simply cease to trust such blocks unless they come with additional authentication information.
If in the future SHA1 (or, indeed, RSA) becomes accepted as broken we will naturally upgrade monotone to a newer hash or public key scheme, and provide migration commands to recalculate existing databases based on the new algorithm.
Similarly, if you do not trust our vigilance in keeping up to date
with cryptography literature, you can modify monotone to use any
stronger hash you like, at the cost of isolating your own
communications to a group using the modified version. Monotone is free
software, and runs atop crypto++, so it is both legal and
relatively simple to change it to use some other algorithm.
Monotone was constructed to serve both as a version control tool and as a quality assurance tool. The quality assurance features permit users to ignore, or "filter out", versions which do not meet their criteria for quality. This section describes the way monotone represents and reasons about quality information.
Monotone often views the collection of manifest ancestor certs
as a directed graph, in which manifest versions are the nodes and the
ancestor certificates are the edges. We call this graph the
ancestry graph. The ancestry graph has a number of important
subgraphs, many of which overlap. For example, each branch is a
subgraph of the ancestry graph, containing only the nodes carrying a
particular branch cert.
Many of monotone's operations involve searching the ancestry graph for
the ancestors or descendants of a particular manifest, or extracting
the "heads" of a subgraph, which is the subgraph's set of nodes with
no descendants. For example, when you run the update command,
monotone searches the subgraph consisting of descendants of the base
version of the current working copy, trying to locate a unique head to
update the base version to.
Monotone's quality assurance mechanisms are mostly based on restricting the subgraph each command operates on. There are three methods used to restrict the subgraph:
ancestor certificates, you
can require that specific code reviewers have approved of each edge in
the graph.
ancestor certificates,
those which coincide with trusted disapproval certificates.
testresult certificates, you
can require that the endpoints of an update operation have a
certificate asserting that the version in question passed a certain
test, or testsuite.
The evaluation of trust is done on a cert-by-cert basis by calling a
pair of lua hooks: get_manifest_cert_trust and
get_file_cert_trust. These hooks are only called when a
cert has at least one good signature from a known key, and are passed
all the keys which have signed the cert, as well as the cert's
id, name and value. The hook can then evaluate the set of signers, as
a group, and decide whether to grant or deny trust to the assertion made
by the cert.
The evaluation of testresults is controlled by the
accept_testresult_change hook. This hook is called when
selecting update candidates, and is passed a pair of tables describing
the testresult certs present on the source and proposed
destination of an update. Only if the change in test results are
deemed "acceptable" does monotone actually select an update target
to merge into your working copy.
For details on these hooks, see the Hook Reference.
Monotone initially dealt with only ASCII characters, in file path names, certificate names, key names, and packets. Some conservative extensions are provided to permit internationalized use. These extensions can be summarized as follows:
MT/manifest file in the working copy; the MT/options and
MT/work files; and the .mt-attrs file. Converting these
files to any other character set will cause monotone to break; do not
do so.
The remainder of this section is a precise specification of monotone's internationalization behavior.
get_charset_conv which takes a filename
and returns a table of two strings: the first represents the
"internal" (database) charset, the second represents the "external"
(file system) charset.
0x0D, 0x0A, or the pair 0x0D 0x0A) from one
convention to another. Per-file line ending conversion is specified by
a Lua hook get_linesep_conv which takes a filename and returns
a table of two strings: the first represents the "internal" (database)
line ending convention, the second represents the "external"
(file system) line ending convention. each string should be one of the
three strings "CR", "LF", or "CRLF".
Note that Line ending conversion is always performed on the internal
character set, when both character set and line ending conversion are
enabled; this behavior is meant to encourage the use of the monotone's
"normal form" (UTF-8, '\n') as an internal form for your source
files, when working with multiple external forms. Also note that line
ending conversion only works on character encodings with the specific
code bytes described above, such as ASCII, ISO-8859x, and UTF-8.
get_system_linesep hook. No hooks exist for adjusting the
system character set, since the system character set must be known
during command-line argument processing, before any Lua hooks are
loaded.
Monotone's normal form is the UTF-8 character set and the 0x0A
(LF) line ending form. This form is used in any files monotone needs
to read, write, and interpret itself, such as: MT/manifest,
MT/work, MT/options, .mt-attrs
0x2D,
0x30..0x39, 0x41..0x5A, and 0x61..0x7A.
{ACE-prefix}{LDH-sanitized(punycode(nameprep(UTF-8-string)))}
It is important to understand that IDNA encoding does not preserve the input string: it both prohibits a wide variety of possible strings and normalizes non-equal strings to supposedly "equivalent" forms.
By default, monotone does not decode IDNA when printing to the
console (IDNA names are ASCII, which is a subset of UTF-8, so this
normal form conversion can still apply, albeit oddly). this behavior
is to protect users against security problems associated with
malicious use of "similar-looking" characters. If the hook
display_decoded_idna returns true, IDNA names are decoded for
display.
0x5C '\' path separator to 0x2F '/'. This extra
processing is performed by boost::filesystem.
0x2F (ASCII / ), and
without a leading or trailing 0x2F.
0x2F and any ASCII "control codes"
(0x00..0x1F and 0x7F).
sha1sum will produce
different results than those entries shown in the MT/manifest
file.
UI messages are displayed via calls to gettext().
Host names are read on the command-line and subject to normal form
conversion. Host names are then split at 0x2E (ASCII '.'), each
component is subject to IDNA encoding, and the components are
rejoined.
After processing, host names are stored internally as ASCII. The
invariant is that a host name inside monotone contains only sequences
of LDH separated by 0x2E.
Read on the command line and subject to normal form conversion and IDNA encoding as a single component. The invariant is that a cert name inside monotone is a single LDH ASCII string.
Cert values may be either text or binary, depending on the return
value of the hook cert_is_binary. If binary, the cert value is
never printed to the screen (the literal string "<binary>" is
displayed, instead), and is never subjected to line ending or
character conversion. If text, the cert value is subject to normal
form conversion, as well as having all UTF-8 codes corresponding to
ASCII control codes (0x0..0x1F and 0x7F) prohibited in
the normal form, except 0x0A (ASCII LF).
Read on the command line and subject to normal form conversion and
IDNA encoding as an email address (split and joined at '.' and '@'
characters). The invariant is that a key name inside monotone contains
only LDH, 0x2E (ASCII '.') and 0x40 (ASCII '@')
characters.
Packets are 7-bit ASCII. The characters permitted in packets are the union of these character sets:
MT/work and .mt-attrsIncompatible change: these formats were changed, incompatibly, from earlier versions of monotone which used 0x20 (ASCII SP) as a delimiter. The files now use 0x0A (ASCII LF) as a delimiter, to permit 0x20 in filenames.
A monotone working copy consists of control files and non-control files. Each type of file can be versioned or non-versioned. These classifications lead to four groups of files:
Control files contain special content formatted for use by monotone. Versioned files are recorded in a monotone database and have their state tracked as they are modified.
If a control file is versioned, it is considered part of the state of the working copy, and will be recorded as a manifest entry. If a control file is not versioned, it is used to manage the state of the working copy, but it not considered an intrinsic part of it.
Most files you manage with monotone will be versioned non-control files. For example, if you keep source code or documents in a monotone database, they are versioned non-control files. Non-versioned, non-control files in your working copy are generally temporary or junk files, such as backups made by editors or object files made by compilers. Such files are ignored by monotone.
Control files are identified by their names. Non-control files can have any name except the names reserved for control files. The names of control files follow a regular pattern:
.mt-
MT/
The following control files are currently used. More control files may be added in the future, but they will follow the patterns given above.
.mt-attrs
MT/manifest
MT/options
--database or
--branch, such that you do not need to enter them repeatedly
after checking out a particular working copy.
MT/work
Every certificate has a name. Some names have meaning which is built
in to monotone, others may be used for customization by a particular
user, site, or community. If you wish to define custom certificates,
you should prefix such certificate names with x-. For example,
if you want to make a certificate describing the existence of security
vulnerabilities in a version, you might wish to create a certificate
called x-vulnerability. Monotone reserves all names which do
not begin with x- for possible internal use. If an x-
certificate becomes widely used, montotone will likey adopt it as a
reserved cert name and standardize its semantics.
Most reserved certificate names have no meaning yet; some do. Usually monotone is also responsible for generating these certificates, so you should generally have no cause to make them yourself. They are described here to help you understand monotone's operation.
The pre-defined, reserved certificate names are:
ancestor
author
log command.
changelog
log command.
comment
comment can be applied to files as
well, and will be shown by the log command.
date
log command, and may be used as an
additional heuristic or selection criterion in other commands in the future.
rename
commit, merge, and
propagate commands. The value is used as a hint to the patch
set analysis algorithm, when comparing manifests. If rename
certs are damaged or missing, the analysis of rename events can
potentially degrade, and monotone might consider a renamed file to be
two different files (one deleted and one added). Otherwise the absence
of these certificates is harmless.
tag
checkout.
disapproval
disapprove command,
and is used to facilitate code review. This cert controls the merging
and updating algorithms, along with the ancestor cert.
testresult
0
or 1. It is generated by the testresult command and
represents the results of running a particular test on the underlying
manifest. Typically you will make a separate signing key for each test
you intend to run on tree versions. This cert controls the
update algorithm.
vcheck
Some names in monotone are private to your work, such as filenames. Other names are potentially visible outside your project, such as RSA key identifiers or branch names. It is possible that if you choose such names carelessly, you will choose a name which someone else in the world is using, and subsequently you may cause confusion when your work and their is received simultaneously by some third party.
We therefore recommend two naming conventions:
graydon@pobox.com.
net.venge.monotone branch, because the
author owns the DNS domain venge.net.
Monotone contains a mechanism for storing persistent file attributes. These differ from file certificates in an important way: attributes are associated with a path name in your working copy, rather than a particular version of a file. Otherwise they are similar: a file attribute associates a simple name/value pair with a file in your working copy.
The attribute mechanism is motivated by the fact that some people like
to store executable programs in version control systems, and would like
the programs to remain executable when they check out a working copy.
For example, the configure shell script commonly shipped with
many programs should be executable.
Similarly, some people would like to store devices, symbolic links, read-only files, and all manner of extra attributes of a file, not directly related to a file's data content.
Rather than try to extend the manifest file to accommodate attributes,
monotone requires that you place your attributes in a specially named
file in the root of your working copy. The file is called
.mt-attrs, and it has a simple 3-column format, for example:
execute true src/configure
execute true autogen.sh
Each line of an .mt-attrs file assigns an attribute to a file
in your working copy. The first column of each line contains the name
of the attribute, the second column is the value of the attribute, and
the third column is the file name within the working copy to which the
attribute will be applied.
Every time your working copy is written to, monotone will look for an
.mt-attrs file, and if it finds one, run the corresponding
hooks registered for each attribute found in the file. This way, you
can extend the vocabulary of attributes understood by monotone simply
by writing new hooks.
For more information on defining hooks for new attributes, see the
attr_functions entry in Hook Reference.
Aside from its special interpretation, the .mt-attrs file is a
normal text file. If you want other people to see your attributes, you
should add and commit the .mt-attrs file in your
working copy. If you make changes to it which conflict with changes
other people make, you will have to resolve those conflicts, as plain
text, just as with any other text file in your working copy.
While the state of your database is logically captured in terms of a packet stream, it is sometimes necessary or desirable (especially while monotone is still in active development) to modify the SQL table layout or storage parameters of your version database, or to make backup copies of your database in plain text. These issues are not properly addressed by generating packet streams: instead, you must use migration or dumping commands.
The monotone db migrate command is used to alter the SQL
schema of a database. The schema of a monotone database is identified
by a special hash of its generating SQL, which is stored in the
database's auxiliary tables. Each version of monotone knows which
schema version it is able to work with, and it will refuse to operate
on databases with different schemas. When you run the
migrate command, monotone looks in an internal list of SQL
logic which can be used to perform in-place upgrades. It applies
entries from this list, in order, attempting to change the database it
has into the database it wants. Each step of this
migration is checked to ensure no errors occurred and the resulting
schema hashes to the intended value. The migration is attempted inside
a transaction, so if it fails -- for example if the result of
migration hashes to an unexpected value -- the migration is aborted.
If more drastic changes to the underlying database are made, such as
changing the page size of sqlite, or if you simply want to keep a
plain text version of your database on hand, the monotone db
dump command can produce a plain ASCII SQL statement which generates
the state of your database. This dump can later be reloaded using the
monotone db load command.
Note that when reloading a dumped database, the schema of the dumped
database is included in the dump, so you should not try to
init your database before a load.
Monotone is capable of reading CVS files directly and importing them into a database. This feature is still somewhat immature, but moderately large "real world" CVS trees on the order of 1GB have successfully been imported.
Note however that the machine requirements for CVS trees of this size are not trivial: it can take several hours on a modern system to reconstruct the history of such a tree and calculate the millions of cryptographic certificates involved. We recommend experimenting with smaller trees first, to get a feel for the import process.
We will assume certain values for this example which will differ in your case:
test.net in this example.
import@test.net in this example.
wobbler in this example.
test.db in this example.
/usr/local/cvsroot in this example.
Accounting for these differences at your site, the following is an example procedure for importing a CVS repository "from scratch", and checking the resulting head version of the import out into a working copy:
$ monotone --db=test.db db init
$ monotone --db=test.db genkey import@test.net
$ monotone --db=test.db --branch=net.test.wobbler cvs_import /usr/local/cvsroot
$ monotone --db=test.db --branch=net.test.wobbler checkout wobber-checkout
This chapter translates common CVS commands into monotone commands. It is an easy alternative to reading through the complete command reference.
$ CVSROOT=:pserver:cvs.foo.com/wobbler
$ cvs -d $CVSROOT checkout -r 1.2
|
$ monotone pull www.foo.com com.foo.wobbler
$ monotone checkout fe37 wobbler
|
The CVS command contacts a network server, retrieves a version, and stores it in your working copy. There are two cosmetic differences with the monotone command: remote databases are specified by hostnames and collections, and versions are denoted by SHA1 values (or selectors).
There is also one deep difference: pulling versions into your database is a separate step from checking out a single version; after you have pulled from a network server, your database will contain several versions, possibly the entire history of a project. Checking out is a separate step, after communication, which only copies a particular version out of your database and into a named directory.
$ cvs commit -m 'log message'
|
$ monotone commit 'log message'
$ monotone push www.foo.com com.foo.wobbler
|
As with other networking commands, the communication step with
monotone is explicit: committing changes only saves them to the local
database. A separate command, push, sends the changes to a
remote database.
$ cvs update -d
|
$ monotone pull www.foo.com com.foo.wobbler
$ monotone merge
$ monotone update
|
This command, like other networking commands, involves a separate
communication step with monotone. The extra command, merge,
ensures that the branch your are working on has a unique head. You can
omit the merge step if you only want update to
examine descendants of your base version, and ignore other heads on
your branch.
$ cvs diff
|
$ monotone diff
|
$ cvs diff -r 1.2 -r 1.4
|
$ monotone diff 3e7db 278df
|
Monotone's diff command is modeled on that of CVS, so the
main features are the same: diff alone prints the
differences between your working copy and its base version, whereas
diff accompanied by two version numbers prints the
difference between those two versions. The major difference between
CVS and monotone here is that monotone's version numbers are
manifest versions, so the diff command prints the
difference between the two entire trees.
$ cvs status
|
$ monotone status
|
This command operates similarly in monotone and CVS. The only major
difference is that monotone's status command always gives a
status of the whole tree, and outputs a more compact summary than CVS.
$ cvs add dir
$ cvs add dir/subdir
$ cvs add dir/subdir/file.txt
|
$ monotone add dir/subdir/file.txt
|
Monotone does not explicitly store directories, so adding a file only involves adding the file's complete path, including any directories. Directories are created as needed, and empty directories are ignored.
$ rm file.txt
$ cvs remove file.txt
|
$ monotone drop file.txt
|
Monotone does not require that you erase a file from the working copy before you drop it. Dropping a file merely removes its entry in the manifest.
$ cvs init -d /path/to/repository
|
$ monotone db init --db=/path/to/database.db
|
Monotone's "repository" is a single-file database, which is created and initialized by this command. This file is only ever used by you, and does not need to be in any special location, or readable by other users.
Monotone has a large number of commands. To help navigate through them all, commands are grouped into logical categories.
monotone cat file idmonotone cat manifest idmonotone checkout id directorymonotone co id directorymonotone --branch=branchname checkout directorymonotone --branch=branchname co directory/MT/manifest. Then they copies every
file version listed in the manifest to paths under
directory. For example, if the manifest contains these entries:
84e2c30a2571bd627918deee1e6613d34e64a29e Makefile
c61af2e67eb9b81e46357bb3c409a9a53a7cdfc6 include/hello.h
97dfc6fd4f486df95868d85b4b81197014ae2a84 src/hello.c
Then the following files are created:
directory/Makefile
directory/include/hello.h
directory/src/hello.c
If you wish to checkout in the current directory, you can
supply the special name . (a single period) for
directory.
If no id is provided, as in the latter two commands, you
must provide a branchname; monotone will attempt to infer
id as the unique head of branchname if it exists.
monotone heads --branch=branchnameThe "heads" of a branch is the set of manifests which are members of
the branch, but which have no descendants. These manifests are
generally the "newest" versions committed by you or your colleagues,
at least in terms of ancestry. The heads of a branch may not be the
newest versions, in terms of time, but synchronization of time is not
reliable, so monotone usually ignores time.
monotone merge --branch=branchnameMerging is performed by repeated pairwise merges: two heads are selected, then their least common ancestor is located in the ancestry graph and these 3 manifests are provided to the built-in 3-way merge algorithm. The process then repeats for each additional head, using the result of each previous merge as an input to the next.
monotone propagate sourcebranch destbranchpropagate aborts, doing nothing.
The purpose of propagate is to copy all the changes on
sourcebranch, since the last propagate, to
destbranch. This command supports the idea of making separate
branches for medium-length development activities, such as
maintenance branches for stable software releases, trivial bug fix
branches, public contribution branches, or branches devoted to the
development of a single module within a larger project.
monotone add filename...MT/work, and is a list of
explicit pathname changes you wish to commit at some future time, such
as addition and removal of files.
While this command places an "add" entry on your work list, it does
not affect your database. When you commit your working copy,
the added files on your work list will be copied into your database
and their paths and SHA1 values added to your working copy manifest.
monotone drop filename...MT/work, and is a list of
explicit pathname changes you wish to commit at some future time, such
as addition and removal of files.
While this command places an "drop" entry on your work list, it does
not affect your database. When you commit your working copy,
the dropped files on your work list will have their entries removed
from your working copy's manifest.
monotone rename src dstYou should run monotone rename before you rename a
file or directory in your source tree. Monotone will complain if it
cannot find the source of a rename.
monotone commit
monotone commit logmsgMT/manifest file and
the files in your tree, subject to the additions and deletions in your
MT/work file.
Once the changes are recorded in you database, commit
overwrites the MT/manifest file with the changed manifest, and
deletes the MT/work file.
The commit command also synthesizes a number of
certificates, which it attaches to the new manifest version and copies
into your database:
ancestor cert, connecting the new version to its base
version in the ancestry graph.
author cert, indicating the person responsible for the changes
leading to the new version.
date cert, indicating when the new version was created.
changelog cert, containing the "log message" for these
changes. If you provided logmsg on the command line, this text
will be used, otherwise commit will run the Lua hook
edit_comment (commentary), which typically invokes an
external editor program, in which you can compose a log message for
the change.
monotone revert
MT/manifest to have contents
equal to the SHA1 value listed in the manifest, and by erasing the
MT/work file.
monotone update
monotone serve address collectionmonotone pull address collectionmonotone push address collectionmonotone sync address collectionThe network address specified in each case should be the same: a host name to listen on, or connect to, optionally followed by a colon and a port number. The collection parameter indicates a set of branches to exchange; every branch for which collection is a prefix will be indexed and made available for synchronization.
For example, supposing bob and alice wish to synchronize their
net.venge.monotone.win32 and net.venge.monotone.i18n
branches. Supposing alice's computer has hostname
alice.someisp.com, then alice might run:
$ monotone serve alice.someisp.com net.venge.monotone
And bob might run
$ monotone sync alice.someisp.com net.venge.monotone
When the operation completes, all branches beginning with
net.venge.monotone will be synchronized between alice and bob's
databases.
monotone status
MT/manifest,
which your working copy is an in-progress descendant of.
MT/work to MT/manifest, and
updating any SHA1 values of files to reflect changes you have
made to the working copy. In other words, the current manifest version
is the version you would commit, if you ran monotone commit.
monotone log
monotone log idmonotone complete file partial-idmonotone complete manifest partial-idfile or manifest
versions depending on which variant is used. For example, suppose you
enter this command and get this result:
$ monotone complete manifest fa36
fa36deead87811b0e15208da2853c39d2f6ebe90
fa36b76dd0139177b28b379fe1d56b22342e5306
fa36965ec190bee14c5afcac235f1b8e2239bb2a
Then monotone is telling you that there are 3 manifests it knows
about, in its database, which begin with the 4 hex digits
fa36. This command is intended to be used by programmable
completion systems, such as those in bash and zsh.
monotone diff
monotone diff idmonotone diff id1 id2monotone diff will print the
differences between the base manifest version and the current manifest
version in your working copy. If one argument id is given,
monotone will print the differences between the manifest id and
the current manifest version in your working copy. If two arguments
id1 and id2 are given, monotone will print the differences
between manifests id1 and id2, ignoring your working copy.
In all cases, monotone will print a textual summary - identical to
the summary presented by monotone status - of the logical
differences between manifests in lines proceeding the diff. These
lines begin with a single hash mark #, and should be ignored by
a program processing the diff, such as patch.
monotone list certs idok or bad
For example, this command lists the certificates associated with a particular version of monotone itself, in the monotone development branch:
$ ./monotone list certs manifest 4a96
monotone: expandeding partial id '4a96'
monotone: expanded to '4a96a230293456baa9c6e7167cafb3c5b52a8e7f'
-----------------------------------------------------------------
Key : graydon@pobox.com
Sig : ok
Name : ancestor
Value : 680e439ea20199f76c1aca1dcac5455b445fc965
-----------------------------------------------------------------
Key : graydon@pobox.com
Sig : ok
Name : author
Value : graydon@dub.venge.net
-----------------------------------------------------------------
Key : graydon@pobox.com
Sig : ok
Name : branch
Value : monotone
-----------------------------------------------------------------
Key : graydon@pobox.com
Sig : ok
Name : date
Value : 2003-10-17T03:20:27
-----------------------------------------------------------------
Key : graydon@pobox.com
Sig : ok
Name : changelog
Value : 2003-10-16 graydon hoare <graydon@pobox.com>
:
: * sanity.hh: Add a const version of idx().
: * diff_patch.cc: Change to using idx() everywhere.
: * cert.cc (find_common_ancestor): Rewrite to recursive
: form, stepping over historic merges.
: * tests/t_cross.at: New test for merging merges.
: * testsuite.at: Call t_cross.at.
:
monotone list keys
monotone list keys patternIf pattern is provided, it is used as a glob to limit the keys
listed. Otherwise all keys in your database are listed.
monotone list branches
monotone list unknown
monotone list ignored
ignore_file (filename) hook.
monotone list missing
monotone genkey keyidnon_blocking_rng_ok() returns true, the key
generation will use an unlimited random number generator (such as
/dev/urandom), otherwise it will use a higher quality random
number generator (such as /dev/random) but might run slightly
slower.
The private half of the key is stored in an encrypted form, using the
symmetric cipher ARC4, so that anyone accidentally reading your
database cannot extract your private key and use it. You must provide
a passphrase for your key when it is generated, which is used to key
the ARC4 cipher. In the future you will need to enter this
passphrase again each time you sign a certificate, which happens every
time you commit to your database. You can tell monotone to
automatically use a certain passphrase for a given key using the
get_passphrase(keypair_id), but this significantly
increases the risk of a key compromise on your local computer. Be
careful using this hook.
monotone cert id certnamemonotone cert id certname certvalstdin.
monotone --branch=branchname vcheck create
monotone vcheck create idmonotone --branch=branchname vcheck check
monotone vcheck check idvcheck certs for the given
manifest id. A vcheck cert on a manifest simply reduces
the probability that you and your colleague have different manifests
or files with the same (colliding) SHA1 value. You can produce as
many vcheck certs as you like.
If you omit the id, these commands operate on the heads of branchname, which you must provide.
monotone approve id1 id2monotone cert id2 ancestor id1monotone disapprove id1 id2monotone cert id2 disapproval id1monotone comment idmonotone comment id commentmonotone cert id
comment commentstdin.
monotone tag id tagnamemonotone cert id tag
tagnamemonotone testresult id 0
monotone testresult id 1
monotone cert id
testresult 0 or monotone cert id testresult 1.
Monotone can produce and consume data in a convenient, portable form called packets. A packet is a sequence of ASCII text, wrapped at 70-columns and easily sent through email or other transports. If you wish to manually transmit a piece of information - for example a public key - from one monotone database to another, it is often convenient to read and write packets.
Note: earlier versions of monotone queued and replayed packet streams for their networking system. This older networking system is deprecated and will be removed in a future version, as the netsync protocol has several properties which make it advantageous as a communication system. However, the packet i/o facility will remain in monotone as a utility for moving individual data items around manually.
monotone fcerts idmonotone mcerts idfcert or mcert packet for
each file or manifest cert in your database associated with id.
These can be used to transport certificates safely between monotone
databases.
monotone fdata idmonotone mdata idfdata or mdata packet for
the file or manifest id in your database. These can be used to
transport file or manifest versions, in their entirety, safely between
monotone databases.
monotone fdelta id1 id2monotone mdelta id1 id2fdelta or mdelta packet for
the differences between file or manifest versions id1 and
id2, in your database. These can be used to transport file or
manifest differences safely between monotone databases.
monotone privkey keyidmonotone pubkey keyidprivkey or pubkey packet for
the RSA key keyid. These can be used to transport public or
private keys safely between monotone databases.
monotone read
stdin and applies them to your
database.
monotone db init --db=dbfiledbfile.
monotone db info --db=dbfiledbfile,
including its schema version and various table size statistics.
monotone db version --db=dbfiledbfile.
monotone db dump --db=dbfiledbfile to the standard output stream. It is a very low-level
command, and produces the most "recoverable" dumps of your database
possible. It is sometimes also useful when migrating databases between
variants of the underlying sqlite database format.
monotone db load --db=dbfiledbfile. It is most useful when loading
a database dumped with the dump command.
Note that when reloading a dumped database, the schema of the dumped
database is included in the dump, so you should not try to
init your database before a load.
monotone db migrate --db=dbfiledbfile to the
newest schema known by the version of monotone you are currently
running. If the migration fails, no changes should be made to the
database.
If you have important information in your database, you should back up
a copy of it before migrating, in case there is an untrapped error
during migration.
monotone db execute sql-statementmonotone rcs_import filename...,v. Monotone parses them directly and inserts them into your
database.
monotone cvs_import pathname...Monotone's behavior can be customized and extended by writing hook functions, which are written in the Lua programming language. At certain points in time, when monotone is running, it will call a hook function to help it make a decision or perform some action. If you provide a hook function definition which suits your preferences, monotone will execute it. This way you can modify how monotone behaves.
You can put new definitions for any of these hook functions in a file
$HOME/.monotonerc, or in your working copy in
MT/monotonerc, both of which will be read every time monotone
runs. Definitions in MT/monotonerc shadow (override)
definitions made in your $HOME/.monotonerc. You can also tell
monotone to interpret extra hook functions from any other file
using the --rcfile=file option; hooks defined in files
specified on the command-line will shadow hooks from the the automatic
files.
The remainder of this section documents the existing hook functions and their default definitions.
note_commit (new_id, certs)
Note that since the certs table does not contain cryptographic
or trust information, and only contains one entry per cert name, it is
an incomplete source of information about the committed version. This
hook is only intended as an aid for integrating monotone with informal
commit-notification systems such as mailing lists or news services. It
should not perform any security-critical operations.
get_branch_key (branchname)
--key=keyname--key=keynameget_passphrase (keypair_id)
get_author (branchname)
author certificates when you commit changes to
branchname. Generally this hook remains undefined, and monotone
selects your signing key name for the author certificate. You can use
this hook to override that choice, if you like.
This hook has no default definition, but a possible definition might be:
function get_author(branchname)
local user = os.getenv("USER")
local host = os.getenv("HOSTNAME")
if ((user == nil) or (host == nil)) then return nil end
return string.format("%s@%s", user, host)
end
edit_comment (commentary)
monotone status. This hook is intended to interface with
some sort of editor, so that you can interactively document each
change you make. The result is used as the value for a
changelog certificate, automatically generated when you commit
changes.
The default definition of this hook is:
function edit_comment(commentary)
local exe = "vi"
local visual = os.getenv("VISUAL")
if (visual ~= nil) then exe = visual end
local editor = os.getenv("EDITOR")
if (editor ~= nil) then exe = editor end
local tmp, tname = temp_file()
if (tmp == nil) then return nil end
commentary = "MT: " .. string.gsub(commentary, "\n", "\nMT: ")
tmp:write(commentary)
io.close(tmp)
if (os.execute(string.format("%s %s", exe, tname)) ~= 0) then
os.remove(tname)
return nil
end
tmp = io.open(tname, "r")
if (tmp == nil) then os.remove(tname); return nil end
local res = ""
local line = tmp:read()
while(line ~= nil) do
if (not string.find(line, "^MT:")) then
res = res .. line .. "\n"
end
line = tmp:read()
end
io.close(tmp)
os.remove(tname)
return res
end
persist_phrase_ok ()
true if you want monotone to remember the passphrase of
a private key for the duration of a single command, or false if
you want monotone to prompt you for a passphrase for each certificate
it generates. Since monotone often generates several certificates in
quick succession, unless you are very concerned about security you
probably want this hook to return true.
The default definition of this hook is:
function persist_phrase_ok()
return true
end
non_blocking_rng_ok ()
true if you are willing to let monotone use the
system's non-blocking random number generator, such as
/dev/urandom, for generating random values during cryptographic
operations. This diminishes the cryptographic strength of such
operations, but speeds them up. Returns false if you want to
force monotone to always use higher quality random numbers, such as
those from /dev/random.
The default definition of this hook is:
function non_blocking_rng_ok()
return true
end
get_netsync_read_permitted (collection, identity)
true if a peer authenticated as key identity
should be allowed to read from your database certs, manifests, and
files associated with the netsync index collection; otherwise
false. This hook has no default definition, therefore the
default behavior is to deny all reads.
Note that the identity value is a key id (such as
"graydon@pobox.com") but will correspond to a unique
key fingerprint (hash) in your database. Monotone will not permit two
keys in your database to have the same id. Make sure you confirm the
key fingerprints of each key in your database, as key id strings are
"convenience names", not security tokens.
get_netsync_anonymous_read_permitted (collection)
get_netsync_read_permitted
except that it is called when a connecting client requests anonymous
read access to a collection. There is no corresponding anonymous write
access hook. This hook has no default definition, therefore the
default behavior is to deny all anonymous reads.
get_netsync_write_permitted (collection, identity)
true if a peer authenticated as key identity
should be allowed to write into your database certs, manifests, and
files associated with the netsync index collection; otherwise
false. This hook has no default definition, therefore the
default behavior is to deny all writes.
Note that the identity value is a key id (such as
"graydon@pobox.com") but will correspond to a unique
key fingerprint (hash) in your database. Monotone will not permit two
keys in your database to have the same id. Make sure you confirm the
key fingerprints of each key in your database, as key id strings are
"convenience names", not security tokens.
ignore_file (filename)
true if filename should be ignored while adding,
dropping, or moving files. Otherwise returns false. This is
most important when performing recursive actions on directories, which
may affect multiple files simultaneously. The default definition of
this hook is:
function ignore_file(name)
if (string.find(name, "%.a$")) then return true end
if (string.find(name, "%.so$")) then return true end
if (string.find(name, "%.o$")) then return true end
if (string.find(name, "%.la$")) then return true end
if (string.find(name, "%.lo$")) then return true end
if (string.find(name, "%.aux$")) then return true end
if (string.find(name, "%.bak$")) then return true end
if (string.find(name, "%.orig$")) then return true end
if (string.find(name, "%.rej$")) then return true end
if (string.find(name, "%~$")) then return true end
if (string.find(name, "/core$")) then return true end
if (string.find(name, "^CVS/")) then return true end
if (string.find(name, "/CVS/")) then return true end
if (string.find(name, "^%.svn/")) then return true end
if (string.find(name, "/%.svn/")) then return true end
if (string.find(name, "^SCCS/")) then return true end
if (string.find(name, "/SCCS/")) then return true end
return false;
end
get_manifest_cert_trust (signers, id, name, val)
The default definition of this hook simply returns true, which
corresponds to a form of trust where every key which is defined in
your database is trusted. This is a weak trust setting; you
should change it to something stronger. A possible example of a
stronger trust function (along with a utility function for computing
the intersection of tables) is the following:
function intersection(a,b)
local s={}
local t={}
for k,v in pairs(a) do s[v] = 1 end
for k,v in pairs(b) do if s[v] ~= nil then table.insert(t,v) end end
return t
end
function get_manifest_cert_trust(signers, id, name, val)
local trusted_signers = { "bob@happyplace.com",
"friend@trustedplace.com",
"myself@home.com" }
local t = intersection(signers, trusted_signers)
if t == nil then return false end
if (name ~= "ancestor" and table.getn(t) >= 1)
or (name == "ancestor" and table.getn(t) >= 2)
then
return true
else
return false
end
end
In this example, any manifest certificate is trusted if it is signed
by at least one of three "trusted" keys, unless it is an
ancestor certificate, in which case it must be signed by
two or more trusted keys. This is one way of requiring that
ancestry assertions go through an extra "reviewer" before they are
accepted.
get_file_cert_trust (signers, id, name, val)
get_manifest_cert_trust,
except it is called for file certificates rather than manifest
certificates. See the documentation for that hook for more details.
accept_testresult_change (old_results, new_results)
true if you consider an update from the
version carrying the old_results to the version carrying the
new_results to be acceptable.
The default definition of this hook follows:
function accept_testresult_change(old_results, new_results)
for test,res in pairs(old_results)
do
if res == true and new_results[test] ~= true
then
return false
end
end
return true
end
This definition accepts only those updates which preserve the set of
true test results from update source to target. If no rest
results exist, this hook has no affect; but once a true test
result is present, future updates will require it. If you want a more
lenient behavior you must redefine this hook.
merge2 (left, right)
function merge2(left, right)
local lfile = nil
local rfile = nil
local outfile = nil
local data = nil
lfile = write_to_temporary_file(left)
rfile = write_to_temporary_file(right)
outfile = write_to_temporary_file("")
if lfile ~= nil and
rfile ~= nil and
outfile ~= nil
then
local cmd = nil
if program_exists_in_path("xxdiff") then
cmd = merge2_xxdiff_cmd(lfile, rfile, outfile)
elseif program_exists_in_path("emacs") then
cmd = merge2_emacs_cmd("emacs", lfile, rfile, outfile)
elseif program_exists_in_path("xemacs") then
cmd = merge2_emacs_cmd("xemacs", lfile, rfile, outfile)
end
if cmd ~= nil
then
io.write(string.format("executing external 2-way merge command: %s\n", cmd))
os.execute(cmd)
data = read_contents_of_file(outfile)
else
io.write("no external 2-way merge command found")
end
end
os.remove(lfile)
os.remove(rfile)
os.remove(outfile)
return data
end
merge3 (ancestor, left, right)
function merge3(ancestor, left, right)
local afile = nil
local lfile = nil
local rfile = nil
local outfile = nil
local data = nil
lfile = write_to_temporary_file(left)
afile = write_to_temporary_file(ancestor)
rfile = write_to_temporary_file(right)
outfile = write_to_temporary_file("")
if lfile ~= nil and
rfile ~= nil and
afile ~= nil and
outfile ~= nil
then
local cmd = nil
if program_exists_in_path("xxdiff") then
cmd = merge3_xxdiff_cmd(lfile, afile, rfile, outfile)
elseif program_exists_in_path("emacs") then
cmd = merge3_emacs_cmd("emacs", lfile, afile, rfile, outfile)
elseif program_exists_in_path("xemacs") then
cmd = merge3_emacs_cmd("xemacs", lfile, afile, rfile, outfile)
end
if cmd ~= nil
then
io.write(string.format("executing external 3-way merge command: %s\n", cmd))
os.execute(cmd)
data = read_contents_of_file(outfile)
else
io.write("no external 3-way merge command found")
end
end
os.remove(lfile)
os.remove(rfile)
os.remove(afile)
os.remove(outfile)
return data
end
expand_selector (str)
a: for
authors or d: for dates. Expansion may also mean recognizing
and interpreting special words such as yesterday or 6
months ago and converting them into well formed selectors. For more
detail on the use of selectors, see Selectors. The default
definition of this hook is:
function expand_selector(str)
-- simple date patterns
if string.find(str, "^19%d%d%-%d%d")
or string.find(str, "^20%d%d%-%d%d")
then
return ("d:" .. str)
end
-- something which looks like an email address
if string.find(str, "[%w%-_]+@[%w%-_]+")
then
return ("a:" .. str)
end
-- something which looks like a branch name
if string.find(str, "[%w%-]+%.[%w%-]+")
then
return ("b:" .. str)
end
-- a sequence of nothing but hex digits
if string.find(str, "^%x+$")
then
return ("i:" .. str)
end
-- "yesterday", the source of all hangovers
if str == "yesterday"
then
local t = os.time(os.date('!*t'))
return os.date("d:%F", t - 86400)
end
-- "CVS style" relative dates such as "3 weeks ago"
local trans = {
minute = 60;
hour = 3600;
day = 86400;
week = 604800;
month = 2678400;
year = 31536000
}
local pos, len, n, type = string.find(str, "(%d+) ([minutehordaywk]+)s? ago")
if trans[type] ~= nil
then
local t = os.time(os.date('!*t'))
return os.date("d:%F", t - (n * trans[type]))
end
return nil
end
get_system_linesep ()
CR, LF, or
CRLF. The system line separator may be used when reading or
writing data to the terminal, or otherwise interfacing with the user.
The system line separator is not used to convert files in the working
copy; use get_linesep_conv for converting line endings in the
working copy.
This hook has no default definition. For more information on line
ending conversion, see the section on Internationalization.
get_linesep_conv (filename)
CR, LF, or
CRLF.
When filename is read from the working copy, it is run through line ending conversion from the external form to the internal form. When filename is written to the working copy, it is run through line ending conversion from the internal form to the external form. SHA1 values are calculated from the internal form of filename. It is your responsibility to decide which line ending conversions your work will use.
This hook has no default definition; monotone's default behavior is to
keep external and internal forms byte-for-byte identical. For more
information on line ending conversion, see the section on
Internationalization.
get_charset_conv (filename)
When filename is read from the working copy, it is run through character set conversion from the external form to the internal form. When filename is written to the working copy, it is run through character set conversion from the internal form to the external form. SHA1 values are calculated from the internal form of filename. It is your responsibility to decide which character set conversions your work will use.
This hook has no default definition; monotone's default behavior is to
keep external and internal forms byte-for-byte identical. For more
information on character set conversion, see the section on
Internationalization.
attr_functions [arrtibute] (filename, value)
attr_functions, at table
entry attribute, is a function taking a file name filename
and a attribute value value. The function should "apply" the
attribute to the file, possibly in a platform-specific way.
Persistent attributes are stored in the .mt-attrs, in your
working copy and manifest. If such a file exists, hook functions from
this table are called for each triple found in the file, after any
command which modifies the working copy. This facility can be used to
extend monotone's understanding of files with platform-specific attributes,
such as permission bits, access control lists, or special file types.
By default, there is only one entry in this table, for the execute
attribute. Its definition is:
attr_functions["execute"] =
function(filename, value)
if (value == "true") then
os.execute(string.format("chmod +x %s", filename))
end
end
monotone - distributed version control system
monotone [options] <command> [parameters] Options, which affect global behavior or set default values, come first in the argument list. A single command must follow, indicating the operation to perform, followed by parameters which vary depending on the command.
This man page is a summary of some of the features and commands of monotone, but it is not the most detailed source of information available. For a complete discussion of the concepts and a tutorial on its use, please refer to the texinfo manual (via the info monotone command, or online).
<id> <id1> <id2> <id1> <id2> <id> <tagname> <id> (0|1) [id] <id> <certname> [certval] (create|check) [id]vcheck certs for the given manifest [id].
<keyid> <id> <id> [partial-id] [partial-id] <id> <id> <oldid> <newid> <id> <id> <id> <id> <cvsroot> <rcsfile> ... [manifest-id] <directory> [manifest-id] <directory> (file|manifest) <id> <pathname> [...] [log message] <pathname> [...] <src> <dst> <host> <collection> <host> <collection> <host> <collection> <host> <collection> (init|info|version|dump|load|migrate|execute <sql>)Monotone is a version control system, which allows you to keep old versions of files, as well as special manifest files which describe the location of files in a tree. Unlike other systems, versions in monotone are identified by cryptographic hash, and operations are authenticated by individual users' evaluating cryptographic signatures on meta-data, rather than any central authority.
Monotone keeps a collection of versions in a single-file relational database. It is essentially serverless, using network servers only as untrusted communication facilities. A monotone database is a regular file, which contains all the information needed to extract previous versions of files, verify signatures, merge and modify versions, and communicate with network servers.
Command line options override environment variables and settings in Lua scripts (such as .monotonerc)
see http://savannah.nongnu.org/bugs/?group=monotone
graydon hoare <graydon@pobox.com>
accept_testresult_change (old_results, new_results): Hook Reference
attr_functions [arrtibute] (filename, value): Hook Reference
edit_comment (commentary): Hook Reference
expand_selector (str): Hook Reference
get_author (branchname): Hook Reference
get_branch_key (branchname): Hook Reference
get_charset_conv (filename): Hook Reference
get_file_cert_trust (signers, id, name, val): Hook Reference
get_linesep_conv (filename): Hook Reference
get_manifest_cert_trust (signers, id, name, val): Hook Reference
get_netsync_anonymous_read_permitted (collection): Hook Reference
get_netsync_read_permitted (collection, identity): Hook Reference
get_netsync_write_permitted (collection, identity): Hook Reference
get_passphrase (keypair_id): Hook Reference
get_system_linesep (): Hook Reference
ignore_file (filename): Hook Reference
merge2 (left, right): Hook Reference
merge3 (ancestor, left, right): Hook Reference
monotone --branch=branchname checkout directorymonotone --branch=branchname co directorymonotone --branch=branchname vcheck check: Key and Cert
monotone --branch=branchname vcheck create: Key and Cert
monotone add filename...monotone approve id1 id2monotone cat file idmonotone cat manifest idmonotone cert id certnamemonotone cert id certname certvalmonotone checkout id directorymonotone co id directorymonotone comment idmonotone comment id commentmonotone commit: Working Copy
monotone commit logmsgmonotone complete file partial-idmonotone complete manifest partial-idmonotone cvs_import pathname...monotone db dump --db=dbfilemonotone db execute sql-statementmonotone db info --db=dbfilemonotone db init --db=dbfilemonotone db load --db=dbfilemonotone db migrate --db=dbfilemonotone db version --db=dbfilemonotone diff: Informative
monotone diff idmonotone diff id1 id2monotone disapprove id1 id2monotone drop filename...monotone fcerts idmonotone fdata idmonotone fdelta id1 id2monotone genkey keyidmonotone heads --branch=branchnamemonotone list branches: Informative
monotone list certs idmonotone list ignored: Informative
monotone list keys: Informative
monotone list keys patternmonotone list missing: Informative
monotone list unknown: Informative
monotone log: Informative
monotone log idmonotone mcerts idmonotone mdata idmonotone mdelta id1 id2monotone merge --branch=branchnamemonotone privkey keyidmonotone propagate sourcebranch destbranchmonotone pubkey keyidmonotone pull address collectionmonotone push address collectionmonotone rcs_import filename...monotone read: Packet I/O
monotone rename src dstmonotone revert: Working Copy
monotone serve address collectionmonotone status: Informative
monotone sync address collectionmonotone tag id tagnamemonotone testresult id 0: Certificate
monotone testresult id 1: Certificate
monotone update: Working Copy
monotone vcheck check idmonotone vcheck create idnon_blocking_rng_ok (): Hook Reference
note_commit (new_id, certs): Hook Reference
persist_phrase_ok (): Hook Reference