Tokyo Tyrant is a package of network interface to the DBM called Tokyo Cabinet. Though the DBM has high performance, you might bother in case that multiple processes share the same database, or remote processes access the database. Thus, Tokyo Tyrant is provided for concurrent and remote connections to Tokyo Cabinet. It is composed of the server process managing a database and its access library for client applications.
The server features high concurrency due to thread-pool modeled implementation and the "epoll" mechanism of the modern Linux kernel. The server and its clients communicate with each other by simple binary protocol on TCP/IP. Protocols compatible with memcached and HTTP/1.1 are also supported so that almost all principal platforms and programming languages can use Tokyo Tyrant. High availability and high integrity are also featured due to hot such mechanisms as hot backup, update logging, and replication.
As for now, the server works on Linux only.
Install the latest version of Tokyo Cabinet beforehand and get the package of Tokyo Tyrant.
When an archive file of Tokyo Tyrant is extracted, change the current working directory to the generated directory and perform installation.
Run the configuration script.
./configure
Build programs.
make
Install programs. This operation must be carried out by the root user.
make install
When a series of work finishes, the following files will be installed.
/usr/local/include/ttutil.h /usr/local/include/tculog.h /usr/local/include/tcrdb.h /usr/local/lib/libtokyotyrant.a /usr/local/lib/libtokyotyrant.so.1.15.0 /usr/local/lib/libtokyotyrant.so.1 /usr/local/lib/libtokyotyrant.so /usr/local/lib/pkgconfig/tokyotyrant.pc /usr/local/bin/ttserver /usr/local/bin/ttultest /usr/local/bin/ttulmgr /usr/local/bin/tcrtest /usr/local/bin/tcrmttest /usr/local/bin/tcrmgr /usr/local/sbin/ttservctl /usr/local/share/tokyotyrant/... /usr/local/man/man1/... /usr/local/man/man3/...
To test the server, perform the following command. To finish it, press Ctrl-C on the terminal.
ttserver
To test the client connecting to the above running server, perform the following command on another terminal.
make check
The command `ttserver' runs the server managing a database instance. Because the database is treated by the abstract API of Tokyo Cabinet, you can choose the scheme on start-up of the server. Supported schema are on-memory database, hash database, and B+ tree database. This command is used in the following format. `dbname' specifies the database name. If it is omitted, on-memory database is specified.
ttserver [-host name] [-port num] [-thnum num] [-tout num] [-dmn] [-pid path] [-log path] [-ld|-le] [-ulog path] [-ulim num] [-uas] [-sid num] [-mhost name] [-mport num] [-rts path] [dbname]Options feature the following.
To terminate the server normally, send SIGINT or SIGTERM to the process. It is okay to press Ctrl-C on the controlling terminal. To restart the server, send SIGHUP to the process. If the port number is not more than 0, UNIX domain socket is used and the path of the socket file is specified by the host parameter. This command returns 0 on success, another on failure.
The naming convention of the database is specified by the abstract API of Tokyo Cabinet. If the name is "*", the database will be an on-memory database. If its suffix is ".tch", the database will be a hash database. If its suffix is ".tcb", the database will be a B+ tree database. If its suffix is ".tcf", the database will be a fixed-length database. Otherwise, this function fails. Tuning parameters can trail the name, separated by "#". Each parameter is composed of the name and the number, separated by "=". On-memory database supports "bnum", "capnum", and "capsiz". Hash database supports "mode", "bnum", "apow", "fpow", "opts", and "rcnum". B+ tree database supports "mode", "lmemb", "nmemb", "bnum", "apow", "fpow", "opts", "lcnum", and "ncnum". Fixed-length database supports "mode", "width", and "limsiz". "capnum" specifies the capacity number of records. "capsiz" specifies the capacity size of using memory. Records spilled the capacity are removed by the storing order. "mode" can contain "w" of writer, "r" of reader, "c" of creating, "t" of truncating, "e" of no locking, and "f" of non-blocking lock. The default mode is relevant to "wc". "opts" can contains "l" of large option, "d" of Deflate option, and "b" of TCBS option. "width" specifies the width of the value of each record. "limsiz" specifies the limit size of the database file. For example, "casket.tch#bnum=1000000#opts=ld" means that the name of the database file is "casket.tch", and the bucket number is 1000000, and the options are large and Deflate.
The command `ttservctl' is the startup script of the server. It can be called by the RC script of the bootstrap process of the operating system. This command is used in the following format.
ttservctl startttservctl stopttservctl restartttservctl hupThe database is placed as "/var/ttserver/casket.tch". The log and related files are also placed in "/var/ttserver". This command returns 0 on success, another on failure.
The command `ttulmgr' is the utility to export and import the update log. It is useful to filter the update log with such text utilities as `grep' and `sed'. This command is used in the following format. `upath' specifies the update log directory.
ttulmgr export [-ts num] [-sid num] upathttulmgr import upathOptions feature the following.
This command returns 0 on success, another on failure.
The command `tcrtest' is a utility for facility test and performance test. This command is used in the following format. `host' specifies the host name of the server. `rnum' specifies the number of iterations.
tcrtest write [-port num] [-cnum num] [-nr] [-rnd] host rnumtcrtest read [-port num] [-cnum num] [-mul num] [-rnd] hosttcrtest remove [-port num] [-cnum num] [-rnd] hosttcrtest rcat [-port num] [-cnum num] [-rtt num] host rnumtcrtest misc [-port num] [-cnum num] host rnumtcrtest wicked [-port num] [-cnum num] host rnumOptions feature the following.
If the port number is not more than 0, UNIX domain socket is used and the path of the socket file is specified by the host parameter. This command returns 0 on success, another on failure.
The command `tcrmttest' is a utility for facility test under multi-thread situation. This command is used in the following format. `host' specifies the host name of the server. `rnum' specifies the number of iterations.
tcrmttest write [-port num] [-tnum num] [-nr] [-rnd] host rnumtcrmttest read [-port num] [-tnum num] [-mul num] hosttcrmttest remove [-port num] [-tnum num] hostOptions feature the following.
If the port number is not more than 0, UNIX domain socket is used and the path of the socket file is specified by the host parameter. This command returns 0 on success, another on failure.
The command `tcrmgr' is a utility for test and debugging of the remote database API and its applications. `host' specifies the host name of the server. `key' specifies the key of a record. `value' specifies the value of a record. `dpath' specifies the destination file. `upath' specifies the update log directory. `mhost' specifies the host name of the replication master. `url' specifies the target URL.
tcrmgr inform [-port num] [-st] hosttcrmgr put [-port num] [-sx] [-dk|-dc] host key valuetcrmgr out [-port num] [-sx] host keytcrmgr get [-port num] [-sx] [-px] [-pz] host keytcrmgr mget [-port num] [-sx] [-px] host [key...]tcrmgr list [-port num] [-m num] [-pv] [-px] [-fm str] hosttcrmgr sync [-port num] hosttcrmgr vanish [-port num] hosttcrmgr copy [-port num] host dpathtcrmgr restore [-port num] host upathtcrmgr setmst [-port num] [-mport num] host [mhost]tcrmgr repl [-port num] [-ts num] [-sid num] [-ph] hosttcrmgr http [-ah name value] [-ih] urltcrmgr versionOptions feature the following.
If the port number is not more than 0, UNIX domain socket is used and the path of the socket file is specified by the host parameter. This command returns 0 on success, another on failure.
Remote database is a set of interfaces to use an abstract database of Tokyo Cabinet, mediated by a server of Tokyo Tyrant. See `tcrdb.h' for entire specification.
To use the remote database API, include `tcrdb.h' and related standard header files. Usually, write the following description near the front of a source file.
#include <tcrdb.h>#include <stdlib.h>#include <stdbool.h>#include <stdint.h>Objects whose type is pointer to `TCRDB' are used to handle remote databases. A remote database object is created with the function `tcrdbnew' and is deleted with the function `tcrdbdel'. To avoid memory leak, it is important to delete every object when it is no longer in use.
Before operations to store or retrieve records, it is necessary to connect the remote database object to the server. The function `tcrdbopen' is used to open a database connection and the function `tcrdbclose' is used to close the connection.
The function `tcrdberrmsg' is used in order to get the message string corresponding to an error code.
const char *tcrdberrmsg(int ecode);The function `tcrdbnew' is used in order to create a remote database object.
TCRDB *tcrdbnew(void);The function `tcrdbdel' is used in order to delete a remote database object.
void tcrdbdel(TCRDB *rdb);The function `tcrdbecode' is used in order to get the last happened error code of a remote database object.
int tcrdbecode(TCRDB *rdb);The function `tcrdbopen' is used in order to open a remote database.
bool tcrdbopen(TCRDB *rdb, const char *host, int port);The function `tcrdbclose' is used in order to close a remote database object.
bool tcrdbclose(TCRDB *rdb);The function `tcrdbput' is used in order to store a record into a remote database object.
bool tcrdbput(TCRDB *rdb, const void *kbuf, int ksiz, const void *vbuf, int vsiz);The function `tcrdbput2' is used in order to store a string record into a remote object.
bool tcrdbput2(TCRDB *rdb, const char *kstr, const char *vstr);The function `tcrdbputkeep' is used in order to store a new record into a remote database object.
bool tcrdbputkeep(TCRDB *rdb, const void *kbuf, int ksiz, const void *vbuf, int vsiz);The function `tcrdbputkeep2' is used in order to store a new string record into a remote database object.
bool tcrdbputkeep2(TCRDB *rdb, const char *kstr, const char *vstr);The function `tcrdbputcat' is used in order to concatenate a value at the end of the existing record in a remote database object.
bool tcrdbputcat(TCRDB *rdb, const void *kbuf, int ksiz, const void *vbuf, int vsiz);The function `tcrdbputcat2' is used in order to concatenate a string value at the end of the existing record in a remote database object.
bool tcrdbputcat2(TCRDB *rdb, const char *kstr, const char *vstr);The function `tcrdbputrtt' is used in order to concatenate a value at the end of the existing record and rotate it to the left.
bool tcrdbputrtt(TCRDB *rdb, const void *kbuf, int ksiz, const void *vbuf, int vsiz, int width);The function `tcrdbputrtt2' is used in order to concatenate a string value at the end of the existing record and rotate it to the left.
bool tcrdbputrtt2(TCRDB *rdb, const char *kstr, const char *vstr, int width);The function `tcrdbputnr' is used in order to store a record into a remote database object without response from the server.
bool tcrdbputnr(TCRDB *rdb, const void *kbuf, int ksiz, const void *vbuf, int vsiz);The function `tcrdbputnr2' is used in order to store a string record into a remote object without response from the server.
bool tcrdbputnr2(TCRDB *rdb, const char *kstr, const char *vstr);The function `tcrdbout' is used in order to remove a record of a remote database object.
bool tcrdbout(TCRDB *rdb, const void *kbuf, int ksiz);The function `tcrdbout2' is used in order to remove a string record of a remote database object.
bool tcrdbout2(TCRDB *rdb, const char *kstr);The function `tcrdbget' is used in order to retrieve a record in a remote database object.
void *tcrdbget(TCRDB *rdb, const void *kbuf, int ksiz, int *sp);The function `tcrdbget2' is used in order to retrieve a string record in a remote database object.
char *tcrdbget2(TCRDB *rdb, const char *kstr);The function `tcrdbget3' is used in order to retrieve records in a remote database object.
bool tcrdbget3(TCRDB *rdb, TCMAP *recs);The function `tcrdbvsiz' is used in order to get the size of the value of a record in a remote database object.
int tcrdbvsiz(TCRDB *rdb, const void *kbuf, int ksiz);The function `tcrdbvsiz2' is used in order to get the size of the value of a string record in a remote database object.
int tcrdbvsiz2(TCRDB *rdb, const char *kstr);The function `tcrdbiterinit' is used in order to initialize the iterator of a remote database object.
bool tcrdbiterinit(TCRDB *rdb);The function `tcrdbiternext' is used in order to get the next key of the iterator of a remote database object.
void *tcrdbiternext(TCRDB *rdb, int *sp);The function `tcrdbiternext2' is used in order to get the next key string of the iterator of a remote database object.
char *tcrdbiternext2(TCRDB *rdb);The function `tcrdbfwmkeys' is used in order to get forward matching keys in a remote database object.
TCLIST *tcrdbfwmkeys(TCRDB *rdb, const void *pbuf, int psiz, int max);The function `tcrdbfwmkeys2' is used in order to get forward matching string keys in a remote database object.
TCLIST *tcrdbfwmkeys2(TCRDB *rdb, const char *pstr, int max);The function `tcrdbsync' is used in order to synchronize updated contents of a remote database object with the file and the device.
bool tcrdbsync(TCRDB *rdb);The function `tcrdbvanish' is used in order to remove all records of a remote database object.
bool tcrdbvanish(TCRDB *rdb);The function `tcrdbcopy' is used in order to copy the database file of a remote database object.
bool tcrdbcopy(TCRDB *rdb, const char *path);The function `tcrdbrestore' is used in order to restore the database file of a remote database object from the update log.
bool tcrdbrestore(TCRDB *rdb, const char *path, uint64_t ts);The function `tcrdbsetmst' is used in order to set the replication master of a remote database object from the update log.
bool tcrdbsetmst(TCRDB *rdb, const char *host, int port);The function `tcrdbrnum' is used in order to get the number of records of a remote database object.
uint64_t tcrdbrnum(TCRDB *rdb);The function `tcrdbsize' is used in order to get the size of the database of a remote database object.
uint64_t tcrdbsize(TCRDB *rdb);The function `tcrdbstat' is used in order to get the status string of the database of a remote database object.
char *tcrdbstat(TCRDB *rdb);The following code is an example to use a remote database.
#include <tcrdb.h>
#include <stdlib.h>
#include <stdbool.h>
#include <stdint.h>
int main(int argc, char **argv){
TCRDB *rdb;
int ecode;
char *value;
/* create the object */
rdb = tcrdbnew();
/* open the database */
if(!tcrdbopen(rdb, "localhost", 1978)){
ecode = tcrdbecode(rdb);
fprintf(stderr, "open error: %s\n", tcrdberrmsg(ecode));
}
/* store records */
if(!tcrdbput2(rdb, "foo", "hop") ||
!tcrdbput2(rdb, "bar", "step") ||
!tcrdbput2(rdb, "baz", "jump")){
ecode = tcrdbecode(rdb);
fprintf(stderr, "put error: %s\n", tcrdberrmsg(ecode));
}
/* retrieve records */
value = tcrdbget2(rdb, "foo");
if(value){
printf("%s\n", value);
free(value);
} else {
ecode = tcrdbecode(rdb);
fprintf(stderr, "get error: %s\n", tcrdberrmsg(ecode));
}
/* close the database */
if(!tcrdbclose(rdb)){
ecode = tcrdbecode(rdb);
fprintf(stderr, "close error: %s\n", tcrdberrmsg(ecode));
}
/* delete the object */
tcrdbdel(rdb);
return 0;
}
The API of C is available by programs conforming to the C89 (ANSI C) standard or the C99 standard. As the header files of Tokyo Tyrant are provided as `tcrdb.h', applications should include it to use the API. As the library is provided as `libtokyotyrant.a' and `libtokyotyrant.so' and they depends `libtokyocabinet.so', `libz.so', `libresolv.so', `libnsl.so', `libpthread.so', `libm.so', and `libc.so', linker options `-ltokyotyrant', `-ltokyocabinet', `-lz', `-lresolv', `-lnsl', `-lpthread', `-lm', and `-lc' are required for build command. A typical build command is the following.
gcc -I/usr/local/include tc_example.c -o tc_example \ -L/usr/local/lib -ltokyotyrant -ltokyocabinet -lz -lresolv -lnsl -lpthread -lm -lc
You can also use Tokyo Tyrant in programs written in C++. Because each header is wrapped in C linkage (`extern "C"' block), you can simply include them into your C++ programs.
The protocol between the server and clients stands on TCP/IP. By default, the service port is bound to every address of the local host and the port number is 1978. Each session of the server and a client is composed of a request and a response. Requests are classified into the following commands. Structure of request and response is determined by the command. The byte order of integer in request and response is big endian.
put: for the function `tcrdbput'[magic:2][ksiz:4][vsiz:4][kbuf:*][vbuf:*][code:1]putkeep: for the function `tcrdbputkeep'[magic:2][ksiz:4][vsiz:4][kbuf:*][vbuf:*][code:1]putcat: for the function `tcrdbputcat'[magic:2][ksiz:4][vsiz:4][kbuf:*][vbuf:*][code:1]putrtt: for the function `tcrdbputrtt'[magic:2][ksiz:4][vsiz:4][width:4][kbuf:*][vbuf:*][code:1]putnr: for the function `tcrdbputnr'[magic:2][ksiz:4][vsiz:4][kbuf:*][vbuf:*]out: for the function `tcrdbout'[magic:2][ksiz:4][kbuf:*][code:1]get: for the function `tcrdbget'[magic:2][ksiz:4][kbuf:*][code:1]([vsiz:4][vbuf:*])mget: for the function `tcrdbget3'[magic:2][rnum:4][{[ksiz:4][kbuf:*]}:*][code:1][rnum:4][{[ksiz:4][vsiz:4][kbuf:*][vbuf:*]}:*]vsiz: for the function `tcrdbvsiz'[magic:2][ksiz:4][kbuf:*][code:1]([vsiz:4])iterinit: for the function `tcrdbiterinit'[magic:2][code:1]iternext: for the function `tcrdbiternext'[magic:2][code:1]([ksiz:4][kbuf:*])fwmkeys: for the function `tcrdbfwmkeys'[magic:2][psiz:4][max:4][pbuf:*][code:1][knum:4][{[ksiz:4][kbuf:*]}:*]sync: for the function `tcrdbsync'[magic:2][code:1]vanish: for the function `tcrdbvanish'[magic:2][code:1]copy: for the function `tcrdbcopy'[magic:2][psiz:4][path:*][code:1]restore: for the function `tcrdbrestore'[magic:2][psiz:4][path:*][code:1]setmst: for the function `tcrdbsetmst'[magic:2][hsiz:4][port:4][host:*][code:1]rnum: for the function `tcrdbrnum'[magic:2][code:1][rnum:8]size: for the function `tcrdbsize'[magic:2][code:1][rnum:8]stat: for the function `tcrdbstat'[magic:2][code:1][ssiz:4][sbuf:*]To finish the session, the client can shutdown and close the socket at any time. If not closed, the connection can be reused for the next session. If protocol violation or some fatal error occurs the server immediately breaks the session and closes the connection.
As for the memcached compatible protocol, the server implements the following commands; "set", "add", "replace", "get", "delete", "incr", "decr", "stats", "flush_all", "version", and "quit". "noreply" options of update commands are also supported. However, "flags", "exptime", and "cas unique" parameters are ignored. The limitation size of each query is 8192 bytes.
As for the HTTP/1.1 compatible protocol, the server implements the following commands; "GET" (relevant to `tcrdbget'), "HEAD" (relevant to `tcrdbvsiz'), "PUT" (relevant to `tcrdbput'), "POST" (relevant to `tcrdbputkeep'), and "DELETE" (relevant to `tcrdbout'). The URI of each request is treated as the key encoded by the URL encoding. And the entity body is treated as the value. However, headers except for "Connection" and "Content-Length" are ignored. The limitation size of each query is 8192 bytes.
After installation of Tokyo Tyrant, you can start the server immediately by executing the command `ttserver' in the terminal. By default, the server listens to the port 1978 and serves as the accessor of an on-memory database, which is useful to store cache data.
[terminal-1]$ ttserver
To test storing operations, execute the following commands in another terminal. `tcrmgr put' calls the function `tcrdbput'.
[terminal-2]$ tcrmgr put localhost one first [terminal-2]$ tcrmgr put localhost two second [terminal-2]$ tcrmgr put localhost three third
To test retrieving operations, execute the following commands in another terminal. `tcrmgr get' calls the function `tcrdbget'.
[terminal-2]$ tcrmgr get localhost one [terminal-2]$ tcrmgr get localhost two [terminal-2]$ tcrmgr get localhost three
To retrieve multiple records at once, execute the following command. `tcrmgr mget' calls the function `tcrdbget3'.
[terminal-2]$ tcrmgr mget localhost one two three
To terminate the server, input Ctrl-C in the terminal of the server.
Next, let's run the server that handles a hash database, by specifying the file name whose suffix is `.tch'.
[terminal-1]$ ttserver casket.tch
Store some records.
[terminal-2]$ tcrmgr put localhost one first [terminal-2]$ tcrmgr put localhost two second [terminal-2]$ tcrmgr put localhost three third
Terminate the server by Ctrl-C, and then restart the server.
[terminal-1]$ ttserver casket.tch
Check consistency of stored records.
[terminal-2]$ tcrmgr mget localhost one two three
Terminate the server by Ctrl-C and remove the database, for the successive tutorial.
[terminal-1]$ rm casket.tch
To run the server as a daemon process, specify the option `-dmn'. Moreover, the option `-pid' should be specified to record the process ID into a file. Note that the current directory of the daemon process is changed to the root directory. So, the file path parameter should be expressed as absolute path.
[terminal-1]$ ttserver -dmn -pid /tmp/ttserver.pid /tmp/casket.tch
To terminate the daemonized server, check the process ID from the file specified by `-pid' and send the SIGTERM signal to the process.
[terminal-1]$ kill -TERM `cat /tmp/ttserver.pid`
To run the server by the RC script of the operating system, use `ttservctl'. As for most Linux distribution, append the following line to `/etc/rc.local'.
/usr/local/sbin/ttservctl start
By default, the database file and the related files are placed under `/var/ttserver'. Because `ttservctl' is a tiny shell script, copy and edit it for your purpose. Also, it is suitable to install the modified script into `/etc/init.d' and set symbolic links from `/etc/rc3.d/S98ttserver' and `/etc/rc5.d/S98ttserver'.
Let's run the server again to continue this tutorial.
[terminal-1]$ ttserver casket.tch
Store some records.
[terminal-2]$ tcrmgr put localhost one first [terminal-2]$ tcrmgr put localhost two second [terminal-2]$ tcrmgr put localhost three third
To back up the database file, indicate the destination path to the server by the command `tcrmgr copy'. Note that the backup file is created on the local file system of the server (not on the client side).
[terminal-2]$ tcrmgr copy localhost backup.tch
Terminate the server by Ctrl-C and remove the database.
[terminal-1]$ rm casket.tch
Recover the database from the backup file and restart the server.
[terminal-1]$ cp backup.tch casket.tch [terminal-1]$ ttserver casket.tch
Check consistency of stored records.
[terminal-2]$ tcrmgr mget localhost one two three
Terminate the server by Ctrl-C and remove the databases, for the successive tutorial.
[terminal-1]$ rm casket.tch backup.tch
Let's run the server with update logging enabled. The option `-ulog' specifies the directory to contain the update log files.
[terminal-1]$ mkdir ulog [terminal-1]$ ttserver -ulog ulog casket.tch
Store some records.
[terminal-2]$ tcrmgr put localhost one first [terminal-2]$ tcrmgr put localhost two second [terminal-2]$ tcrmgr put localhost three third
Terminate the server by Ctrl-C and remove the database.
[terminal-1]$ rm casket.tch
Escape the update log directoty and restart the server.
[terminal-1]$ mv ulog ulog-back [terminal-1]$ mkdir ulog [terminal-1]$ ttserver -ulog ulog casket.tch
Restore the database from the escaped update log, by the command `tcrmgr restore' on the client side.
[terminal-2]$ tcrmgr restore localhost ulog-back
Check consistency of stored records.
[terminal-2]$ tcrmgr mget localhost one two three
Terminate the server by Ctrl-C and remove the database, for the successive tutorial.
[terminal-1]$ rm -rf casket.tch ulog ulog-back
Replication is a mechanism to synchronize two or more database servers for high availability and high integrity. The replication source server is called "master" and each destination server is called "slave". Replication requires the following preconditions.
This section describes how to set up one master (at port 1978) and one slave (at port 1979) replication. First, let's run the master server.
[terminal-1]$ mkdir ulog-1 [terminal-1]$ ttserver -port 1978 -ulog ulog-1 -sid 1 casket-1.tch
Next, let's run the slave server in another terminal.
[terminal-2]$ mkdir ulog-2
[terminal-2]$ ttserver -port 1979 -ulog ulog-2 -sid 2 \
-mhost localhost -mport 1978 -rts 2.rts casket-2.tch
Store some records into the master.
[terminal-3]$ tcrmgr put -port 1978 localhost one first [terminal-3]$ tcrmgr put -port 1978 localhost two second [terminal-3]$ tcrmgr put -port 1978 localhost three third
Check consistency of stored records in the master and the slave.
[terminal-2]$ tcrmgr mget -port 1978 localhost one two three [terminal-2]$ tcrmgr mget -port 1979 localhost one two three
Let's simulate the case that the master is crashed. Terminate the master by Ctrl-C and remove the database file.
[terminal-1]$ rm casket-1.tch
Terminate the slave by Ctrl-C and restart it as the new master.
[terminal-2]$ ttserver -port 1979 -ulog ulog-2 -sid 2 casket-2.tch
Add the new slave (at port 1980).
[terminal-1]$ mkdir ulog-3
[terminal-1]$ ttserver -port 1980 -ulog ulog-3 -sid 3 \
-mhost localhost -mport 1979 -rts 3.rts casket-3.tch
Check consistency of stored records in the new master and the new slave.
[terminal-2]$ tcrmgr mget -port 1979 localhost one two three [terminal-2]$ tcrmgr mget -port 1980 localhost one two three
Terminate the two servers by Ctrl-C and remove the database and related files.
[terminal-1]$ rm -rf casket-1.tch ulog-1 1.rts [terminal-2]$ rm -rf casket-2.tch ulog-2 2.rts [terminal-1]$ rm -rf casket-3.tch ulog-3 3.rts
Tokyo Tyrant supports "dual master" replication which realizes higher availability. To do it, run two servers which replicate each other.
You can set replication of the running database service without any downtime. First, prepare the following script for backup operation and save it as "ttbackup.sh" with executable permission (0755).
#! /bin/sh srcpath="$1" destpath="$1.$2" rm -f "$destpath" cp -f "$srcpath" "$destpath"
Next, let's run the master with update log enabled.
[terminal-1]$ mkdir ulog-1 [terminal-1]$ ttserver -port 1978 -ulog ulog-1 -sid 1 casket-1.tch
Store a volume of records into the master.
[terminal-2]$ tcrtest write -port 1978 localhost 10000
Check consistency of stored records.
[terminal-2]$ tcrmgr list -port 1978 -pv localhost
Backup the database.
[terminal-2]$ tcrmgr copy -port 1978 localhost '@./ttbackup.sh'
Confirm that the backup file was saved as "casket-1.tch.xxxxx" ("xxxxx" stands for the time stamp of the backup file). Then, run the slave with the backup file.
[terminal-2]$ ls [terminal-2]$ cp casket-1.tch.xxxxx casket-2.tch [terminal-2]$ echo xxxxx > 2.rts [terminal-2]$ mkdir ulog-2 [terminal-2]$ ttserver -port 1979 -ulog ulog-2 -sid 2 -rts 2.rts casket-2.tch
Note that the above operation did not specify the master server to the slave. For tutorial, let's simulate that some records are stored into the master by users while you are setting replication.
[terminal-3]$ tcrmgr put -port 1978 localhost one first [terminal-3]$ tcrmgr put -port 1978 localhost two second [terminal-3]$ tcrmgr put -port 1978 localhost three third
Check the difference between the master and the slave.
[terminal-3]$ tcrmgr inform -port 1978 localhost [terminal-3]$ tcrmgr inform -port 1979 localhost
Specify the master to the slave so that replication will start and the difference will be resolved.
[terminal-3]$ tcrmgr setmst -port 1979 -mport 1978 localhost localhost
Confirm that the slave knows the master and the difference has been resolved.
[terminal-3]$ tcrmgr inform -port 1979 -st localhost
Terminate the two servers by Ctrl-C and remove the database and related files.
[terminal-1]$ rm -rf casket-1.tch casket-1.tch.* ulog-1 1.rts ttbackup.sh [terminal-2]$ rm -rf casket-2.tch ulog-2 2.rts
If you use a hash database, set the tuning parameter "#bnum=xxx" to improve performance. It specifies the bucket number and should be more than the number of record to be stored.
If you use a B+ tree database, set the tuning parameters "#lcnum=xxx#bnum=yyy" to improve performance. The former specifies the maximum number of leaf nodes to be cached. It should be larger as long as the capacity of RAM on the system allows. The latter specifies the bucket number and should be more than 1/128 of the number of records to be stored.
If huge number of clients access the server, make sure the limit number of file descriptors per a process is cleared. By default on most systems, it is set as 1024. If so, use `ulimit' to clear it.
This section describes how to use a memcached client library of Perl (Cache::Memcached) with Tokyo Tyrant. Run the server of Tokyo Tyrant as usual. And, the following script is a typical example.
use Cache::Memcached;
my $memd = Cache::Memcached->new();
$memd->set_servers(['localhost:1978']);
$memd->set('one', 'first');
$memd->set('two', 'second');
$memd->set('three', 'third');
my $val = $memd->get('one');
printf('one: %s\n', $val);
$val = $memd->get_multi('one', 'two', 'three');
printf("one: %s\n", $val->{one});
printf("two: %s\n", $val->{two});
printf("three: %s\n", $val->{three});
$memd->delete('one');
This section describes how to use an HTTP client library of Perl (LWP::UserAgent) with Tokyo Tyrant. Run the server of Tokyo Tyrant as usual. And, the following script is a typical example.
use LWP::UserAgent;
my $ua = LWP::UserAgent->new(keep_alive => 1);
my $baseurl = 'http://localhost:1978/';
my $req;
$req = HTTP::Request->new(PUT => $baseurl . 'one', [], 'first');
$ua->request($req);
$req = HTTP::Request->new(PUT => $baseurl . 'two', [], 'second');
$ua->request($req);
$req = HTTP::Request->new(PUT => $baseurl . 'three', [], 'third');
$ua->request($req);
$req = HTTP::Request->new(GET => $baseurl . 'one');
my $res = $ua->request($req);
if($res->is_success()){
printf("%s\n", $res->content());
}
$req = HTTP::Request->new(DELETE => $baseurl . 'one');
$ua->request($req);
Tokyo Tyrant is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 2.1 of the License or any later version.
Tokyo Tyrant is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License along with Tokyo Tyrant (See the file `COPYING'); if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA.
Tokyo Tyrant was written by Mikio Hirabayashi. You can contact the author by e-mail to `mikio@users.sourceforge.net'.