用java并发测试tokyo Cabinet的性能[五四陈手记]

03 Sep, 2009

Tokyo Cabinet 是小人平林幹雄开发的一款 DBM 数据库，该数据库读写非常快，哈希模式写入100万条数据只需0.643秒，读取100万条数据只需0.773秒，是 Berkeley DB 等 DBM 的几倍。

编译安装tokyocabinet数据库

wget http://tokyocabinet.sourceforge.net/tokyocabinet-1.4.28.tar.gz tar zxvf tokyocabinet-1.4.28.tar.gz
cd tokyocabinet-1.4.28/
./configure
make
make install
cd ../

http://tokyocabinet.sourceforge.net/javapkg/tokyocabinet-java-1.22.tar.gz

下载这个包

tar zxvf tokyocabinet-java-1.22.tar.gz
cd tokyocabinet-java-1.22
./configure
make
make install

install会将libjtokyocabinet.so 和 tokyocabinet.jar放到/usr/lib64下面。

将生成的tokyocabinet.jar拖到本地，新建项目，引用这个jar包。使用如下代码：

package test;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.concurrent.atomic.AtomicLong;

import tokyocabinet.*;

public class BenchMark {

public static class Stat {
private AtomicLong _count = new AtomicLong(0);
private static Stat _instance = new Stat();

public static Stat getInstance() {
return _instance;
}

private Stat() {
_printer = new RatePrinter(_count);
_printer.setDaemon(true);
_printer.start();
}

public void inc() {
_count.incrementAndGet();
}

private RatePrinter _printer;

private static class RatePrinter extends Thread {
private long _last;
private AtomicLong _c;

public RatePrinter(AtomicLong c) {
_c = c;
}

@Override
public void run() {
while (true) {
try {
long current = _c.get();
System.out.println(“Rate: " + (current - _last)
+ " req/s”);
_last = current;
Thread.sleep(1000L);
} catch (Throwable e) {
e.printStackTrace();
} }
} }
}

public static class EchoThread extends Thread {
private TDB tdb;
public EchoThread(String ip,String port, int in, ThreadGroup group){
super(group, “EchoThread-” + in);
// create the object
tdb = new TDB();

// open the database
if(!tdb.open(“casket.tct”, TDB.OWRITER | TDB.OCREAT)){
int ecode = tdb.ecode();
System.err.println(“open error: " + tdb.errmsg(ecode));
} }

@Override
public void run() {
int index = 0;
while (true) {
try {

String pkey = new Long(tdb.genuid()).toString();
Map cols = new HashMap();
cols.put(“name”, “mikio”+index);
cols.put(“age”, “30”);
cols.put(“lang”, “ja,en,c”);
if(!tdb.put(pkey, cols)){
int ecode = tdb.ecode();
//System.err.println(“put error: " + tdb.errmsg(ecode));
}

//client.insert(“Table1”, “name”+index, “Standard1:name”, (“name”+index).getBytes(“UTF-8”), System.currentTimeMillis(), true);
//client.get_column(“Table1”, “name0”, “Standard1:name”);
index ++;
Stat.getInstance().inc();
} catch (Throwable e) {
e.printStackTrace();
break;
} finally {
// close the database
if(!tdb.close()){
int ecode = tdb.ecode();
//System.err.println(“close error: " + tdb.errmsg(ecode));
}

}
} }
}

/**
* @param args
* @throws TTransportException
*/
public static void main(String[] args){
if (args.length != 1) {
System.out
.println(“Usage: Benchmark “);
System.exit(1);
}

String ip = args[0];
String port = args[0];

Integer concurrent = Integer.valueOf(args[0]);
System.out.println(“ip = “+ip+",port = “+port+",concurrent = " + concurrent);

ThreadGroup group = new ThreadGroup(“Benchmark”);
List
threads = new ArrayList
();
for (int x = 0; x < concurrent; ++x) {
Thread t =new EchoThread(ip,port, x, group);
threads.add(t);
t.start();
} }

} 运行结果反馈：

并发

随机结果

起始文件大小

运行中机器情况

Rate: 255153 req/s

517k

load average: 0.53, 0.39, 1.30

Rate: 505795 req/s - Rate: 909310 req/s

517k

load average: 3.89, 1.93, 1.65 Mem: 8195328k total, 8098204k used, 97124k free, 6685284k buffersSwap: 4000144k total, 148k used, 3999996k free, 36852k cached

Rate: 658089 req/s - Rate: 907266 req/s

517k

load average: 5.34, 2.30, 1.81 Mem: 8195328k total, 8112984k used, 82344k free, 6703212k buffersSwap: 4000144k total, 148k used, 3999996k free, 36800k cached

Rate: 694200 req/s - Rate: 1003093 req/s

517k

load average: 9.40, 6.71, 3.62 Mem: 8195328k total, 7423824k used, 771504k free, 6684268k buffersSwap: 4000144k total, 148k used, 3999996k free, 36784k cached

官方数据

70W req/s 基本上是一个的值了

原创文章如转载，请注明：转载自五四陈科学院[http://www.54chen.com]

Posted by 54chen java