54 CHEN

Cassandra的thrift用法学习手记

thrift __English Version__The notes about the usage of Thrift in Cassandra

Cassandra在client访问server cluster的时候使用了thrift,在cluster node间的通讯,依旧是自己实现的二进制协议。

先决条件 thrift 0.9.1
mac
libthrift-0.9.1
Cassandra 2.0.3

本文所涉及的代码:https://github.com/54chen/cassandra-thrift

看代码 thrift的定义:server的逻辑实现代码叫做Processor,创建的等待socket代码里叫做Transport,最后启动的进程叫做Server,大致就会有要启动一个thrift server,需要有一个socket(Transport)和一堆逻辑(Processor)。

这里有一篇2011年写的thrift入门手记:http://www.54chen.com/java-ee/thrift-quick-start.html

示例代码中以TNonBlockingServer来举例。TNonBlockingServer是采用libevent lib实现的一种thrift内置的server类型,理论上是最快的一种。

CustomTNonBlockingServer Cassandra并没有默认使用这种类型的server,默认的是基于TServerTransport连接完成的CustomTThreadPoolServer(自行实现了连接池)。

以示例代码为例:
CassandraTestServer -> CustomTNonBlockingServer.Factory().buildTServer -> new CustomTNonBlockingServer -> new TCustomNonblockingServerSocket.

这个调用传递中,在接入客户端连接时,可以插入一定的代码(比如说来来保存session之类的)。

代码中被插入的代码和阶段有:
TCustomNonblockingServerSocket->acceptImpl 一次客户端请求进入时,可用来设置服务器端的超时设置之类的数值。
CustomTNonBlockingServer->requestInvoke 紧接着上面的执行后具体的调用会打到这里来,这里看上去Cassandra把一个父类强转为子类,为了取当时的socket信息,有点小bug。

其他server Cassandra默认的CustomTThreadPoolServer也有类似前文提到的插入代码的地方,主要目的也是超时设置buffer设置等,而且因为使用的是普通TServerTransport,设置的点更加灵活。
不得不提的是另一个的实现,THsHaDisruptorServer,大有来头。
THsHaDisruptorServer使用TCustomNonblockingServerSocket当socket,上层TDisruptorServer使用的是基于LMAX(一家牛B的金融平台公司)开源的Disruptor(一个巨牛B的高吞吐低延迟的并发处理机制)来实现的TDisruptorServer,TDisruptorServer的代码在:https://github.com/xedin/disruptor_thrift_server 可以理解为一个经过队列改造的TNonBlockingServer。

以后再找机会分析这个disruptor_thrift_server。
__EOF__

__English Version__ Cassandra uses thrift when the client visits the server, but it still uses binary protocol between the cluster nodes.

Prerequisite thrift 0.9.1
mac
libthrift-0.9.1
Cassandra 2.0.3

The codes mentioned in this blog:https://github.com/54chen/cassandra-thrift

See Codes The definition of Thrift: The codes which implements the server is named Processor. The codes which implements the socket for waiting is named Transport. At last, the codes which starts all logic is named Server. All of this, we need a thrift server,a socket(Transport) and a Processor.

This is a blog link about thrift written in 2011:http://www.54chen.com/java-ee/thrift-quick-start.html

The example codes take TNonBlockingServer for example.TNonBlockingServer is a built-in type server designed by lib-event.It is the fast type in theory.

CustomTNonBlockingServer Cassandra does not use this type by default.The default type is based on TServerTransport named CustomTThreadPoolServer(implements the thread pool).

For example:
CassandraTestServer -> CustomTNonBlockingServer.Factory().buildTServer -> new CustomTNonBlockingServer -> new TCustomNonblockingServerSocket.

In this link,when client is come,we can plug in some codes(for saving session, etc.)。

the codes and the stages which plugged in:
TCustomNonblockingServerSocket->acceptImpl When the client is comming,we can set the configuration like the value of timeout,etc.
CustomTNonBlockingServer->requestInvoke And then it will be called here.It seems that Cassandra’s codes cast a parent class to children one.It is maybe a bug。

Else Server Cassandra’s default server CustomTThreadPoolServer is also can be plugged in like above talked.The plugin main target is set timeout,buffer,etc.And the TServerTransport it used is more flexible to setting values。

It’s hard to ignore the another type,THsHaDisruptorServer,is not normaI。
THsHaDisruptorServer is use TCustomNonblockingServerSocket to socket.The TDisruptorServer is based on LMAX Disruptor.The TDisruptorServer’s codes is here:https://github.com/xedin/disruptor_thrift_server Can be understood as,a queue-improved TNonBlockingServer。

I’ll write a blog about disruptor_thrift_server later.
__EOF__

Happy New Year!

原创文章如转载,请注明:转载自五四陈科学院[http://www.54chen.com]

Posted by 54chen java

« BLUNO试用手记 disruptor thrift server连接参数与rps数值影响记录 »