concept protocol in category apache thrift

appears as: protocols, protocol, The protocol, protocol
Programmer's Guide to Apache Thrift

This is an excerpt from Manning's book Programmer's Guide to Apache Thrift.

  • Reach—Support for an impressive range of languages, protocols, and platforms
  • Protocols create and consume a single, central representation of logically equivalent structures across languages. Using Apache Thrift protocols, a C++ object can be serialized into a common format, sent to a Ruby program, and reconstituted as a Ruby object (see figure 5.2). Protocols also support communications between dissimilar hardware platforms where byte ordering, padding, and pointer width may vary. For example, Apache Thrift supports communicating between 32-bit and 64-bit C++ applications without incompatibilities. The Apache Thrift protocols standardize the representation of serialized objects independent of language, operating system, or hardware platform.

    Given the ease with which protocols can be swapped in and out, there’s little sense in not performing a direct comparison in cases where performance is critical. Listing 5.12 is a sample C++-based timing test built around our trivial struct serialization process. While this is the most generic of examples, it may help you develop intuition about protocol and transport interaction. The program allows you to select a transport, optional buffering layer, and a protocol. It then proceeds to write 1,000,000 Trade structs through the protocol, reporting write size and timing at the end of the process.

    Listing 5.12. ~/ThriftBook/part2/protocols/proto_write_times.cpp
    #include <iostream>
    #include <string>
    #include <chrono>
    #include <memory>
    #include <boost/shared_ptr.hpp>
    #include <thrift/transport/TSimpleFileTransport.h>
    #include <thrift/transport/TBufferTransports.h>
    #include <thrift/protocol/TCompactProtocol.h>
    #include <thrift/protocol/TBinaryProtocol.h>
    #include <thrift/protocol/TJSONProtocol.h>
    
    using namespace apache::thrift::transport;
    using namespace apache::thrift::protocol;
    
    struct Trade {
       char symbol[16];
       double price;
       int size;
    };
    
    int main(int argc, char *argv[]) {
        std::string usage("usage: " + std::string(argv[0]) +
            " (m[emory]|f[file]) (b[inary]|c[ompact]|j[son]) [b[uffering]]");
        if (argc != 3 && argc != 4) {
           std::cout << usage << std::endl;
           return -1;
        }
    
        boost::shared_ptr<TTransport> trans;           #1
        if (argv[1][0] == 'm' || argv[1][0] == 'M') {
           const int mem_size = 64*1024*1024;
           trans.reset(new TMemoryBuffer(mem_size));
           std::cout << "TMemoryBuffer(" << mem_size << ")/";
        }
        else if (argv[1][0] == 'f' || argv[1][0] == 'F') {
           const std::string path_name("/tmp/thrift_data");
           trans.reset(new TSimpleFileTransport(path_name, false, true));
           std::cout << "TSimpleFileTransport(" << path_name << ")/";
        }
        else {
           std::cout << usage << std::endl;
           return -1;
        }
    
        if (argc == 4 && (argv[3][0] == 'b' || argv[3][0] == 'B')) {    #2
           std::cout << "TBufferedTransport/";
           trans.reset(new TBufferedTransport(trans));
        }
        else if (argc == 4) {
           std::cout << usage << std::endl;
           return -1;
        }
    
        std::unique_ptr<TProtocol> proto;              #3
        if (argv[2][0] == 'b' || argv[2][0] == 'B') {
           std::cout << "TBinaryProtocol" << std::endl;
           proto.reset(new TBinaryProtocol(trans));
        }
        else if (argv[2][0] == 'c' || argv[2][0] == 'C') {
           std::cout << "TCompactProtocol" << std::endl;
           proto.reset(new TCompactProtocol(trans));
        }
        else if (argv[2][0] == 'j' || argv[2][0] == 'J') {
           std::cout << "TJSONProtocol" << std::endl;
           proto.reset(new TJSONProtocol(trans));
        }
        else {
            std::cout << usage << std::endl;
            return -1;
        }
    
        Trade trade;
        trade.symbol[0] = 'F'; trade.symbol[1] = '\0';
        trade.price = 13.10;
        trade.size = 2500;
    
        auto start = std::chrono::steady_clock::now();     #4
        int i = 0;
        for (int loop_count = 0; loop_count < 1000000; ++loop_count) {
            i += proto->writeStructBegin("Trade");
    
    
            i += proto->writeFieldBegin("symbol", T_STRING, 1);
            i += proto->writeString(std::string(trade.symbol));
            i += proto->writeFieldEnd();
    
            i += proto->writeFieldBegin("price", T_DOUBLE, 2);
            i += proto->writeDouble(trade.price);
            i += proto->writeFieldEnd();
    
            i += proto->writeFieldBegin("size", T_I32, 3);
            i += proto->writeI32(trade.size);
            i += proto->writeFieldEnd();
    
            i += proto->writeFieldStop();
            i += proto->writeStructEnd();
            proto->getTransport()->flush();                              #5
        }
        auto stop = std::chrono::steady_clock::now();      #6
        std::cout << "Bytes: " << i << ", seconds: "
                  << std::chrono::duration<double>(stop - start).count()
                  << std::endl;
    }
    
    
    !@%STYLE%@!
    {"css":"{\"css\": \"font-weight: bold;\"}","target":"[[{\"line\":28,\"ch\":51},{\"line\":28,\"ch\":53}],[{\"line\":44,\"ch\":68},{\"line\":44,\"ch\":70}],[{\"line\":53,\"ch\":51},{\"line\":53,\"ch\":53}],[{\"line\":76,\"ch\":55},{\"line\":76,\"ch\":57}],[{\"line\":96,\"ch\":69},{\"line\":96,\"ch\":71}],[{\"line\":98,\"ch\":55},{\"line\":98,\"ch\":57}]]"}
    !@%STYLE%@!

    Pointers to the abstract bases TTransport 1 and TProtocol 3 are used throughout the code, allowing us to select the type of transport and protocol at runtime. The user sets the transport and protocol via the command line.

    A buffering transport layer can be added optionally 2. This will have little effect on the memory transport, but it will make a big difference in the performance of the disk-based transport. The buffering layer will aggregate the many small data writes made by the protocol and, when flush is called 5, write a single larger block to disk. This will improve disk-based write performance on most platforms by reducing the number of system calls by an order of magnitude.