<?xml version="1.0" encoding="UTF-8"?>
<appendix xml:id="tracing"
      version="5.0" xmlns="http://docbook.org/ns/docbook"
      xmlns:xlink="http://www.w3.org/1999/xlink"
      xmlns:xi="http://www.w3.org/2001/XInclude"
      xmlns:svg="http://www.w3.org/2000/svg"
      xmlns:m="http://www.w3.org/1998/Math/MathML"
      xmlns:html="http://www.w3.org/1999/xhtml"
      xmlns:db="http://docbook.org/ns/docbook">
  <!--/**
 * Licensed to the Apache Software Foundation (ASF) under one
 * or more contributor license agreements.  See the NOTICE file
 * distributed with this work for additional information
 * regarding copyright ownership.  The ASF licenses this file
 * to you under the Apache License, Version 2.0 (the
 * "License"); you may not use this file except in compliance
 * with the License.  You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
-->
  <title>Enabling Dapper-like Tracing in HBase</title>

  <para>
    <link xlink:href="https://issues.apache.org/jira/browse/HBASE-6449">HBASE-6449</link>
    added support for tracing requests through HBase, using the open source tracing library,
    <link xlink:href="http://github.com/cloudera/htrace">HTrace</link>.
    Setting up tracing is quite simple,
    however it currently requires some very minor changes to your client code
    (it would not be very difficult to remove this requirement).
  </para>

  <section xml:id="tracing.spanreceivers">
    <title>SpanReceivers</title>
    <para>
      The tracing system works by collecting information in structs called 'Spans'.
      It is up to you to choose how you want to receive this information
      by implementing the <classname>SpanReceiver</classname> interface,
      which defines one method:
<programlisting><![CDATA[
  public void receiveSpan(Span span);
]]></programlisting>
      This method serves as a callback whenever a span is completed.
      HTrace allows you to use as many SpanReceivers as you want
      so you can easily send trace information to multiple destinations.
    </para>

    <para>
      Configure what SpanReceivers you'd like to us
      by putting a comma separated list of the
      fully-qualified class name of classes implementing
      <classname>SpanReceiver</classname> in <filename>hbase-site.xml</filename>
      property: <varname>hbase.trace.spanreceiver.classes</varname>.
    </para>

    <para>
      HTrace includes a <classname>LocalFileSpanReceiver</classname>
      that writes all span information to local files in a JSON-based format.
      The <classname>LocalFileSpanReceiver</classname>
      looks in <filename>hbase-site.xml</filename>
      for a <varname>hbase.local-file-span-receiver.path</varname>
      property with a value describing the name of the file
      to which nodes should write their span information.
<programlisting><![CDATA[
  <property>
    <name>hbase.trace.spanreceiver.classes</name>
    <value>org.htrace.impl.LocalFileSpanReceiver</value>
  </property>
  <property>
    <name>hbase.local-file-span-receiver.path</name>
    <value>/var/log/hbase/htrace.out</value>
  </property>
]]></programlisting>
    </para>

    <para>
      HTrace also includes a <classname>ZipkinSpanReceiver</classname>
      that converts all span information to
      <link xlink:href="http://github.com/twitter/zipkin">Zipkin</link>
      span format and send them to Zipkin server.
      You need to install htrace-zipkin jar and add it to your HBase classpath
      in order to use this receiver.
      The <classname>ZipkinSpanReceiver</classname>
      looks in <filename>hbase-site.xml</filename>
      for a <varname>hbase.zipkin.collector-hostname</varname>
      and <varname>hbase.zipkin.collector-port</varname>
      property with a value describing the Zipkin server
      to which span information are sent.
<programlisting><![CDATA[
  <property>
    <name>hbase.trace.spanreceiver.classes</name>
    <value>org.htrace.impl.ZipkinSpanReceiver</value>
  </property> 
  <property>
    <name>hbase.zipkin.collector-hostname</name>
    <value>localhost</value>
  </property> 
  <property>
    <name>hbase.zipkin.collector-port</name>
    <value>9410</value>
  </property> 
]]></programlisting>
    </para>

    <para>
      If you do not want to use the included span receivers,
      you are encouraged to write your own receiver
      (take a look at <classname>LocalFileSpanReceiver</classname> for an example).
      If you think others would benefit from your receiver,
      file a JIRA or send a pull request to
      <link xlink:href="http://github.com/cloudera/htrace">HTrace</link>.
    </para>
  </section>

  <section xml:id="tracing.client.modifications">
    <title>Client Modifications</title>
    <para>
      In order to turn on tracing in your client code,
      you must initialize the module sending spans to receiver
      once per client process.
      (Because <classname>SpanReceiverHost</classname> is included in hbase-server jar,
      you need it on the client classpath in order to run this example.)
<programlisting><![CDATA[
  private SpanReceiverHost spanReceiverHost;
  
  ...
  
    Configuration conf = HBaseConfiguration.create();
    SpanReceiverHost spanReceiverHost = SpanReceiverHost.getInstance(conf);
]]></programlisting>
      Then you simply start tracing span before requests you think are interesting,
      and close it when the request is done.
      For example, if you wanted to trace all of your get operations,
      you change this:
<programlisting><![CDATA[
  HTable table = new HTable(conf, "t1");
  Get get = new Get(Bytes.toBytes("r1"));
  Result res = table.get(get);
]]></programlisting>
      into:
<programlisting><![CDATA[
  TraceScope ts = Trace.startSpan("Gets", Sampler.ALWAYS);
  try {
    HTable table = new HTable(conf, "t1");
    Get get = new Get(Bytes.toBytes("r1"));
    Result res = table.get(get);
  } finally {
    ts.close();
  }
]]></programlisting>
      If you wanted to trace half of your 'get' operations, you would pass in:
<programlisting><![CDATA[
  new ProbabilitySampler(0.5)
]]></programlisting>
      in lieu of <varname>Sampler.ALWAYS</varname>
      to <classname>Trace.startSpan()</classname>.
      See the HTrace <filename>README</filename> for more information on Samplers.
    </para>
  </section>

  <section xml:id="tracing.client.shell">
    <title>Tracing from HBase Shell</title>
    <para>
      You can use <command>trace</command> command
      for tracing requests from HBase Shell.
      <command>trace 'start'</command> command turns on tracing and 
      <command>trace 'stop'</command> command turns off tracing.
<programlisting><![CDATA[
  hbase(main):001:0> trace 'start'
  hbase(main):002:0> put 'test', 'row1', 'f:', 'val1'   # traced commands
  hbase(main):003:0> trace 'stop'
]]></programlisting>
    </para>
    <para>
      <command>trace 'start'</command> and 
      <command>trace 'stop'</command> always
      returns boolean value representing 
      if or not there is ongoing tracing.
      As a result, <command>trace 'stop'</command>
      returns false on suceess.
      <command>trace 'status'</command>
      just returns if or not tracing is turned on.
<programlisting><![CDATA[
  hbase(main):001:0> trace 'start'
  => true
  
  hbase(main):002:0> trace 'status'
  => true
  
  hbase(main):003:0> trace 'stop'
  => false
  
  hbase(main):004:0> trace 'status'
  => false
]]></programlisting>
    </para>
  </section>

</appendix>
