concept .META in category hbase

This is an excerpt from Manning's book Hbase in Action. Login to get full access to this book.

Figure 3.9. -ROOT-, .META., and user tables viewed as a B+Tree

Two special tables in HBase, -ROOT- and .META., help find where regions for various tables are hosted. Like all tables in HBase, -ROOT- and .META. are also split into regions. -ROOT- and .META. are both special tables, but -ROOT- is more special than .META.; -ROOT- never splits into more than one region. .META. behaves like all other tables and can split into as many regions as required.

When a client application wants to access a particular row, it goes to the -ROOT- table and asks it where it can find the region responsible for that particular row. -ROOT- points it to the region of the .META. table that contains the answer to that question. The .META. table consists of entries that the client application uses to determine which RegionServer is hosting the region in question. Think of this like a distributed B+Tree of height 3 (see figure 3.9). The -ROOT- table is the -ROOT- node of the B+Tree. The .META. regions are the children of the -ROOT- node (-ROOT-region)n and the regions of the user tables (leaf nodes of the B+Tree) are the children of the .META. regions.

Figure 3.9. -ROOT-, .META., and user tables viewed as a B+Tree

Let’s put -ROOT- and .META. into the example; see figure 3.10. Note that the region assignments shown here are arbitrary and don’t represent how they will happen when such a system is deployed.

Figure 3.10. User table T1 in HBase, along with -ROOT- and .META., distributed across the various RegionServers

The client automatically handles communicating with ZooKeeper and finding the relevant RegionServer with which to interact. Let’s still examine the -ROOT- and .META. tables to get a better understanding of what information they contain and how the client uses that information.

The previous entry in the -ROOT- table also tells you which server the .META. region is hosted on. In this case, because it’s a standalone instance and everything is on local-host, that column contains the value localhost:port. There is also a column for regioninfo that contains the name of the region, start key, end key, and encoded name. (The encoded name is used internally in the system and isn’t of any consequence to you.) The HBase client library uses all this information to locate the correct region to talk to while performing the operations in your application code. When no other table exists in the system, .META. looks like the following:

hbase(main):030:0> scan '.META.'
ROW              COLUMN+CELL
0 row(s) in 5.4180 seconds

copy

Notice that there are no entries in the .META. table. This is because there are no user-defined tables in this HBase instance. On instantiating the users table, .META. looks like the following:

hbase(main):030:0> scan '.META.'


 ROW                      COLUMN+CELL

 users,,1335466383956.4a1 column=info:regioninfo,
 5eba38d58db711e1c7693581 timestamp=1335466384006, value={NAME =>
 af7f1.                   'users,,1335466383956.4a15eba38d58
                          db711e1c7693581af7f1.', STARTKEY => '',
                          ENDKEY => '', ENCODED =>
                          4a15eba38d58db711e1c7693581af7f1,}

 users,,1335466383956.4a1 column=info:server, timestamp=1335466384045,
 5eba38d58db711e1c7693581 value=localhost:58269
 af7f1.

 users,,1335466383956.4a1 column=info:serverstartcode,
 5eba38d58db711e1c7693581 timestamp=1335466384045, value=1335465653436
 af7f1.

 1 row(s) in 0.4540 seconds

copy

In your current setup, that’s probably what it looks like. If you want to play around a bit, you can disable and delete the users table, examine .META., and re-create and repopulate the users table. Disabling and deletion can be done in the HBase shell just like creating a table.

Similar to what you saw in -ROOT-, .META. contains information about the users table and other tables in your system. Information for all the tables you instantiate goes here. The structure of .META. is similar to that of -ROOT-.

Once you’ve examined the .META. table after instantiating the TwitBase users table, populate some users into the system using the LoadUsers command provided in the sample application code. When the users table outgrows a single region, that region will split. You can also split the table manually for experimentation right now:

hbase(main):030:0> split 'users' 0 row(s) in 6.1650 seconds hbase(main):030:0> scan '.META.' ROW COLUMN+CELL users,,1335466383956.4a15eba column=info:regioninfo, 38d58db711e1c7693581af7f1. timestamp=1335466889942, value={NAME => 'users,,1335466383956.4a15eba38d58db711e 1c7693581af7f1.', STARTKEY => '', ENDKEY => '', ENCODED => 4a15eba38d58db711e1c7693581af7f1, OFFLINE => true, SPLIT => true,} users,,1335466383956.4a15eba column=info:server, 38d58db711e1c7693581af7f1. timestamp=1335466384045, value=localhost:58269 users,,1335466383956.4a15eba column=info:serverstartcode, 38d58db711e1c7693581af7f1. timestamp=1335466384045, value=1335465653436 users,,1335466383956.4a15eba column=info:splitA, 38d58db711e1c7693581af7f1. timestamp=1335466889942, value= {NAME => 'users,,1335466889926.9fd558ed44a63f016 c0a99c4cf141eb5.', STARTKEY => '', ENDKEY => '}7\ x8E\xC3\xD1\xE3\x0F\x0D\xE9\xFE'fIK\xB7\ xD6', ENCODED => 9fd558ed44a63f016c0a99c4cf141eb5,} users,,1335466383956.4a15eba column=info:splitB, 38d58db711e1c7693581af7f1. timestamp=1335466889942, value={NAME => 'users,}7\x8E\xC3\xD1\xE3\x0F\ x0D\xE9\xFE'fIK\xB7\xD6,1335466889926.a3 c3a9162eeeb8abc0358e9e31b892e6.', STARTKEY => '}7\x8E\ xC3\xD1\xE3\x0F\x0D\xE9\xFE'fIK\xB7\xD6' , ENDKEY => '', ENCODED => a3c3a9162eeeb8abc0358 e9e31b892e6,} users,,1335466889926.9fd558e column=info:regioninfo, d44a63f016c0a99c4cf141eb5. timestamp=1335466889968, value={NAME => 'users,,1335466889926.9fd558ed44a63f016c 0a99c4cf141eb5.', STARTKEY => '', ENDKEY => '}7\x8E\xC3\ xD1\xE3\x0F\x0D\xE9\xFE'fIK\xB7\xD6', ENCODED => 9fd558ed44a63f016c0a99c4cf141eb5,} users,,1335466889926.9fd558e column=info:server, d44a63f016c0a99c4cf141eb5. timestamp=1335466889968, value=localhost:58269 users,,1335466889926.9fd558e column=info:serverstartcode, d44a63f016c0a99c4cf141eb5. timestamp=1335466889968, value=1335465653436 users,}7\x8E\xC3\xD1\xE3\x0F column=info:regioninfo, \x0D\xE9\xFE'fIK\xB7\xD6,133 timestamp=1335466889966, value={NAME => 5466889926.a3c3a9162eeeb8abc 'users,}7\x8E\xC3\xD1\xE3\x0F\x0D\xE9\xF 0358e9e31b892e6. E'fIK\xB7\xD6,1335466889926.a3c3a9162eee b8abc0358e9e31b892e6.', STARTKEY => '}7\x8E\xC3\xD1\xE3\x0F\ x0D\xE9\xFE'fIK\xB7\xD6', ENDKEY => '', ENCODED => a3c3a9162eeeb8abc0358e9e31b892e6,} users,}7\x8E\xC3\xD1\xE3\x0F column=info:server, \x0D\xE9\xFE'fIK\xB7\xD6,133 timestamp=1335466889966, 5466889926.a3c3a9162eeeb8abc value=localhost:58269 0358e9e31b892e6. users,}7\x8E\xC3\xD1\xE3\x0F column=info:serverstartcode, \x0D\xE9\xFE'fIK\xB7\xD6,133 timestamp=1335466889966, 5466889926.a3c3a9162eeeb8abc value=1335465653436 0358e9e31b892e6.

3 row(s) in 0.5660 seconds

After you split users, new entries are made in the .META. table for the daughter regions. These daughter regions replace the parent region that was split. The entry for the parent region contains the information about the splits, and requests are no longer served by the parent region. For a brief time, the information for the parent region remains in .META. Once the hosting RegionServer has completed the split and cleaned up the parent region, the entry is deleted from .META.. After the split is complete, .META. looks like this: