concept user in category hadoop

This is an excerpt from Manning's book Hadoop in Practice, Second Edition.
MapReduce is a batch-based, distributed computing framework modeled after Google’s paper on MapReduce.[5] It allows you to parallelize work over a large amount of raw data, such as combining web logs with relational data from an OLTP database to model how users interact with your website. This type of work, which could take days or longer using conventional serial programming techniques, can be reduced to minutes using MapReduce on a Hadoop cluster.
In our example there’s a good chance that the users that appear in your logs are a small percentage of the overall set of users in your OLTP database, so by removing all the OLTP users that don’t appear in your logs, you could get the dataset down to a size that fits into memory. If this is the case, a semi-join is the solution. Figure 6.5 shows the three MapReduce jobs you need to execute to perform a semi-join.
The second step is an elaborate filtering MapReduce job, where the goal is to remove users from the user dataset that don’t exist in the log data. This is a map-only job that uses a replicated join to cache the user names that appear in the log files and join them with the user dataset. The unique user output from job 1 will be substantially smaller than the entire user dataset, which makes it the natural selection for caching. Figure 6.7 shows the flow of this job.
The partitioner should only partition based on the user name, so that all the records for a user arrive at the same reducer. Sorting should use both the user name and dataset indicator, so that the smaller dataset is ordered first (by virtue of the fact that the USERS constant is a smaller number than the USER_LOGS constant, resulting in the user records being sorted before the user logs). The grouping should group on users so that both datasets are streamed to the same reducer invocation: ShuffleUtils.configBuilder() .setPartitionerIndices(KeyFields.USER) .setSortIndices(KeyFields.USER, KeyFields.DATASET) .setGroupIndices(KeyFields.USER) .configure(job.getConfiguration());ShuffleUtils.configBuilder() .setPartitionerIndices(KeyFields.USER) .setSortIndices(KeyFields.USER, KeyFields.DATASET) .setGroupIndices(KeyFields.USER) .configure(job.getConfiguration());Finally, you’ll modify the reducer to cache the incoming user records, and then join them with the user logs:[12]

This is an excerpt from Manning's book Hadoop in Action.
Simple— Hadoop allows users to quickly write efficient parallel code.
Even though the operations are relational in style, Pig Latin remains a data flow language. A data flow language is friendlier to programmers who think in terms of algorithms, which are more naturally expressed by the data and control flows. On the other hand, a declarative language such as SQL is sometimes easier for analysts who prefer to just state the results one expects from a program. Hive is a different Hadoop subproject that targets users who prefer the SQL model. We’ll learn about Hive in detail in chapter 11.
SELECT user, COUNT(*) FROM excite-small.log GROUP BY user;