(foremost)

(role)
(that)
(mobile)
(phones)
(play)
(in)
(our)
(lives)
(is)
(that)
(they)
(provide)
(us)
(an)
(easy)
(and)
(fast)
(way)
(of)
(communication.)
(Go)
(wherever)
(you)
(want)
(whenever)
(you)
(want;)
(your)
(mobile)
(phone)
(will)
(stay)
(you)
(connected)
(with)
(all.)
(Got)
(an)
(important)
(message)
(for)
(your)
(friend/family)
(member?)
(Just)
(send)
(instant)
(SMS)
(or)
(call)
(straight)
(away)
(and)
(make)
(your)
(important)
(message)
(reach)
(the)

(intended)
(person)
(in)
(no)
(time.)
(Simple!)
()
(Internet)
(Access)
(&)
(Social)
(Media)
()
(The)
(Internet)
(is)
(one)
(of)
(the)
(biggest)
(blessings)
(to)
(man)
(by)
(technology.)
(One)
(just)
(cannot)
(imagine)
(the)
(life)
(without)
(the)
(Internet.)
(Everyone)
(likes/wants)
(to)
(stay)
(connected)
(with)
(the)
(Internet)
(all)
(the)
(time.)
(Thanks)
(to)
(the)
(mobile)
(phones)
(that)
(make)
(it)
(possible.)
(The)
(mobile)
(phones)
(let)
(users)
(enjoy)

(the)
(social)
(media)
(on)
(the)
(go.)
(A)
(major)
(part)
(of)
(modern)
(world)
(is)
(addicted)
(to)
(social)
(networking)
(sites.)
(Mobile)
(Phones)
(let)
(them)
(be)
(active)
(in)
(social)
(media)
(on)
(the)
(move.)
()
(Mobile)
(Phones)
(Save)
(Money)
()
(Yes)
(they)
(do.)
(Just)
(buy)
(mid-range)
(or)
(high-end)
(mobile)
(phone)
(and)
(ultimately)
(you)
(become)
(proud)
(owner)
(of)
(many)
(gadgets)
(the)
(gadgets)
(that)
(are)
(packed)

(in)
(the)
(mobile)
(phone.)
(For)
(example)
(by)
(buying)
(mobile)
(phone)
(you)
(don t)
(need)
(to)
(spend)
(money)
(on)
(a)
(home)
(phone)
(you)
(don t)
(need)
(to)
(buy)
(any)
(music)
(or)
(video)
(player)
(you)
(don t)
(need)
(to)
(buy)
(radio)
(etc.)
(If)
(the)
(mobile)
(phone)
(is)
(high-end)
(then)
(you)
(need)
(not)
(to)
(buy)
(gaming)
(console)
(and)
(maybe)
(a)
(computer)
(as)
(well.)
()
(Mobile)
(Phones)

(Ensure)
(Safety)
()
(The)
(mobile)
(phones)
(provide)
(us)
(security.)
(How?)
(Well)
(to)
(understand)
(this)
(better)
(just)
(consider)
(any)
(situation)
(in)
(which)
(you)
(urgently)
(need)
(to)
(contact)
(fire)
(brigade)
(or)
(police.)
(What)
(would)
(you)
(do)
(in)
(such)
(case?)
(The)
(landlines)
(are)
(not)
(available)
(everywhere)
(but)
(the)
(mobile)
(phones)
(do.)
(Just)
(take)
(your)
(mobile)
(phone)
(out)
(dial)
(the)
(number)
(and)
(that s)
(it.)

(Your)
(one)
(good)
(step)
(of)
(buying)
(mobile)
(phone)
(will)
(prove)
(as)
(great)
(money)
(saver)
(for)
(you.)
()
(Mobile)
(Phones)
(Helps)
(in)
(Business)
()
(Got)
(an)
(important)
(message)
(for)
(your)
(client)
(or)
(employee?)
(Pick)
(up)
(your)
(mobile)
(phone)
(and)
(communicate)
(with)
(the)
(intended)
(person)
(straight)
(away.)
(Since)
(businesses)
(require)
(constant)
(communication)
(so)
(one)
(just)
(can t)
(imagine)
(any)
(business)
(without)
(the)
(use)

(of)
(mobile)
(phones.)
(And)
(with)
(all)
(those)
(new)
(smart)
(phones)
(available)
(in)
(market)
(these)
(days)
(a)
(businessman)
(can)
(organize)
(the)
(schedule)
(and)
(set)
(reminders)
(so)
(he)
(does)
(not)
(forget)
(any)
(important)
(meeting.)
(Mobile)
(phones)
(come)
(packed)
(with)
(lot)
(of)
(business)
(apps)
(which)
(makes)
(their)
(life)
(easier.)
(Thus)
(mobile)
(phones)
(are)
(like)
(a)
(boon)
(for)
(businesses.)
()
(So)
(folks!)
(These)
(are)

(the)
(main)
(benefits)
(of)
(having)
(mobile)
(phones)
(in)
(our)
(life.)
(Now)
(it s)
(your)
(turn)
(tell)
(us)
(why)
(Mobile)
(phone)
(is)
(important)
(for)
(you)
(in)
(the)
(comments)
(below.)
()
(More:)
(Positive)
(&)
(negative)
(impact)
(of)
(mobile)
()
()
()
(Read)
(more:)
(http://myessaypoint.com/importance-of-mobile-phone-technology#ixzz43FAnUmFx)
()
grunt>
grunt>
grunt>
grunt>
grunt>
grunt>
grunt>
grunt>
grunt> group1 = group words1 by $0;
2016-03-19 07:02:57,041 [main] WARN org.apache.pig.newplan.BaseOperatorPlan ncountered Warning USING_OVERLOADED_FUNCTION 1 time(s).
2016-03-19 07:02:57,041 [main] WARN org.apache.pig.newplan.BaseOperatorPlan ncountered Warning IMPLICIT_CAST_TO_CHARARRAY 1 time(s).
grunt> dump group1
2016-03-19 07:03:07,786 [main] WARN org.apache.pig.newplan.BaseOperatorPlan ncountered Warning USING_OVERLOADED_FUNCTION 1 time(s).
2016-03-19 07:03:07,786 [main] WARN org.apache.pig.newplan.BaseOperatorPlan ncountered Warning IMPLICIT_CAST_TO_CHARARRAY 1 time(s).

E
E
E
E

2016-03-19 07:03:07,787 [main] INFO org.apache.pig.tools.pigstats.ScriptState Pig features used in the script: GROUP_BY
2016-03-19 07:03:07,809 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-03-19 07:03:07,809 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2016-03-19 07:03:07,809 [main] INFO org.apache.pig.data.SchemaTupleBackend - Ke
y [pig.schematuple] was not set... will not generate code.
2016-03-19 07:03:07,809 [main] INFO org.apache.pig.newplan.logical.optimizer.Lo
gicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, ConstantCalc
ulator, GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeF
ilter, MergeForEach, PartitionFilterOptimizer, PredicatePushdownOptimizer, PushD
ownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter]}
2016-03-19 07:03:07,813 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? fal
se
2016-03-19 07:03:07,815 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
2016-03-19 07:03:07,815 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
2016-03-19 07:03:07,823 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-03-19 07:03:07,823 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2016-03-19 07:03:07,824 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 07:03:07,827 [main] INFO org.apache.pig.tools.pigstats.mapreduce.MRS
criptState - Pig script settings are added to the job
2016-03-19 07:03:07,827 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percen
t is not set, set to default 0.3
2016-03-19 07:03:07,828 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - Reduce phase detected, estimating # of re
quired reducers.
2016-03-19 07:03:07,828 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - Using reducer estimator: org.apache.pig.b
ackend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
2016-03-19 07:03:07,828 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.InputSizeReducerEstimator - BytesPerReducer=1000000000 maxRedu
cers=999 totalInputFileSize=3386
2016-03-19 07:03:07,828 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - Setting Parallelism to 1
2016-03-19 07:03:07,852 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - Setting up single store job
2016-03-19 07:03:07,854 [main] INFO org.apache.pig.data.SchemaTupleFrontend - K
ey [pig.schematuple] is false, will not generate code.
2016-03-19 07:03:07,854 [main] INFO org.apache.pig.data.SchemaTupleFrontend - S
tarting process to move generated code to distributed cacche
2016-03-19 07:03:07,854 [main] INFO org.apache.pig.data.SchemaTupleFrontend - D
istributed cache not supported or needed in local mode. Setting key [pig.schemat
uple.local.dir] with code temp directory: /tmp/1458370987854-0
2016-03-19 07:03:07,885 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission
.
2016-03-19 07:03:07,888 [JobControl] INFO org.apache.hadoop.metrics.jvm.JvmMetr
ics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - al
ready initialized
2016-03-19 07:03:07,894 [JobControl] WARN org.apache.hadoop.mapreduce.JobResour
ceUploader - No job jar file set. User classes may not be found. See Job or Job

#setJar(String).
2016-03-19 07:03:07,903 [JobControl] INFO org.apache.hadoop.mapreduce.lib.input
.FileInputFormat - Total input paths to process : 1
2016-03-19 07:03:07,905 [JobControl] INFO org.apache.pig.backend.hadoop.executi
onengine.util.MapRedUtil - Total input paths to process : 1
2016-03-19 07:03:07,905 [JobControl] INFO org.apache.pig.backend.hadoop.executi
onengine.util.MapRedUtil - Total input paths (combined) to process : 1
2016-03-19 07:03:07,907 [JobControl] INFO org.apache.hadoop.mapreduce.JobSubmit
ter - number of splits:1
2016-03-19 07:03:07,941 [JobControl] INFO org.apache.hadoop.mapreduce.JobSubmit
ter - Submitting tokens for job: job_local1118178464_0005
2016-03-19 07:03:08,030 [JobControl] INFO org.apache.hadoop.mapreduce.Job - The
url to track the job: http://localhost:8080/
2016-03-19 07:03:08,031 [Thread-27] INFO org.apache.hadoop.mapred.LocalJobRunne
r - OutputCommitter set in config null
2016-03-19 07:03:08,036 [Thread-27] INFO org.apache.hadoop.conf.Configuration.d
eprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-03-19 07:03:08,036 [Thread-27] INFO org.apache.hadoop.conf.Configuration.d
eprecation - mapred.job.reduce.markreset.buffer.percent is deprecated. Instead,
use mapreduce.reduce.markreset.buffer.percent
2016-03-19 07:03:08,036 [Thread-27] INFO org.apache.hadoop.conf.Configuration.d
eprecation - mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduc
es
2016-03-19 07:03:08,036 [Thread-27] INFO org.apache.hadoop.conf.Configuration.d
eprecation - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker
.address
2016-03-19 07:03:08,036 [Thread-27] INFO org.apache.hadoop.conf.Configuration.d
eprecation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-che
cksum
2016-03-19 07:03:08,036 [Thread-27] INFO org.apache.hadoop.mapreduce.lib.output
.FileOutputCommitter - File Output Committer Algorithm version is 1
2016-03-19 07:03:08,037 [Thread-27] INFO org.apache.hadoop.mapred.LocalJobRunne
r - OutputCommitter is org.apache.pig.backend.hadoop.executionengine.mapReduceLa
yer.PigOutputCommitter
2016-03-19 07:03:08,039 [Thread-27] INFO org.apache.hadoop.mapred.LocalJobRunne
r - Waiting for map tasks
2016-03-19 07:03:08,039 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner - Starting task: attempt_local1118178464_0005_m_0000
00_0
2016-03-19 07:03:08,046 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapreduce.lib.output.FileOutputCommitter - File Output Committer Algorithm
version is 1
2016-03-19 07:03:08,046 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.Task - Using ResourceCalculatorProcessTree : [ ]
2016-03-19 07:03:08,048 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Processing split: Number of splits :1
Total Length = 3386
Input split[0]:
Length = 3386
ClassName: org.apache.hadoop.mapreduce.lib.input.FileSplit
Locations:
----------------------2016-03-19 07:03:08,050 [LocalJobRunner Map Task Executor #0] INFO
ig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader being processed file:/root/essay:0+3386
2016-03-19 07:03:08,124 [LocalJobRunner Map Task Executor #0] INFO
adoop.mapred.MapTask - (EQUATOR) 0 kvi 26214396(104857584)
2016-03-19 07:03:08,124 [LocalJobRunner Map Task Executor #0] INFO

org.apache.p
Current split
org.apache.h
org.apache.h

adoop.mapred.MapTask - mapreduce.task.io.sort.mb: 100

2016-03-19 07:03:08,124 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - soft limit at 83886080
2016-03-19 07:03:08,124 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - bufstart = 0; bufvoid = 104857600
2016-03-19 07:03:08,124 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - kvstart = 26214396; length = 6553600
2016-03-19 07:03:08,125 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Map output collector class = org.apache.hadoop.mapred.Map
Task$MapOutputBuffer
2016-03-19 07:03:08,129 [LocalJobRunner Map Task Executor #0] INFO org.apache.p
ig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not gener
ate code.
2016-03-19 07:03:08,133 [LocalJobRunner Map Task Executor #0] INFO org.apache.p
ig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map - Alias
es being processed per job phase (AliasName[line,offset]): M: input1[4,9],words1
[-1,-1],group1[7,9] C: R:
2016-03-19 07:03:08,164 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner 2016-03-19 07:03:08,164 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Starting flush of map output
2016-03-19 07:03:08,164 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Spilling map output
2016-03-19 07:03:08,164 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - bufstart = 0; bufend = 6318; bufvoid = 104857600
2016-03-19 07:03:08,164 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - kvstart = 26214396(104857584); kvend = 26212024(104848096
); length = 2373/6553600
2016-03-19 07:03:08,202 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Finished spill 0
2016-03-19 07:03:08,205 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.Task - Task:attempt_local1118178464_0005_m_000000_0 is done. And is
in the process of committing
2016-03-19 07:03:08,207 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner - map
2016-03-19 07:03:08,207 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.Task - Task 'attempt_local1118178464_0005_m_000000_0' done.
2016-03-19 07:03:08,208 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner - Finishing task: attempt_local1118178464_0005_m_000
000_0
2016-03-19 07:03:08,208 [Thread-27] INFO org.apache.hadoop.mapred.LocalJobRunne
r - map task executor complete.
2016-03-19 07:03:08,214 [Thread-27] INFO org.apache.hadoop.mapred.LocalJobRunne
r - Waiting for reduce tasks
2016-03-19 07:03:08,214 [pool-14-thread-1] INFO org.apache.hadoop.mapred.LocalJ
obRunner - Starting task: attempt_local1118178464_0005_r_000000_0
2016-03-19 07:03:08,221 [pool-14-thread-1] INFO org.apache.hadoop.mapreduce.lib
.output.FileOutputCommitter - File Output Committer Algorithm version is 1
2016-03-19 07:03:08,227 [pool-14-thread-1] INFO org.apache.hadoop.mapred.Task Using ResourceCalculatorProcessTree : [ ]
2016-03-19 07:03:08,227 [pool-14-thread-1] INFO org.apache.hadoop.mapred.Reduce
Task - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuf
fle@6ab6fc14
2016-03-19 07:03:08,228 [pool-14-thread-1] INFO org.apache.hadoop.mapreduce.tas
k.reduce.MergeManagerImpl - MergerManager: memoryLimit=709551680, maxSingleShuff
leLimit=177387920, mergeThreshold=468304128, ioSortFactor=10, memToMemMergeOutpu
tsThreshold=10
2016-03-19 07:03:08,229 [EventFetcher for fetching Map Completion Events] INFO
org.apache.hadoop.mapreduce.task.reduce.EventFetcher - attempt_local1118178464_0
005_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events

2016-03-19 07:03:08,231 [localfetcher#2] INFO org.apache.hadoop.mapreduce.task.

reduce.LocalFetcher - localfetcher#2 about to shuffle output of map attempt_loca
l1118178464_0005_m_000000_0 decomp: 7508 len: 7512 to MEMORY
2016-03-19 07:03:08,231 [localfetcher#2] INFO org.apache.hadoop.mapreduce.task.
reduce.InMemoryMapOutput - Read 7508 bytes from map-output for attempt_local1118
178464_0005_m_000000_0
2016-03-19 07:03:08,231 [localfetcher#2] INFO org.apache.hadoop.mapreduce.task.
reduce.MergeManagerImpl - closeInMemoryFile -> map-output of size: 7508, inMemor
yMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->7508
2016-03-19 07:03:08,234 [EventFetcher for fetching Map Completion Events] INFO
org.apache.hadoop.mapreduce.task.reduce.EventFetcher - EventFetcher is interrupt
ed.. Returning
2016-03-19 07:03:08,235 [pool-14-thread-1] INFO org.apache.hadoop.mapred.LocalJ
obRunner - 1 / 1 copied.
2016-03-19 07:03:08,235 [pool-14-thread-1] INFO org.apache.hadoop.mapreduce.tas
k.reduce.MergeManagerImpl - finalMerge called with 1 in-memory map-outputs and 0
on-disk map-outputs
2016-03-19 07:03:08,236 [pool-14-thread-1] INFO org.apache.hadoop.mapred.Merger
- Merging 1 sorted segments
2016-03-19 07:03:08,236 [pool-14-thread-1] INFO org.apache.hadoop.mapred.Merger
- Down to the last merge-pass, with 1 segments left of total size: 7502 bytes
2016-03-19 07:03:08,261 [Readahead Thread #3] WARN org.apache.hadoop.io.Readahe
adPool - Failed readahead on ifile
EBADF: Bad file descriptor
at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(Native Met
hod)
at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(N
ativeIO.java:267)
at org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFa
dviseIfPossible(NativeIO.java:146)
at org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(Readahead
Pool.java:206)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-03-19 07:03:08,268 [pool-14-thread-1] INFO org.apache.hadoop.mapreduce.tas
k.reduce.MergeManagerImpl - Merged 1 segments, 7508 bytes to disk to satisfy red
uce memory limit
2016-03-19 07:03:08,268 [pool-14-thread-1] INFO org.apache.hadoop.mapreduce.tas
k.reduce.MergeManagerImpl - Merging 1 files, 7512 bytes from disk
2016-03-19 07:03:08,268 [pool-14-thread-1] INFO org.apache.hadoop.mapreduce.tas
k.reduce.MergeManagerImpl - Merging 0 segments, 0 bytes from memory into reduce
2016-03-19 07:03:08,268 [pool-14-thread-1] INFO org.apache.hadoop.mapred.Merger
- Merging 1 sorted segments
2016-03-19 07:03:08,268 [pool-14-thread-1] INFO org.apache.hadoop.mapred.Merger
- Down to the last merge-pass, with 1 segments left of total size: 7502 bytes
2016-03-19 07:03:08,269 [pool-14-thread-1] INFO org.apache.hadoop.mapred.LocalJ
obRunner - 1 / 1 copied.
2016-03-19 07:03:08,270 [pool-14-thread-1] INFO org.apache.hadoop.mapreduce.lib
.output.FileOutputCommitter - File Output Committer Algorithm version is 1
2016-03-19 07:03:08,274 [pool-14-thread-1] WARN org.apache.pig.data.SchemaTuple
Backend - SchemaTupleBackend has already been initialized
2016-03-19 07:03:08,276 [pool-14-thread-1] INFO org.apache.pig.backend.hadoop.e
xecutionengine.mapReduceLayer.PigMapReduce$Reduce - Aliases being processed per
job phase (AliasName[line,offset]): M: input1[4,9],words1[-1,-1],group1[7,9] C:
R:
2016-03-19 07:03:08,349 [pool-14-thread-1] INFO org.apache.hadoop.mapred.Task Task:attempt_local1118178464_0005_r_000000_0 is done. And is in the process of

committing
2016-03-19 07:03:08,351 [pool-14-thread-1] INFO org.apache.hadoop.mapred.LocalJ
obRunner - 1 / 1 copied.
2016-03-19 07:03:08,351 [pool-14-thread-1] INFO org.apache.hadoop.mapred.Task Task attempt_local1118178464_0005_r_000000_0 is allowed to commit now
2016-03-19 07:03:08,353 [pool-14-thread-1] INFO org.apache.hadoop.mapreduce.lib
.output.FileOutputCommitter - Saved output of task 'attempt_local1118178464_0005
_r_000000_0' to file:/tmp/temp2073304142/tmp664834969/_temporary/0/task_local111
8178464_0005_r_000000
2016-03-19 07:03:08,354 [pool-14-thread-1] INFO org.apache.hadoop.mapred.LocalJ
obRunner - reduce > reduce
2016-03-19 07:03:08,354 [pool-14-thread-1] INFO org.apache.hadoop.mapred.Task Task 'attempt_local1118178464_0005_r_000000_0' done.
2016-03-19 07:03:08,354 [pool-14-thread-1] INFO org.apache.hadoop.mapred.LocalJ
obRunner - Finishing task: attempt_local1118178464_0005_r_000000_0
2016-03-19 07:03:08,354 [Thread-27] INFO org.apache.hadoop.mapred.LocalJobRunne
r - reduce task executor complete.
2016-03-19 07:03:08,387 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_local1118178464_0005
2016-03-19 07:03:08,387 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - Processing aliases group1,input1,words1
2016-03-19 07:03:08,388 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - detailed locations: M: input1[4,9],words1[
-1,-1],group1[7,9] C: R:
2016-03-19 07:03:08,390 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - Running jobs are [job_local1118178464_0005
]
2016-03-19 07:03:08,534 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 07:03:08,536 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 07:03:08,537 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 07:03:08,542 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - 100% complete
2016-03-19 07:03:08,542 [main] INFO org.apache.pig.tools.pigstats.mapreduce.Sim
plePigStats - Script Statistics:
HadoopVersion PigVersion
UserId StartedAt
FinishedAt
2.7.1 0.15.0 root
2016-03-19 07:03:07
2016-03-19 07:03:08

Features
GROUP_BY

Success!
Job Stats (time in seconds):
JobId Maps
Reduces MaxMapTime
MinMapTime
AvgMapTime
MedianMa
pTime MaxReduceTime MinReduceTime AvgReduceTimeMedianReducetime Alias Fe
ature Outputs
job_local1118178464_0005
1
1
n/a
n/a
n/a
n/a
n/a
n/a
n/a
n/a
group1,input1,words1 GROUP_BY file:/tmp/temp2073
304142/tmp664834969,
Input(s):
Successfully read 30 records from: "/root/essay"
Output(s):
Successfully stored 296 records in: "file:/tmp/temp2073304142/tmp664834969"

Counters:
Total records written : 296
Total bytes written : 0
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0
Job DAG:
job_local1118178464_0005
2016-03-19 07:03:08,544 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 07:03:08,546 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 07:03:08,547 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 07:03:08,553 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - Success!
2016-03-19 07:03:08,553 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-03-19 07:03:08,554 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2016-03-19 07:03:08,554 [main] WARN org.apache.pig.data.SchemaTupleBackend - Sc
hemaTupleBackend has already been initialized
2016-03-19 07:03:08,572 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileI
nputFormat - Total input paths to process : 1
2016-03-19 07:03:08,572 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.util.MapRedUtil - Total input paths to process : 1
(&,{(&),(&)})
(A,{(A)})
(a,{(a),(a),(a),(a),(a),(a),(a),(a),(a),(a),(a)})
(Go,{(Go)})
(If,{(If)})
(In,{(In)})
(So,{(So),(So)})
(an,{(an),(an),(an)})
(as,{(as),(as),(as),(as)})
(be,{(be)})
(by,{(by),(by)})
(do,{(do)})
(he,{(he)})
(in,{(in),(in),(in),(in),(in),(in),(in),(in),(in),(in),(in),(in)})
(is,{(is),(is),(is),(is),(is),(is),(is),(is),(is)})
(it,{(it),(it)})
(no,{(no)})
(of,{(of),(of),(of),(of),(of),(of),(of),(of),(of),(of),(of),(of),(of),(of),(of)}
)
(on,{(on),(on),(on)})
(or,{(or),(or),(or),(or),(or)})
(so,{(so),(so)})
(to,{(to),(to),(to),(to),(to),(to),(to),(to),(to),(to),(to),(to),(to)})
(up,{(up)})
(us,{(us),(us),(us)})
(And,{(And)})
(But,{(But)})
(For,{(For)})

(Got,{(Got),(Got)})
(Now,{(Now)})
(One,{(One)})
(SMS,{(SMS)})
(The,{(The),(The),(The),(The),(The),(The)})
(Way,{(Way)})
(Yes,{(Yes)})
(all,{(all),(all)})
(and,{(and),(and),(and),(and),(and),(and),(and),(and),(and)})
(any,{(any),(any),(any),(any)})
(are,{(are),(are),(are),(are),(are),(are),(are)})
(big,{(big)})
(but,{(but)})
(buy,{(buy),(buy),(buy),(buy)})
(can,{(can)})
(do.,{(do.),(do.)})
(few,{(few)})
(for,{(for),(for),(for),(for),(for),(for)})
(go.,{(go.)})
(has,{(has)})
(it.,{(it.)})
(let,{(let),(let)})
(lot,{(lot)})
(man,{(man)})
(new,{(new)})
(not,{(not),(not),(not),(not),(not)})
(one,{(one),(one),(one)})
(our,{(our),(our),(our)})
(out,{(out)})
(own,{(own)})
(set,{(set)})
(the,{(the),(the),(the),(the),(the),(the),(the),(the),(the),(the),(the),(the),(t
he),(the),(the),(the),(the),(the),(the),(the),(the),(the),(the),(the),(the),(the
),(the)})
(use,{(use)})
(way,{(way)})
(who,{(who)})
(why,{(why),(why)})
(you,{(you),(you),(you),(you),(you),(you),(you),(you),(you),(you),(you),(you),(y
ou)})
(Easy,{(Easy)})
(Gone,{(Gone)})
(Here,{(Here)})
(How?,{(How?)})
(Just,{(Just),(Just),(Just),(Just)})
(Pick,{(Pick)})
(Read,{(Read)})
(Save,{(Save)})
(Thus,{(Thus)})
(Well,{(Well)})
(What,{(What),(What),(What)})
(Yeah,{(Yeah)})
(Your,{(Your)})
(all.,{(all.)})
(apps,{(apps)})
(away,{(away)})
(boon,{(boon)})
(call,{(call)})
(come,{(come)})
(days,{(days),(days)})

(dial,{(dial)})
(does,{(does),(does)})
(easy,{(easy)})
(etc.,{(etc.)})
(fast,{(fast)})
(find,{(find)})
(fire,{(fire)})
(for?,{(for?)})
(good,{(good)})
(hard,{(hard)})
(have,{(have)})
(home,{(home)})
(just,{(just),(just),(just)})
(life,{(life),(life)})
(like,{(like)})
(main,{(main)})
(make,{(make),(make)})
(many,{(many)})
(need,{(need),(need),(need),(need),(need)})
(part,{(part)})
(play,{(play)})
(role,{(role)})
(send,{(send)})
(stay,{(stay),(stay)})
(step,{(step)})
(such,{(such)})
(tags,{(tags)})
(take,{(take)})
(tell,{(tell)})
(that,{(that),(that),(that),(that),(that),(that)})
(them,{(them)})
(then,{(then)})
(they,{(they),(they)})
(this,{(this)})
(time,{(time)})
(turn,{(turn)})
(very,{(very)})
(want,{(want)})
(were,{(were)})
(when,{(when)})
(will,{(will),(will)})
(with,{(with),(with),(with),(with),(with)})
(you.,{(you.)})
(your,{(your),(your),(your),(your),(your),(your),(your)})
(Helps,{(Helps)})
(Media,{(Media)})
(Money,{(Money)})
(More:,{(More:)})
(Since,{(Since)})
(These,{(These)})
(away.,{(away.)})
(basic,{(basic),(basic)})
(bucks,{(bucks)})
(case?,{(case?)})
(deal.,{(deal.)})
(enjoy,{(enjoy)})
(first,{(first)})
(great,{(great)})
(life.,{(life.),(life.)})
(life?,{(life?)})

(lives,{(lives)})
(major,{(major)})
(makes,{(makes)})
(maybe,{(maybe)})
(meant,{(meant)})
(media,{(media),(media)})
(money,{(money),(money)})
(more:,{(more:)})
(move.,{(move.)})
(music,{(music)})
(owner,{(owner),(owner)})
(phone,{(phone),(phone),(phone),(phone),(phone),(phone),(phone),(phone),(phone),
(phone)})
(proud,{(proud),(proud)})
(prove,{(prove)})
(radio,{(radio)})
(reach,{(reach)})
(saver,{(saver)})
(small,{(small)})
(smart,{(smart)})
(spend,{(spend),(spend)})
(their,{(their)})
(these,{(these)})
(thing,{(thing)})
(those,{(those)})
(time.,{(time.),(time.)})
(users,{(users)})
(video,{(video)})
(want;,{(want;)})
(well.,{(well.)})
(which,{(which),(which)})
(with.,{(with.)})
(world,{(world)})
(would,{(would)})
(Access,{(Access)})
(Ensure,{(Ensure)})
(Mobile,{(Mobile),(Mobile),(Mobile),(Mobile),(Mobile),(Mobile)})
(Phones,{(Phones),(Phones),(Phones),(Phones)})
(Safety,{(Safety)})
(Social,{(Social)})
(Thanks,{(Thanks)})
(active,{(active)})
(become,{(become)})
(below.,{(below.)})
(better,{(better)})
(buying,{(buying),(buying),(buying)})
(cannot,{(cannot)})
(client,{(client)})
(extent,{(extent)})
(folks!,{(folks!)})
(forget,{(forget)})
(gadget,{(gadget)})
(gaming,{(gaming)})
(having,{(having)})
(impact,{(impact)})
(it s,{(it s),(it s)})
(lives?,{(lives?)})
(luxury,{(luxury)})
(market,{(market)})
(mobile,{(mobile),(mobile),(mobile),(mobile),(mobile),(mobile),(mobile),(mobile)

,(mobile),(mobile),(mobile),(mobile),(mobile),(mobile),(mobile),(mobile),(mobile
),(mobile),(mobile),(mobile),(mobile),(mobile),(mobile),(mobile),(mobile),(mobil
e)})
(modern,{(modern)})
(number,{(number)})
(packed,{(packed),(packed)})
(person,{(person),(person),(person)})
(phone.,{(phone.),(phone.),(phone.)})
(phones,{(phones),(phones),(phones),(phones),(phones),(phones),(phones),(phones)
,(phones),(phones),(phones),(phones),(phones),(phones)})
(player,{(player)})
(prices,{(prices)})
(right-,{(right-)})
(sites.,{(sites.)})
(social,{(social),(social),(social)})
(Simple!,{(Simple!)})
(answer.,{(answer.)})
(biggest,{(biggest)})
(brigade,{(brigade)})
(can t,{(can t)})
(console,{(console)})
(contact,{(contact)})
(don t,{(don t),(don t),(don t)})
(easier.,{(easier.)})
(example,{(example)})
(gadgets,{(gadgets),(gadgets)})
(growing,{(growing)})
(imagine,{(imagine),(imagine)})
(instant,{(instant)})
(lowered,{(lowered)})
(member?,{(member?)})
(message,{(message),(message),(message)})
(phones.,{(phones.)})
(police.,{(police.)})
(provide,{(provide),(provide)})
(require,{(require)})
(without,{(without),(without)})
(Business,{(Business)})
(Everyone,{(Everyone)})
(Internet,{(Internet),(Internet),(Internet)})
(Positive,{(Positive)})
(addicted,{(addicted)})
(answered,{(answered)})
(benefits,{(benefits)})
(business,{(business),(business)})
(comments,{(comments)})
(computer,{(computer)})
(consider,{(consider)})
(constant,{(constant)})
(everyone,{(everyone)})
(foremost,{(foremost)})
(high-end,{(high-end),(high-end)})
(intended,{(intended),(intended)})
(meeting.,{(meeting.)})
(negative,{(negative)})
(nowadays,{(nowadays)})
(organize,{(organize)})
(schedule,{(schedule)})
(straight,{(straight),(straight)})
(that s,{(that s)})

(urgently,{(urgently)})
(whenever,{(whenever)})
(wherever,{(wherever)})
(Internet.,{(Internet.)})
(available,{(available),(available)})
(blessings,{(blessings)})
(connected,{(connected),(connected)})
(employee?,{(employee?)})
(important,{(important),(important),(important),(important),(important)})
(landlines,{(landlines)})
(mid-range,{(mid-range)})
(necessity,{(necessity),(necessity)})
(possible.,{(possible.)})
(reminders,{(reminders)})
(security.,{(security.)})
(situation,{(situation)})
(today s,{(today s)})
(businesses,{(businesses)})
(considered,{(considered)})
(everywhere,{(everywhere)})
(importance,{(importance)})
(networking,{(networking)})
(ultimately,{(ultimately)})
(understand,{(understand)})
(businesses.,{(businesses.)})
(businessman,{(businessman)})
(communicate,{(communicate)})
(competition,{(competition)})
(likes/wants,{(likes/wants)})
(technology.,{(technology.)})
(Communication,{(Communication)})
(communication,{(communication)})
(friend/family,{(friend/family)})
(manufacturers,{(manufacturers)})
(communication.,{(communication.),(communication.)})
(http://myessaypoint.com/importance-of-mobile-phone-technology#ixzz43FAnUmFx,{(h
ttp://myessaypoint.com/importance-of-mobile-phone-technology#ixzz43FAnUmFx)})
(,{(),(),(),(),(),(),(),(),(),(),(),(),(),(),(),()})
grunt>
grunt>
grunt>
grunt>
grunt>
grunt>
grunt>
grunt>
grunt>
grunt> count1 = foreach group1 generate $0,count($1);
2016-03-19 07:05:19,883 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 10
70: Could not resolve count using imports: [, java.lang., org.apache.pig.builtin
., org.apache.pig.impl.builtin.]
Details at logfile: /root/pig_1458370235691.log
grunt> count1 = foreach group1 generate $0,COUNT($1);
2016-03-19 07:05:29,168 [main] WARN org.apache.pig.newplan.BaseOperatorPlan - E
ncountered Warning USING_OVERLOADED_FUNCTION 1 time(s).
2016-03-19 07:05:29,168 [main] WARN org.apache.pig.newplan.BaseOperatorPlan - E
ncountered Warning IMPLICIT_CAST_TO_CHARARRAY 1 time(s).
grunt> DUMP count1
2016-03-19 07:05:39,921 [main] WARN org.apache.pig.newplan.BaseOperatorPlan - E
ncountered Warning USING_OVERLOADED_FUNCTION 1 time(s).

2016-03-19 07:05:39,921 [main] WARN org.apache.pig.newplan.BaseOperatorPlan - E

ncountered Warning IMPLICIT_CAST_TO_CHARARRAY 1 time(s).
2016-03-19 07:05:39,923 [main] INFO org.apache.pig.tools.pigstats.ScriptState Pig features used in the script: GROUP_BY
2016-03-19 07:05:39,975 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-03-19 07:05:39,975 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2016-03-19 07:05:39,975 [main] INFO org.apache.pig.data.SchemaTupleBackend - Ke
y [pig.schematuple] was not set... will not generate code.
2016-03-19 07:05:39,976 [main] INFO org.apache.pig.newplan.logical.optimizer.Lo
gicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, ConstantCalc
ulator, GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeF
ilter, MergeForEach, PartitionFilterOptimizer, PredicatePushdownOptimizer, PushD
ownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter]}
2016-03-19 07:05:39,981 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? fal
se
2016-03-19 07:05:39,984 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.util.CombinerOptimizerUtil - Choosing to move algebraic foreach to combiner
2016-03-19 07:05:39,991 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
2016-03-19 07:05:39,991 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
2016-03-19 07:05:40,016 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-03-19 07:05:40,018 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2016-03-19 07:05:40,019 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 07:05:40,022 [main] INFO org.apache.pig.tools.pigstats.mapreduce.MRS
criptState - Pig script settings are added to the job
2016-03-19 07:05:40,023 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percen
t is not set, set to default 0.3
2016-03-19 07:05:40,026 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - Reduce phase detected, estimating # of re
quired reducers.
2016-03-19 07:05:40,026 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - Using reducer estimator: org.apache.pig.b
ackend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
2016-03-19 07:05:40,027 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.InputSizeReducerEstimator - BytesPerReducer=1000000000 maxRedu
cers=999 totalInputFileSize=3386
2016-03-19 07:05:40,028 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - Setting Parallelism to 1
2016-03-19 07:05:40,042 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - Setting up single store job
2016-03-19 07:05:40,045 [main] INFO org.apache.pig.data.SchemaTupleFrontend - K
ey [pig.schematuple] is false, will not generate code.
2016-03-19 07:05:40,045 [main] INFO org.apache.pig.data.SchemaTupleFrontend - S
tarting process to move generated code to distributed cacche
2016-03-19 07:05:40,045 [main] INFO org.apache.pig.data.SchemaTupleFrontend - D
istributed cache not supported or needed in local mode. Setting key [pig.schemat
uple.local.dir] with code temp directory: /tmp/1458371140044-0
2016-03-19 07:05:40,073 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission
.
2016-03-19 07:05:40,074 [JobControl] INFO org.apache.hadoop.metrics.jvm.JvmMetr

ics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - al

ready initialized
2016-03-19 07:05:40,083 [JobControl] WARN org.apache.hadoop.mapreduce.JobResour
ceUploader - No job jar file set. User classes may not be found. See Job or Job
#setJar(String).
2016-03-19 07:05:40,087 [JobControl] INFO org.apache.hadoop.mapreduce.lib.input
.FileInputFormat - Total input paths to process : 1
2016-03-19 07:05:40,087 [JobControl] INFO org.apache.pig.backend.hadoop.executi
onengine.util.MapRedUtil - Total input paths to process : 1
2016-03-19 07:05:40,087 [JobControl] INFO org.apache.pig.backend.hadoop.executi
onengine.util.MapRedUtil - Total input paths (combined) to process : 1
2016-03-19 07:05:40,089 [JobControl] INFO org.apache.hadoop.mapreduce.JobSubmit
ter - number of splits:1
2016-03-19 07:05:40,095 [JobControl] INFO org.apache.hadoop.mapreduce.JobSubmit
ter - Submitting tokens for job: job_local709000243_0006
2016-03-19 07:05:40,159 [JobControl] INFO org.apache.hadoop.mapreduce.Job - The
url to track the job: http://localhost:8080/
2016-03-19 07:05:40,160 [Thread-36] INFO org.apache.hadoop.mapred.LocalJobRunne
r - OutputCommitter set in config null
2016-03-19 07:05:40,170 [Thread-36] INFO org.apache.hadoop.conf.Configuration.d
eprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-03-19 07:05:40,171 [Thread-36] INFO org.apache.hadoop.conf.Configuration.d
eprecation - mapred.job.reduce.markreset.buffer.percent is deprecated. Instead,
use mapreduce.reduce.markreset.buffer.percent
2016-03-19 07:05:40,171 [Thread-36] INFO org.apache.hadoop.conf.Configuration.d
eprecation - mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduc
es
2016-03-19 07:05:40,171 [Thread-36] INFO org.apache.hadoop.conf.Configuration.d
eprecation - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker
.address
2016-03-19 07:05:40,171 [Thread-36] INFO org.apache.hadoop.conf.Configuration.d
eprecation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-che
cksum
2016-03-19 07:05:40,171 [Thread-36] INFO org.apache.hadoop.mapreduce.lib.output
.FileOutputCommitter - File Output Committer Algorithm version is 1
2016-03-19 07:05:40,171 [Thread-36] INFO org.apache.hadoop.mapred.LocalJobRunne
r - OutputCommitter is org.apache.pig.backend.hadoop.executionengine.mapReduceLa
yer.PigOutputCommitter
2016-03-19 07:05:40,176 [Thread-36] INFO org.apache.hadoop.mapred.LocalJobRunne
r - Waiting for map tasks
2016-03-19 07:05:40,176 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner - Starting task: attempt_local709000243_0006_m_00000
0_0
2016-03-19 07:05:40,182 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapreduce.lib.output.FileOutputCommitter - File Output Committer Algorithm
version is 1
2016-03-19 07:05:40,183 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.Task - Using ResourceCalculatorProcessTree : [ ]
2016-03-19 07:05:40,184 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Processing split: Number of splits :1
Total Length = 3386
Input split[0]:
Length = 3386
ClassName: org.apache.hadoop.mapreduce.lib.input.FileSplit
Locations:
----------------------2016-03-19 07:05:40,186 [LocalJobRunner Map Task Executor #0] INFO org.apache.p
ig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader - Current split

being processed file:/root/essay:0+3386

2016-03-19 07:05:40,222 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - (EQUATOR) 0 kvi 26214396(104857584)
2016-03-19 07:05:40,223 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - mapreduce.task.io.sort.mb: 100
2016-03-19 07:05:40,223 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - soft limit at 83886080
2016-03-19 07:05:40,223 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - bufstart = 0; bufvoid = 104857600
2016-03-19 07:05:40,223 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - kvstart = 26214396; length = 6553600
2016-03-19 07:05:40,229 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Map output collector class = org.apache.hadoop.mapred.Map
Task$MapOutputBuffer
2016-03-19 07:05:40,233 [LocalJobRunner Map Task Executor #0] INFO org.apache.p
ig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not gener
ate code.
2016-03-19 07:05:40,240 [LocalJobRunner Map Task Executor #0] INFO org.apache.p
ig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map - Alias
es being processed per job phase (AliasName[line,offset]): M: input1[4,9],words1
[-1,-1],count1[8,9],group1[7,9] C: count1[8,9],group1[7,9] R: count1[8,9]
2016-03-19 07:05:40,300 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner 2016-03-19 07:05:40,300 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Starting flush of map output
2016-03-19 07:05:40,300 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Spilling map output
2016-03-19 07:05:40,300 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - bufstart = 0; bufend = 7506; bufvoid = 104857600
2016-03-19 07:05:40,300 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - kvstart = 26214396(104857584); kvend = 26212024(104848096
); length = 2373/6553600
2016-03-19 07:05:40,336 [LocalJobRunner Map Task Executor #0] INFO org.apache.p
ig.backend.hadoop.executionengine.mapReduceLayer.PigCombiner$Combine - Aliases b
eing processed per job phase (AliasName[line,offset]): M: input1[4,9],words1[-1,
-1],count1[8,9],group1[7,9] C: count1[8,9],group1[7,9] R: count1[8,9]
2016-03-19 07:05:40,404 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Finished spill 0
2016-03-19 07:05:40,405 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.Task - Task:attempt_local709000243_0006_m_000000_0 is done. And is
in the process of committing
2016-03-19 07:05:40,407 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner - map
2016-03-19 07:05:40,408 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.Task - Task 'attempt_local709000243_0006_m_000000_0' done.
2016-03-19 07:05:40,408 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner - Finishing task: attempt_local709000243_0006_m_0000
00_0
2016-03-19 07:05:40,408 [Thread-36] INFO org.apache.hadoop.mapred.LocalJobRunne
r - map task executor complete.
2016-03-19 07:05:40,410 [Thread-36] INFO org.apache.hadoop.mapred.LocalJobRunne
r - Waiting for reduce tasks
2016-03-19 07:05:40,412 [pool-17-thread-1] INFO org.apache.hadoop.mapred.LocalJ
obRunner - Starting task: attempt_local709000243_0006_r_000000_0
2016-03-19 07:05:40,435 [pool-17-thread-1] INFO org.apache.hadoop.mapreduce.lib
.output.FileOutputCommitter - File Output Committer Algorithm version is 1
2016-03-19 07:05:40,442 [pool-17-thread-1] INFO org.apache.hadoop.mapred.Task Using ResourceCalculatorProcessTree : [ ]
2016-03-19 07:05:40,442 [pool-17-thread-1] INFO org.apache.hadoop.mapred.Reduce
Task - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuf

fle@5f81a1f3
2016-03-19 07:05:40,446 [pool-17-thread-1] INFO org.apache.hadoop.mapreduce.tas
k.reduce.MergeManagerImpl - MergerManager: memoryLimit=709551680, maxSingleShuff
leLimit=177387920, mergeThreshold=468304128, ioSortFactor=10, memToMemMergeOutpu
tsThreshold=10
2016-03-19 07:05:40,453 [EventFetcher for fetching Map Completion Events] INFO
org.apache.hadoop.mapreduce.task.reduce.EventFetcher - attempt_local709000243_00
06_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2016-03-19 07:05:40,457 [localfetcher#3] INFO org.apache.hadoop.mapreduce.task.
reduce.LocalFetcher - localfetcher#3 about to shuffle output of map attempt_loca
l709000243_0006_m_000000_0 decomp: 4695 len: 4699 to MEMORY
2016-03-19 07:05:40,458 [localfetcher#3] INFO org.apache.hadoop.mapreduce.task.
reduce.InMemoryMapOutput - Read 4695 bytes from map-output for attempt_local7090
00243_0006_m_000000_0
2016-03-19 07:05:40,458 [localfetcher#3] INFO org.apache.hadoop.mapreduce.task.
reduce.MergeManagerImpl - closeInMemoryFile -> map-output of size: 4695, inMemor
yMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->4695
2016-03-19 07:05:40,459 [EventFetcher for fetching Map Completion Events] INFO
org.apache.hadoop.mapreduce.task.reduce.EventFetcher - EventFetcher is interrupt
ed.. Returning
2016-03-19 07:05:40,495 [pool-17-thread-1] INFO org.apache.hadoop.mapred.LocalJ
obRunner - 1 / 1 copied.
2016-03-19 07:05:40,495 [pool-17-thread-1] INFO org.apache.hadoop.mapreduce.tas
k.reduce.MergeManagerImpl - finalMerge called with 1 in-memory map-outputs and 0
on-disk map-outputs
2016-03-19 07:05:40,496 [pool-17-thread-1] INFO org.apache.hadoop.mapred.Merger
- Merging 1 sorted segments
2016-03-19 07:05:40,496 [pool-17-thread-1] INFO org.apache.hadoop.mapred.Merger
- Down to the last merge-pass, with 1 segments left of total size: 4689 bytes
2016-03-19 07:05:40,494 [Readahead Thread #1] WARN org.apache.hadoop.io.Readahe
adPool - Failed readahead on ifile
EBADF: Bad file descriptor
at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(Native Met
hod)
at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(N
ativeIO.java:267)
at org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFa
dviseIfPossible(NativeIO.java:146)
at org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(Readahead
Pool.java:206)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-03-19 07:05:40,503 [pool-17-thread-1] INFO org.apache.hadoop.mapreduce.tas
k.reduce.MergeManagerImpl - Merged 1 segments, 4695 bytes to disk to satisfy red
uce memory limit
2016-03-19 07:05:40,503 [pool-17-thread-1] INFO org.apache.hadoop.mapreduce.tas
k.reduce.MergeManagerImpl - Merging 1 files, 4699 bytes from disk
2016-03-19 07:05:40,506 [pool-17-thread-1] INFO org.apache.hadoop.mapreduce.tas
k.reduce.MergeManagerImpl - Merging 0 segments, 0 bytes from memory into reduce
2016-03-19 07:05:40,506 [pool-17-thread-1] INFO org.apache.hadoop.mapred.Merger
- Merging 1 sorted segments
2016-03-19 07:05:40,506 [pool-17-thread-1] INFO org.apache.hadoop.mapred.Merger
- Down to the last merge-pass, with 1 segments left of total size: 4689 bytes
2016-03-19 07:05:40,507 [pool-17-thread-1] INFO org.apache.hadoop.mapred.LocalJ
obRunner - 1 / 1 copied.
2016-03-19 07:05:40,508 [pool-17-thread-1] INFO org.apache.hadoop.mapreduce.lib
.output.FileOutputCommitter - File Output Committer Algorithm version is 1

2016-03-19 07:05:40,516 [pool-17-thread-1] WARN org.apache.pig.data.SchemaTuple

Backend - SchemaTupleBackend has already been initialized
2016-03-19 07:05:40,523 [pool-17-thread-1] INFO org.apache.pig.backend.hadoop.e
xecutionengine.mapReduceLayer.PigMapReduce$Reduce - Aliases being processed per
job phase (AliasName[line,offset]): M: input1[4,9],words1[-1,-1],count1[8,9],gro
up1[7,9] C: count1[8,9],group1[7,9] R: count1[8,9]
2016-03-19 07:05:40,573 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_local709000243_0006
2016-03-19 07:05:40,573 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - Processing aliases count1,group1,input1,wo
rds1
2016-03-19 07:05:40,573 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - detailed locations: M: input1[4,9],words1[
-1,-1],count1[8,9],group1[7,9] C: count1[8,9],group1[7,9] R: count1[8,9]
2016-03-19 07:05:40,577 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - 50% complete
2016-03-19 07:05:40,577 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - Running jobs are [job_local709000243_0006]
2016-03-19 07:05:40,602 [pool-17-thread-1] INFO org.apache.hadoop.mapred.Task Task:attempt_local709000243_0006_r_000000_0 is done. And is in the process of c
ommitting
2016-03-19 07:05:40,605 [pool-17-thread-1] INFO org.apache.hadoop.mapred.LocalJ
obRunner - 1 / 1 copied.
2016-03-19 07:05:40,606 [pool-17-thread-1] INFO org.apache.hadoop.mapred.Task Task attempt_local709000243_0006_r_000000_0 is allowed to commit now
2016-03-19 07:05:40,608 [pool-17-thread-1] INFO org.apache.hadoop.mapreduce.lib
.output.FileOutputCommitter - Saved output of task 'attempt_local709000243_0006_
r_000000_0' to file:/tmp/temp2073304142/tmp1910013485/_temporary/0/task_local709
000243_0006_r_000000
2016-03-19 07:05:40,609 [pool-17-thread-1] INFO org.apache.hadoop.mapred.LocalJ
obRunner - reduce > reduce
2016-03-19 07:05:40,609 [pool-17-thread-1] INFO org.apache.hadoop.mapred.Task Task 'attempt_local709000243_0006_r_000000_0' done.
2016-03-19 07:05:40,609 [pool-17-thread-1] INFO org.apache.hadoop.mapred.LocalJ
obRunner - Finishing task: attempt_local709000243_0006_r_000000_0
2016-03-19 07:05:40,609 [Thread-36] INFO org.apache.hadoop.mapred.LocalJobRunne
r - reduce task executor complete.
2016-03-19 07:05:40,797 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 07:05:40,799 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 07:05:40,800 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 07:05:40,805 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - 100% complete
2016-03-19 07:05:40,806 [main] INFO org.apache.pig.tools.pigstats.mapreduce.Sim
plePigStats - Script Statistics:
HadoopVersion PigVersion
UserId StartedAt
FinishedAt
2.7.1 0.15.0 root
2016-03-19 07:05:40
2016-03-19 07:05:40

Features
GROUP_BY

Success!
Job Stats (time in seconds):
JobId Maps
Reduces MaxMapTime
pTime MaxReduceTime MinReduceTime
ature Outputs

MinMapTime
AvgMapTime
MedianMa
AvgReduceTimeMedianReducetime Alias Fe

job_local709000243_0006 1
1
n/a
n/a
n/a
n/a
n/a
n/a
n/a
n/a
count1,group1,input1,words1 GROUP_BY,COMBINER
le:/tmp/temp2073304142/tmp1910013485,

fi

Input(s):
Successfully read 30 records from: "/root/essay"
Output(s):
Successfully stored 296 records in: "file:/tmp/temp2073304142/tmp1910013485"
Counters:
Total records written : 296
Total bytes written : 0
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0
Job DAG:
job_local709000243_0006
2016-03-19 07:05:40,808 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 07:05:40,809 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 07:05:40,811 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 07:05:40,817 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - Success!
2016-03-19 07:05:40,817 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-03-19 07:05:40,817 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2016-03-19 07:05:40,818 [main] WARN org.apache.pig.data.SchemaTupleBackend - Sc
hemaTupleBackend has already been initialized
2016-03-19 07:05:40,852 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileI
nputFormat - Total input paths to process : 1
2016-03-19 07:05:40,852 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.util.MapRedUtil - Total input paths to process : 1
(&,2)
(A,1)
(a,11)
(Go,1)
(If,1)
(In,1)
(So,2)
(an,3)
(as,4)
(be,1)
(by,2)
(do,1)
(he,1)
(in,12)
(is,9)
(it,2)
(no,1)
(of,15)

(on,3)
(or,5)
(so,2)
(to,13)
(up,1)
(us,3)
(And,1)
(But,1)
(For,1)
(Got,2)
(Now,1)
(One,1)
(SMS,1)
(The,6)
(Way,1)
(Yes,1)
(all,2)
(and,9)
(any,4)
(are,7)
(big,1)
(but,1)
(buy,4)
(can,1)
(do.,2)
(few,1)
(for,6)
(go.,1)
(has,1)
(it.,1)
(let,2)
(lot,1)
(man,1)
(new,1)
(not,5)
(one,3)
(our,3)
(out,1)
(own,1)
(set,1)
(the,27)
(use,1)
(way,1)
(who,1)
(why,2)
(you,13)
(Easy,1)
(Gone,1)
(Here,1)
(How?,1)
(Just,4)
(Pick,1)
(Read,1)
(Save,1)
(Thus,1)
(Well,1)
(What,3)
(Yeah,1)
(Your,1)
(all.,1)

(apps,1)
(away,1)
(boon,1)
(call,1)
(come,1)
(days,2)
(dial,1)
(does,2)
(easy,1)
(etc.,1)
(fast,1)
(find,1)
(fire,1)
(for?,1)
(good,1)
(hard,1)
(have,1)
(home,1)
(just,3)
(life,2)
(like,1)
(main,1)
(make,2)
(many,1)
(need,5)
(part,1)
(play,1)
(role,1)
(send,1)
(stay,2)
(step,1)
(such,1)
(tags,1)
(take,1)
(tell,1)
(that,6)
(them,1)
(then,1)
(they,2)
(this,1)
(time,1)
(turn,1)
(very,1)
(want,1)
(were,1)
(when,1)
(will,2)
(with,5)
(you.,1)
(your,7)
(Helps,1)
(Media,1)
(Money,1)
(More:,1)
(Since,1)
(These,1)
(away.,1)
(basic,2)
(bucks,1)
(case?,1)

(deal.,1)
(enjoy,1)
(first,1)
(great,1)
(life.,2)
(life?,1)
(lives,1)
(major,1)
(makes,1)
(maybe,1)
(meant,1)
(media,2)
(money,2)
(more:,1)
(move.,1)
(music,1)
(owner,2)
(phone,10)
(proud,2)
(prove,1)
(radio,1)
(reach,1)
(saver,1)
(small,1)
(smart,1)
(spend,2)
(their,1)
(these,1)
(thing,1)
(those,1)
(time.,2)
(users,1)
(video,1)
(want;,1)
(well.,1)
(which,2)
(with.,1)
(world,1)
(would,1)
(Access,1)
(Ensure,1)
(Mobile,6)
(Phones,4)
(Safety,1)
(Social,1)
(Thanks,1)
(active,1)
(become,1)
(below.,1)
(better,1)
(buying,3)
(cannot,1)
(client,1)
(extent,1)
(folks!,1)
(forget,1)
(gadget,1)
(gaming,1)
(having,1)
(impact,1)

(it s,2)
(lives?,1)
(luxury,1)
(market,1)
(mobile,26)
(modern,1)
(number,1)
(packed,2)
(person,3)
(phone.,3)
(phones,14)
(player,1)
(prices,1)
(right-,1)
(sites.,1)
(social,3)
(Simple!,1)
(answer.,1)
(biggest,1)
(brigade,1)
(can t,1)
(console,1)
(contact,1)
(don t,3)
(easier.,1)
(example,1)
(gadgets,2)
(growing,1)
(imagine,2)
(instant,1)
(lowered,1)
(member?,1)
(message,3)
(phones.,1)
(police.,1)
(provide,2)
(require,1)
(without,2)
(Business,1)
(Everyone,1)
(Internet,3)
(Positive,1)
(addicted,1)
(answered,1)
(benefits,1)
(business,2)
(comments,1)
(computer,1)
(consider,1)
(constant,1)
(everyone,1)
(foremost,1)
(high-end,2)
(intended,2)
(meeting.,1)
(negative,1)
(nowadays,1)
(organize,1)
(schedule,1)
(straight,2)

(that s,1)
(urgently,1)
(whenever,1)
(wherever,1)
(Internet.,1)
(available,2)
(blessings,1)
(connected,2)
(employee?,1)
(important,5)
(landlines,1)
(mid-range,1)
(necessity,2)
(possible.,1)
(reminders,1)
(security.,1)
(situation,1)
(today s,1)
(businesses,1)
(considered,1)
(everywhere,1)
(importance,1)
(networking,1)
(ultimately,1)
(understand,1)
(businesses.,1)
(businessman,1)
(communicate,1)
(competition,1)
(likes/wants,1)
(technology.,1)
(Communication,1)
(communication,1)
(friend/family,1)
(manufacturers,1)
(communication.,2)
(http://myessaypoint.com/importance-of-mobile-phone-technology#ixzz43FAnUmFx,1)
(,0)
grunt>
grunt>
grunt>
grunt>
grunt>
grunt>
grunt> largest = Group count1 as $1>2;
2016-03-19 07:17:36,313 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 12
00: <line 9, column 23> Syntax error, unexpected symbol at or near 'as'
Details at logfile: /root/pig_1458370235691.log
grunt> largest = GROUP count1 as $1>2;
2016-03-19 07:17:48,818 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 12
00: <line 9, column 23> Syntax error, unexpected symbol at or near 'as'
Details at logfile: /root/pig_1458370235691.log
grunt> largest = foreach count1 as $1>2;
2016-03-19 07:18:05,419 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 12
00: <line 9, column 25> Syntax error, unexpected symbol at or near 'as'
Details at logfile: /root/pig_1458370235691.log
grunt> largest1 = foreach count1 as $1>2;
2016-03-19 07:18:23,138 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 12
00: <line 9, column 26> Syntax error, unexpected symbol at or near 'as'
Details at logfile: /root/pig_1458370235691.log

grunt> largest1 = filter count1 as $1>2;

2016-03-19 07:20:12,577 [main] ERROR org.apache.pig.tools.grunt.Grunt 00: <line 9, column 25> mismatched input 'as' expecting BY
Details at logfile: /root/pig_1458370235691.log
grunt> largest1 = FILTER count1 as $1>2;
2016-03-19 07:20:26,255 [main] ERROR org.apache.pig.tools.grunt.Grunt 00: <line 9, column 25> mismatched input 'as' expecting BY
Details at logfile: /root/pig_1458370235691.log
grunt> largest1 = FILTER count1 AS $1>2;
2016-03-19 07:20:48,791 [main] ERROR org.apache.pig.tools.grunt.Grunt 00: <line 9, column 25> mismatched input 'AS' expecting BY
Details at logfile: /root/pig_1458370235691.log
grunt> largest1 = FILTER count1 $1>2;
2016-03-19 07:21:03,917 [main] ERROR org.apache.pig.tools.grunt.Grunt 00: <line 9, column 25> mismatched input '$1' expecting BY
Details at logfile: /root/pig_1458370235691.log
grunt>
grunt> \
2016-03-19 07:22:25,550 [main] ERROR org.apache.pig.tools.grunt.Grunt 00: Error during parsing. Encountered " <PATH> "\\ "" at line 2, column
Was expecting one of:
<EOF>
"cat" ...
"clear" ...
"fs" ...
"sh" ...
"cd" ...
"cp" ...
"copyFromLocal" ...
"copyToLocal" ...
"dump" ...
"\\d" ...
"describe" ...
"\\de" ...
"aliases" ...
"explain" ...
"\\e" ...
"help" ...
"history" ...
"kill" ...
"ls" ...
"mv" ...
"mkdir" ...
"pwd" ...
"quit" ...
"\\q" ...
"register" ...
"rm" ...
"rmf" ...
"set" ...
"illustrate" ...
"\\i" ...
"run" ...
"exec" ...
"%default" ...
"%declare" ...
"scriptDone" ...
"" ...
"" ...
<EOL> ...

ERROR 12

ERROR 12

ERROR 12

ERROR 12

ERROR 10
1.

";" ...
Details at logfile: /root/pig_1458370235691.log
grunt>
grunt>
grunt>
grunt>
grunt>
grunt>
grunt>
grunt>
grunt>
grunt>
grunt> largest1 = FILTER count1 by $1>2;
2016-03-19 07:22:34,576 [main] WARN org.apache.pig.newplan.BaseOperatorPlan - E
ncountered Warning USING_OVERLOADED_FUNCTION 1 time(s).
2016-03-19 07:22:34,577 [main] WARN org.apache.pig.newplan.BaseOperatorPlan - E
ncountered Warning IMPLICIT_CAST_TO_LONG 1 time(s).
2016-03-19 07:22:34,577 [main] WARN org.apache.pig.newplan.BaseOperatorPlan - E
ncountered Warning IMPLICIT_CAST_TO_CHARARRAY 1 time(s).
grunt> DUMP largest1
2016-03-19 07:22:47,333 [main] WARN org.apache.pig.newplan.BaseOperatorPlan - E
ncountered Warning USING_OVERLOADED_FUNCTION 1 time(s).
2016-03-19 07:22:47,333 [main] WARN org.apache.pig.newplan.BaseOperatorPlan - E
ncountered Warning IMPLICIT_CAST_TO_LONG 1 time(s).
2016-03-19 07:22:47,333 [main] WARN org.apache.pig.newplan.BaseOperatorPlan - E
ncountered Warning IMPLICIT_CAST_TO_CHARARRAY 1 time(s).
2016-03-19 07:22:47,335 [main] INFO org.apache.pig.tools.pigstats.ScriptState Pig features used in the script: GROUP_BY,FILTER
2016-03-19 07:22:47,350 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-03-19 07:22:47,350 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2016-03-19 07:22:47,350 [main] INFO org.apache.pig.data.SchemaTupleBackend - Ke
y [pig.schematuple] was not set... will not generate code.
2016-03-19 07:22:47,350 [main] INFO org.apache.pig.newplan.logical.optimizer.Lo
gicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, ConstantCalc
ulator, GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeF
ilter, MergeForEach, PartitionFilterOptimizer, PredicatePushdownOptimizer, PushD
ownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter]}
2016-03-19 07:22:47,364 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-03-19 07:22:47,364 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2016-03-19 07:22:47,369 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? fal
se
2016-03-19 07:22:47,372 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.util.CombinerOptimizerUtil - Choosing to move algebraic foreach to combiner
2016-03-19 07:22:47,379 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
2016-03-19 07:22:47,379 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
2016-03-19 07:22:47,393 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-03-19 07:22:47,394 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2016-03-19 07:22:47,394 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized

2016-03-19 07:22:47,397 [main] INFO org.apache.pig.tools.pigstats.mapreduce.MRS

criptState - Pig script settings are added to the job
2016-03-19 07:22:47,398 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percen
t is not set, set to default 0.3
2016-03-19 07:22:47,399 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - Reduce phase detected, estimating # of re
quired reducers.
2016-03-19 07:22:47,399 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - Using reducer estimator: org.apache.pig.b
ackend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
2016-03-19 07:22:47,399 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.InputSizeReducerEstimator - BytesPerReducer=1000000000 maxRedu
cers=999 totalInputFileSize=3386
2016-03-19 07:22:47,399 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - Setting Parallelism to 1
2016-03-19 07:22:47,404 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - Setting up single store job
2016-03-19 07:22:47,405 [main] INFO org.apache.pig.data.SchemaTupleFrontend - K
ey [pig.schematuple] is false, will not generate code.
2016-03-19 07:22:47,405 [main] INFO org.apache.pig.data.SchemaTupleFrontend - S
tarting process to move generated code to distributed cacche
2016-03-19 07:22:47,405 [main] INFO org.apache.pig.data.SchemaTupleFrontend - D
istributed cache not supported or needed in local mode. Setting key [pig.schemat
uple.local.dir] with code temp directory: /tmp/1458372167405-0
2016-03-19 07:22:47,426 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission
.
2016-03-19 07:22:47,427 [JobControl] INFO org.apache.hadoop.metrics.jvm.JvmMetr
ics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - al
ready initialized
2016-03-19 07:22:47,437 [JobControl] WARN org.apache.hadoop.mapreduce.JobResour
ceUploader - No job jar file set. User classes may not be found. See Job or Job
#setJar(String).
2016-03-19 07:22:47,443 [JobControl] INFO org.apache.hadoop.mapreduce.lib.input
.FileInputFormat - Total input paths to process : 1
2016-03-19 07:22:47,443 [JobControl] INFO org.apache.pig.backend.hadoop.executi
onengine.util.MapRedUtil - Total input paths to process : 1
2016-03-19 07:22:47,444 [JobControl] INFO org.apache.pig.backend.hadoop.executi
onengine.util.MapRedUtil - Total input paths (combined) to process : 1
2016-03-19 07:22:47,446 [JobControl] INFO org.apache.hadoop.mapreduce.JobSubmit
ter - number of splits:1
2016-03-19 07:22:47,452 [JobControl] INFO org.apache.hadoop.mapreduce.JobSubmit
ter - Submitting tokens for job: job_local1726022349_0007
2016-03-19 07:22:47,512 [JobControl] INFO org.apache.hadoop.mapreduce.Job - The
url to track the job: http://localhost:8080/
2016-03-19 07:22:47,512 [Thread-45] INFO org.apache.hadoop.mapred.LocalJobRunne
r - OutputCommitter set in config null
2016-03-19 07:22:47,517 [Thread-45] INFO org.apache.hadoop.conf.Configuration.d
eprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-03-19 07:22:47,517 [Thread-45] INFO org.apache.hadoop.conf.Configuration.d
eprecation - mapred.job.reduce.markreset.buffer.percent is deprecated. Instead,
use mapreduce.reduce.markreset.buffer.percent
2016-03-19 07:22:47,517 [Thread-45] INFO org.apache.hadoop.conf.Configuration.d
eprecation - mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduc
es
2016-03-19 07:22:47,517 [Thread-45] INFO org.apache.hadoop.conf.Configuration.d
eprecation - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker
.address
2016-03-19 07:22:47,517 [Thread-45] INFO org.apache.hadoop.conf.Configuration.d

eprecation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-che

cksum
2016-03-19 07:22:47,518 [Thread-45] INFO org.apache.hadoop.mapreduce.lib.output
.FileOutputCommitter - File Output Committer Algorithm version is 1
2016-03-19 07:22:47,518 [Thread-45] INFO org.apache.hadoop.mapred.LocalJobRunne
r - OutputCommitter is org.apache.pig.backend.hadoop.executionengine.mapReduceLa
yer.PigOutputCommitter
2016-03-19 07:22:47,522 [Thread-45] INFO org.apache.hadoop.mapred.LocalJobRunne
r - Waiting for map tasks
2016-03-19 07:22:47,522 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner - Starting task: attempt_local1726022349_0007_m_0000
00_0
2016-03-19 07:22:47,527 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapreduce.lib.output.FileOutputCommitter - File Output Committer Algorithm
version is 1
2016-03-19 07:22:47,528 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.Task - Using ResourceCalculatorProcessTree : [ ]
2016-03-19 07:22:47,529 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Processing split: Number of splits :1
Total Length = 3386
Input split[0]:
Length = 3386
ClassName: org.apache.hadoop.mapreduce.lib.input.FileSplit
Locations:
----------------------2016-03-19 07:22:47,531 [LocalJobRunner Map Task Executor #0] INFO org.apache.p
ig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader - Current split
being processed file:/root/essay:0+3386
2016-03-19 07:22:47,552 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - (EQUATOR) 0 kvi 26214396(104857584)
2016-03-19 07:22:47,552 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - mapreduce.task.io.sort.mb: 100
2016-03-19 07:22:47,552 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - soft limit at 83886080
2016-03-19 07:22:47,552 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - bufstart = 0; bufvoid = 104857600
2016-03-19 07:22:47,552 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - kvstart = 26214396; length = 6553600
2016-03-19 07:22:47,553 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Map output collector class = org.apache.hadoop.mapred.Map
Task$MapOutputBuffer
2016-03-19 07:22:47,557 [LocalJobRunner Map Task Executor #0] INFO org.apache.p
ig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not gener
ate code.
2016-03-19 07:22:47,562 [LocalJobRunner Map Task Executor #0] INFO org.apache.p
ig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map - Alias
es being processed per job phase (AliasName[line,offset]): M: input1[4,9],words1
[-1,-1],count1[8,9],group1[7,9] C: count1[8,9],group1[7,9] R: count1[8,9],larges
t1[9,11]
2016-03-19 07:22:47,607 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner 2016-03-19 07:22:47,607 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Starting flush of map output
2016-03-19 07:22:47,608 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Spilling map output
2016-03-19 07:22:47,608 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - bufstart = 0; bufend = 7506; bufvoid = 104857600
2016-03-19 07:22:47,608 [LocalJobRunner Map Task Executor #0] INFO org.apache.h

adoop.mapred.MapTask - kvstart = 26214396(104857584); kvend = 26212024(104848096

); length = 2373/6553600
2016-03-19 07:22:47,731 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Finished spill 0
2016-03-19 07:22:47,739 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.Task - Task:attempt_local1726022349_0007_m_000000_0 is done. And is
in the process of committing
2016-03-19 07:22:47,740 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner - map
2016-03-19 07:22:47,740 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.Task - Task 'attempt_local1726022349_0007_m_000000_0' done.
2016-03-19 07:22:47,740 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner - Finishing task: attempt_local1726022349_0007_m_000
000_0
2016-03-19 07:22:47,741 [Thread-45] INFO org.apache.hadoop.mapred.LocalJobRunne
r - map task executor complete.
2016-03-19 07:22:47,741 [Thread-45] INFO org.apache.hadoop.mapred.LocalJobRunne
r - Waiting for reduce tasks
2016-03-19 07:22:47,743 [pool-20-thread-1] INFO org.apache.hadoop.mapred.LocalJ
obRunner - Starting task: attempt_local1726022349_0007_r_000000_0
2016-03-19 07:22:47,751 [pool-20-thread-1] INFO org.apache.hadoop.mapreduce.lib
.output.FileOutputCommitter - File Output Committer Algorithm version is 1
2016-03-19 07:22:47,759 [pool-20-thread-1] INFO org.apache.hadoop.mapred.Task Using ResourceCalculatorProcessTree : [ ]
2016-03-19 07:22:47,759 [pool-20-thread-1] INFO org.apache.hadoop.mapred.Reduce
Task - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuf
fle@4f578be8
2016-03-19 07:22:47,766 [pool-20-thread-1] INFO org.apache.hadoop.mapreduce.tas
k.reduce.MergeManagerImpl - MergerManager: memoryLimit=709551680, maxSingleShuff
leLimit=177387920, mergeThreshold=468304128, ioSortFactor=10, memToMemMergeOutpu
tsThreshold=10
2016-03-19 07:22:47,771 [EventFetcher for fetching Map Completion Events] INFO
org.apache.hadoop.mapreduce.task.reduce.EventFetcher - attempt_local1726022349_0
007_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2016-03-19 07:22:47,772 [localfetcher#4] INFO org.apache.hadoop.mapreduce.task.
reduce.LocalFetcher - localfetcher#4 about to shuffle output of map attempt_loca
l1726022349_0007_m_000000_0 decomp: 4695 len: 4699 to MEMORY
2016-03-19 07:22:47,773 [localfetcher#4] INFO org.apache.hadoop.mapreduce.task.
reduce.InMemoryMapOutput - Read 4695 bytes from map-output for attempt_local1726
022349_0007_m_000000_0
2016-03-19 07:22:47,773 [localfetcher#4] INFO org.apache.hadoop.mapreduce.task.
reduce.MergeManagerImpl - closeInMemoryFile -> map-output of size: 4695, inMemor
yMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->4695
2016-03-19 07:22:47,773 [EventFetcher for fetching Map Completion Events] INFO
org.apache.hadoop.mapreduce.task.reduce.EventFetcher - EventFetcher is interrupt
ed.. Returning
2016-03-19 07:22:47,774 [pool-20-thread-1] INFO org.apache.hadoop.mapred.LocalJ
obRunner - 1 / 1 copied.
2016-03-19 07:22:47,774 [pool-20-thread-1] INFO org.apache.hadoop.mapreduce.tas
k.reduce.MergeManagerImpl - finalMerge called with 1 in-memory map-outputs and 0
on-disk map-outputs
2016-03-19 07:22:47,775 [pool-20-thread-1] INFO org.apache.hadoop.mapred.Merger
- Merging 1 sorted segments
2016-03-19 07:22:47,775 [pool-20-thread-1] INFO org.apache.hadoop.mapred.Merger
- Down to the last merge-pass, with 1 segments left of total size: 4689 bytes
2016-03-19 07:22:47,782 [pool-20-thread-1] INFO org.apache.hadoop.mapreduce.tas
k.reduce.MergeManagerImpl - Merged 1 segments, 4695 bytes to disk to satisfy red
uce memory limit
2016-03-19 07:22:47,782 [pool-20-thread-1] INFO org.apache.hadoop.mapreduce.tas
k.reduce.MergeManagerImpl - Merging 1 files, 4699 bytes from disk

2016-03-19 07:22:47,783 [pool-20-thread-1] INFO org.apache.hadoop.mapreduce.tas

k.reduce.MergeManagerImpl - Merging 0 segments, 0 bytes from memory into reduce
2016-03-19 07:22:47,783 [pool-20-thread-1] INFO org.apache.hadoop.mapred.Merger
- Merging 1 sorted segments
2016-03-19 07:22:47,783 [pool-20-thread-1] INFO org.apache.hadoop.mapred.Merger
- Down to the last merge-pass, with 1 segments left of total size: 4689 bytes
2016-03-19 07:22:47,783 [pool-20-thread-1] INFO org.apache.hadoop.mapred.LocalJ
obRunner - 1 / 1 copied.
2016-03-19 07:22:47,785 [pool-20-thread-1] INFO org.apache.hadoop.mapreduce.lib
.output.FileOutputCommitter - File Output Committer Algorithm version is 1
2016-03-19 07:22:47,792 [pool-20-thread-1] WARN org.apache.pig.data.SchemaTuple
Backend - SchemaTupleBackend has already been initialized
2016-03-19 07:22:47,806 [pool-20-thread-1] INFO org.apache.pig.backend.hadoop.e
xecutionengine.mapReduceLayer.PigMapReduce$Reduce - Aliases being processed per
job phase (AliasName[line,offset]): M: input1[4,9],words1[-1,-1],count1[8,9],gro
up1[7,9] C: count1[8,9],group1[7,9] R: count1[8,9],largest1[9,11]
2016-03-19 07:22:47,919 [pool-20-thread-1] INFO org.apache.hadoop.mapred.Task Task:attempt_local1726022349_0007_r_000000_0 is done. And is in the process of
committing
2016-03-19 07:22:47,924 [pool-20-thread-1] INFO org.apache.hadoop.mapred.LocalJ
obRunner - 1 / 1 copied.
2016-03-19 07:22:47,924 [pool-20-thread-1] INFO org.apache.hadoop.mapred.Task Task attempt_local1726022349_0007_r_000000_0 is allowed to commit now
2016-03-19 07:22:47,926 [pool-20-thread-1] INFO org.apache.hadoop.mapreduce.lib
.output.FileOutputCommitter - Saved output of task 'attempt_local1726022349_0007
_r_000000_0' to file:/tmp/temp2073304142/tmp-479985687/_temporary/0/task_local17
26022349_0007_r_000000
2016-03-19 07:22:47,927 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_local1726022349_0007
2016-03-19 07:22:47,927 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - Processing aliases count1,group1,input1,la
rgest1,words1
2016-03-19 07:22:47,927 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - detailed locations: M: input1[4,9],words1[
-1,-1],count1[8,9],group1[7,9] C: count1[8,9],group1[7,9] R: count1[8,9],largest
1[9,11]
2016-03-19 07:22:47,929 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - 50% complete
2016-03-19 07:22:47,929 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - Running jobs are [job_local1726022349_0007
]
2016-03-19 07:22:47,930 [pool-20-thread-1] INFO org.apache.hadoop.mapred.LocalJ
obRunner - reduce > reduce
2016-03-19 07:22:47,938 [pool-20-thread-1] INFO org.apache.hadoop.mapred.Task Task 'attempt_local1726022349_0007_r_000000_0' done.
2016-03-19 07:22:47,938 [pool-20-thread-1] INFO org.apache.hadoop.mapred.LocalJ
obRunner - Finishing task: attempt_local1726022349_0007_r_000000_0
2016-03-19 07:22:47,938 [Thread-45] INFO org.apache.hadoop.mapred.LocalJobRunne
r - reduce task executor complete.
2016-03-19 07:22:48,115 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 07:22:48,117 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 07:22:48,118 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 07:22:48,123 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - 100% complete

2016-03-19 07:22:48,124 [main] INFO org.apache.pig.tools.pigstats.mapreduce.Sim

plePigStats - Script Statistics:
HadoopVersion PigVersion
UserId StartedAt
FinishedAt
2.7.1 0.15.0 root
2016-03-19 07:22:47
2016-03-19 07:22:48
,FILTER

Features
GROUP_BY

Success!
Job Stats (time in seconds):
JobId Maps
Reduces MaxMapTime
MinMapTime
AvgMapTime
MedianMa
pTime MaxReduceTime MinReduceTime AvgReduceTimeMedianReducetime Alias Fe
ature Outputs
job_local1726022349_0007
1
1
n/a
n/a
n/a
n/a
n/a
n/a
n/a
n/a
count1,group1,input1,largest1,words1 GROUP_BY,C
OMBINER
file:/tmp/temp2073304142/tmp-479985687,
Input(s):
Successfully read 30 records from: "/root/essay"
Output(s):
Successfully stored 41 records in: "file:/tmp/temp2073304142/tmp-479985687"
Counters:
Total records written : 41
Total bytes written : 0
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0
Job DAG:
job_local1726022349_0007
2016-03-19 07:22:48,126 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 07:22:48,127 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 07:22:48,128 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 07:22:48,135 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - Success!
2016-03-19 07:22:48,135 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-03-19 07:22:48,135 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2016-03-19 07:22:48,135 [main] WARN org.apache.pig.data.SchemaTupleBackend - Sc
hemaTupleBackend has already been initialized
2016-03-19 07:22:48,149 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileI
nputFormat - Total input paths to process : 1
2016-03-19 07:22:48,149 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.util.MapRedUtil - Total input paths to process : 1
(a,11)
(an,3)
(as,4)
(in,12)
(is,9)

(of,15)
(on,3)
(or,5)
(to,13)
(us,3)
(The,6)
(and,9)
(any,4)
(are,7)
(buy,4)
(for,6)
(not,5)
(one,3)
(our,3)
(the,27)
(you,13)
(Just,4)
(What,3)
(just,3)
(need,5)
(that,6)
(with,5)
(your,7)
(phone,10)
(Mobile,6)
(Phones,4)
(buying,3)
(mobile,26)
(person,3)
(phone.,3)
(phones,14)
(social,3)
(don t,3)
(message,3)
(Internet,3)
(important,5)
grunt>
grunt>
grunt>
grunt>
grunt> quit
2016-03-19 07:24:18,254 [main] INFO org.apache.pig.Main - Pig script completed
in 33 minutes, 43 seconds and 327 milliseconds (2023327 ms)
root@ip-172-31-37-176:~# bin dev home
lib
lost+found mnt proc ru
n srv tmp var
No command 'bin' found, did you mean:
Command 'bing' from package 'bing' (universe)
Command 'win' from package 'wily' (universe)
Command 'ben' from package 'ben' (universe)
Command 'bins' from package 'bins' (universe)
Command 'bip' from package 'bip' (universe)
Command 'tin' from package 'tin' (universe)
Command 'bio' from package 'barry-util' (universe)
Command 'bino' from package 'bino' (universe)
Command 'din' from package 'din' (universe)
bin: command not found
root@ip-172-31-37-176:~# boot etc initrd.img lib64 media
opt root sb
in sys usr vmlinuz
No command 'boot' found, did you mean:
Command 'boog' from package 'alliance' (universe)

Command 'booth' from package 'booth' (universe)

Command 'brot' from package 'fortunes-de' (universe)
Command 'boom' from package 'prboom-plus' (universe)
Command 'boom' from package 'alliance' (universe)
Command 'hboot' from package 'lam-runtime' (universe)
Command 'booi' from package 'boo' (universe)
Command 'root' from package 'root-system-bin' (universe)
Command 'booc' from package 'boo' (universe)
boot: command not found
root@ip-172-31-37-176:~# bin dev home
lib
lost+found mnt proc ru
n srv tmp var
No command 'bin' found, did you mean:
Command 'win' from package 'wily' (universe)
Command 'ben' from package 'ben' (universe)
Command 'bing' from package 'bing' (universe)
Command 'bip' from package 'bip' (universe)
Command 'bino' from package 'bino' (universe)
Command 'din' from package 'din' (universe)
Command 'tin' from package 'tin' (universe)
Command 'bio' from package 'barry-util' (universe)
Command 'bins' from package 'bins' (universe)
bin: command not found
root@ip-172-31-37-176:~# boot etc initrd.img lib64 media
opt root sb
in sys usr vmlinuz
No command 'boot' found, did you mean:
Command 'boog' from package 'alliance' (universe)
Command 'root' from package 'root-system-bin' (universe)
Command 'booth' from package 'booth' (universe)
Command 'booc' from package 'boo' (universe)
Command 'brot' from package 'fortunes-de' (universe)
Command 'booi' from package 'boo' (universe)
Command 'hboot' from package 'lam-runtime' (universe)
Command 'boom' from package 'prboom-plus' (universe)
Command 'boom' from package 'alliance' (universe)
boot: command not found
root@ip-172-31-37-176:~#

login as: ubuntu

Authenticating with public key "imported-openssh-key"
Welcome to Ubuntu 14.04.3 LTS (GNU/Linux 3.13.0-74-generic x86_64)
* Documentation: https://help.ubuntu.com/
System information as of Sat Mar 19 04:15:18 UTC 2016
System load:
Usage of /:
Memory usage:
Swap usage:

0.27
24.5% of 7.74GB
6%
0%

Processes:
102
Users logged in:
0
IP address for eth0: 172.31.37.176

Graph this data and manage this system at:

https://landscape.canonical.com/
Get cloud support with Ubuntu Advantage Cloud Guest:
http://www.ubuntu.com/business/services/cloud

67 packages can be updated.

37 updates are security updates.
Last login: Sat Mar 19 04:15:18 2016 from 182.73.111.182
ubuntu@ip-172-31-37-176:~$ sudo -i
root@ip-172-31-37-176:~# pig -x local
16/03/19 06:31:01 INFO pig.ExecTypeProvider: Trying ExecType : LOCAL
16/03/19 06:31:01 INFO pig.ExecTypeProvider: Picked LOCAL as the ExecType
2016-03-19 06:31:01,738 [main] INFO org.apache.pig.Main - Apache Pig version 0.
15.0 (r1682971) compiled J
un 01 2015, 11:44:35
2016-03-19 06:31:01,738 [main] INFO org.apache.pig.Main - Logging error message
s to: /root/pig_1458369061
737.log
2016-03-19 06:31:01,757 [main] INFO org.apache.pig.impl.util.Utils - Default bo
otup file /root/.pigbootup
not found
2016-03-19 06:31:02,433 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - fs.default.name is
deprecated. Instead, use fs.defaultFS
2016-03-19 06:31:02,434 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - mapred.job.tracker
is deprecated. Instead, use mapreduce.jobtracker.addr
ess
2016-03-19 06:31:02,436 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.HExecutionEngine - Conn
ecting to hadoop file system at: file:///
2016-03-19 06:31:02,579 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - io.bytes.per.check
sum is deprecated. Instead, use dfs.bytes-per-checksum
grunt> Empemail = LOAD '/root/input/EmpEmailID.txt' as (id:int , email:chararray
);
2016-03-19 06:32:15,619 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - fs.default.name is
deprecated. Instead, use fs.defaultFS
2016-03-19 06:32:15,631 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - io.bytes.per.check
sum is deprecated. Instead, use dfs.bytes-per-checksum
grunt> dump Empemail
2016-03-19 06:32:33,698 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-03-19 06:32:33,701 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2016-03-19 06:32:33,746 [main] INFO org.apache.pig.tools.pigstats.ScriptState Pig features used in the script: UNKNOWN
2016-03-19 06:32:33,787 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-03-19 06:32:33,792 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2016-03-19 06:32:33,827 [main] INFO org.apache.pig.newplan.logical.optimizer.Lo
gicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, ConstantCalc
ulator, GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeF
ilter, MergeForEach, PartitionFilterOptimizer, PredicatePushdownOptimizer, PushD
ownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter]}
2016-03-19 06:32:33,931 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? fal
se
2016-03-19 06:32:33,990 [main] INFO org.apache.pig.backend.hadoop.executionengi

ne.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1

2016-03-19 06:32:33,993 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
2016-03-19 06:32:34,060 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-03-19 06:32:34,061 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2016-03-19 06:32:34,094 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - session.id is deprecated. Instead, use dfs.metrics.session-id
2016-03-19 06:32:34,103 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Initializing JVM Metrics with processName=JobTracker, sessionId=
2016-03-19 06:32:34,173 [main] INFO org.apache.pig.tools.pigstats.mapreduce.MRS
criptState - Pig script settings are added to the job
2016-03-19 06:32:34,191 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - mapred.job.reduce.markreset.buffer.percent is deprecated. Instead, use m
apreduce.reduce.markreset.buffer.percent
2016-03-19 06:32:34,194 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percen
t is not set, set to default 0.3
2016-03-19 06:32:34,194 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - mapred.output.compress is deprecated. Instead, use mapreduce.output.file
outputformat.compress
2016-03-19 06:32:34,315 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - Setting up single store job
2016-03-19 06:32:34,333 [main] INFO org.apache.pig.data.SchemaTupleFrontend - K
ey [pig.schematuple] is false, will not generate code.
2016-03-19 06:32:34,338 [main] INFO org.apache.pig.data.SchemaTupleFrontend - S
tarting process to move generated code to distributed cacche
2016-03-19 06:32:34,338 [main] INFO org.apache.pig.data.SchemaTupleFrontend - D
istributed cache not supported or needed in local mode. Setting key [pig.schemat
uple.local.dir] with code temp directory: /tmp/1458369154333-0
2016-03-19 06:32:34,396 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission
.
2016-03-19 06:32:34,397 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - mapred.job.tracker.http.address is deprecated. Instead, use mapreduce.jo
btracker.http.address
2016-03-19 06:32:34,411 [JobControl] INFO org.apache.hadoop.metrics.jvm.JvmMetr
ics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - al
ready initialized
2016-03-19 06:32:34,500 [JobControl] WARN org.apache.hadoop.mapreduce.JobResour
ceUploader - No job jar file set. User classes may not be found. See Job or Job
#setJar(String).
2016-03-19 06:32:34,568 [JobControl] INFO org.apache.hadoop.mapreduce.JobSubmit
ter - Cleaning up the staging area file:/tmp/hadoop-root/mapred/staging/root9357
16635/.staging/job_local935716635_0001
2016-03-19 06:32:34,572 [JobControl] INFO org.apache.hadoop.mapreduce.lib.jobco
ntrol.ControlledJob - PigLatin:DefaultJobName got an error while submitting
org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Input path doe
s not exist: file:/root/input/EmpEmailID.txt
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInput
Format.getSplits(PigInputFormat.java:279)
at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.
java:301)
at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.jav
a:318)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitt
er.java:196)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)

at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInforma
tion.java:1657)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(Contr
olledJob.java:335)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
sorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.pig.backend.hadoop23.PigJobControl.submit(PigJobControl.ja
va:128)
at org.apache.pig.backend.hadoop23.PigJobControl.run(PigJobControl.java:
194)
at java.lang.Thread.run(Thread.java:745)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduc
eLauncher$1.run(MapReduceLauncher.java:276)
Caused by: org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input pa
th does not exist: file:/root/input/EmpEmailID.txt
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedL
istStatus(FileInputFormat.java:323)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(File
InputFormat.java:265)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigTextI
nputFormat.listStatus(PigTextInputFormat.java:36)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileI
nputFormat.java:387)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInput
Format.getSplits(PigInputFormat.java:265)
... 18 more
2016-03-19 06:32:34,780 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_local935716635_0001
2016-03-19 06:32:34,781 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - Processing aliases Empemail
2016-03-19 06:32:34,782 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - detailed locations: M: Empemail[1,11],Empe
mail[-1,-1] C: R:
2016-03-19 06:32:34,788 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - 0% complete
2016-03-19 06:32:34,796 [main] WARN org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - Ooops! Some job has failed! Specify -stop_
on_failure if you want Pig to stop immediately on failure.
2016-03-19 06:32:34,798 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - job job_local935716635_0001 has failed! St
op running all dependent jobs
2016-03-19 06:32:34,798 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - 100% complete
2016-03-19 06:32:34,801 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 06:32:34,803 [main] ERROR org.apache.pig.tools.pigstats.PigStats - ER
ROR 0: java.lang.IllegalStateException: Job in state DEFINE instead of RUNNING
2016-03-19 06:32:34,803 [main] ERROR org.apache.pig.tools.pigstats.mapreduce.MRP
igStatsUtil - 1 map reduce job(s) failed!
2016-03-19 06:32:34,804 [main] INFO org.apache.pig.tools.pigstats.mapreduce.Sim
plePigStats - Script Statistics:

HadoopVersion PigVersion
UserId StartedAt
FinishedAt
2.7.1 0.15.0 root
2016-03-19 06:32:34
2016-03-19 06:32:34

Features
UNKNOWN

Failed!
Failed Jobs:
JobId Alias Feature Message Outputs
job_local935716635_0001 Empemail
MAP_ONLY
Message: org.apache.pig.
backend.executionengine.ExecException: ERROR 2118: Input path does not exist: fi
le:/root/input/EmpEmailID.txt
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInput
Format.getSplits(PigInputFormat.java:279)
at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.
java:301)
at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.jav
a:318)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitt
er.java:196)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInforma
tion.java:1657)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(Contr
olledJob.java:335)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
sorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.pig.backend.hadoop23.PigJobControl.submit(PigJobControl.ja
va:128)
at org.apache.pig.backend.hadoop23.PigJobControl.run(PigJobControl.java:
194)
at java.lang.Thread.run(Thread.java:745)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduc
eLauncher$1.run(MapReduceLauncher.java:276)
Caused by: org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input pa
th does not exist: file:/root/input/EmpEmailID.txt
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedL
istStatus(FileInputFormat.java:323)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(File
InputFormat.java:265)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigTextI
nputFormat.listStatus(PigTextInputFormat.java:36)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileI
nputFormat.java:387)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInput
Format.getSplits(PigInputFormat.java:265)
... 18 more
file:/tmp/temp666787139/tmp-2036215924,
Input(s):
Failed to read data from "/root/input/EmpEmailID.txt"
Output(s):
Failed to produce result in "file:/tmp/temp666787139/tmp-2036215924"

Counters:
Total records written : 0
Total bytes written : 0
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0
Job DAG:
job_local935716635_0001
2016-03-19 06:32:34,806 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - Failed!
2016-03-19 06:32:34,822 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 10
66: Unable to open iterator for alias Empemail. Backend error : java.lang.Illega
lStateException: Job in state DEFINE instead of RUNNING
Details at logfile: /root/pig_1458369061737.log
grunt> Empemail = LOAD '/root/input/EmpEmailId.txt' as (id:int , email:chararray
);
2016-03-19 06:33:45,134 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-03-19 06:33:45,135 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
grunt> dump Empemail
2016-03-19 06:33:56,264 [main] INFO org.apache.pig.tools.pigstats.ScriptState Pig features used in the script: UNKNOWN
2016-03-19 06:33:56,294 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-03-19 06:33:56,295 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2016-03-19 06:33:56,295 [main] INFO org.apache.pig.data.SchemaTupleBackend - Ke
y [pig.schematuple] was not set... will not generate code.
2016-03-19 06:33:56,295 [main] INFO org.apache.pig.newplan.logical.optimizer.Lo
gicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, ConstantCalc
ulator, GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeF
ilter, MergeForEach, PartitionFilterOptimizer, PredicatePushdownOptimizer, PushD
ownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter]}
2016-03-19 06:33:56,298 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? fal
se
2016-03-19 06:33:56,299 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
2016-03-19 06:33:56,299 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
2016-03-19 06:33:56,323 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-03-19 06:33:56,324 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2016-03-19 06:33:56,325 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 06:33:56,327 [main] INFO org.apache.pig.tools.pigstats.mapreduce.MRS
criptState - Pig script settings are added to the job
2016-03-19 06:33:56,328 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percen
t is not set, set to default 0.3
2016-03-19 06:33:56,342 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - Setting up single store job
2016-03-19 06:33:56,343 [main] INFO org.apache.pig.data.SchemaTupleFrontend - K

ey [pig.schematuple] is false, will not generate code.

2016-03-19 06:33:56,343 [main] INFO org.apache.pig.data.SchemaTupleFrontend - S
tarting process to move generated code to distributed cacche
2016-03-19 06:33:56,343 [main] INFO org.apache.pig.data.SchemaTupleFrontend - D
istributed cache not supported or needed in local mode. Setting key [pig.schemat
uple.local.dir] with code temp directory: /tmp/1458369236343-0
2016-03-19 06:33:56,369 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission
.
2016-03-19 06:33:56,374 [JobControl] INFO org.apache.hadoop.metrics.jvm.JvmMetr
ics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - al
ready initialized
2016-03-19 06:33:56,394 [JobControl] WARN org.apache.hadoop.mapreduce.JobResour
ceUploader - No job jar file set. User classes may not be found. See Job or Job
#setJar(String).
2016-03-19 06:33:56,442 [JobControl] INFO org.apache.hadoop.mapreduce.lib.input
.FileInputFormat - Total input paths to process : 1
2016-03-19 06:33:56,442 [JobControl] INFO org.apache.pig.backend.hadoop.executi
onengine.util.MapRedUtil - Total input paths to process : 1
2016-03-19 06:33:56,456 [JobControl] INFO org.apache.pig.backend.hadoop.executi
onengine.util.MapRedUtil - Total input paths (combined) to process : 1
2016-03-19 06:33:56,483 [JobControl] INFO org.apache.hadoop.mapreduce.JobSubmit
ter - number of splits:1
2016-03-19 06:33:56,867 [JobControl] INFO org.apache.hadoop.mapreduce.JobSubmit
ter - Submitting tokens for job: job_local1224478176_0002
2016-03-19 06:33:57,361 [JobControl] INFO org.apache.hadoop.mapreduce.Job - The
url to track the job: http://localhost:8080/
2016-03-19 06:33:57,363 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_local1224478176_0002
2016-03-19 06:33:57,363 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - Processing aliases Empemail
2016-03-19 06:33:57,363 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - detailed locations: M: Empemail[2,11],Empe
mail[-1,-1] C: R:
2016-03-19 06:33:57,365 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - 0% complete
2016-03-19 06:33:57,366 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - Running jobs are [job_local1224478176_0002
]
2016-03-19 06:33:57,366 [Thread-6] INFO org.apache.hadoop.mapred.LocalJobRunner
- OutputCommitter set in config null
2016-03-19 06:33:57,412 [Thread-6] INFO org.apache.hadoop.conf.Configuration.de
precation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-03-19 06:33:57,412 [Thread-6] INFO org.apache.hadoop.conf.Configuration.de
precation - mapred.job.reduce.markreset.buffer.percent is deprecated. Instead, u
se mapreduce.reduce.markreset.buffer.percent
2016-03-19 06:33:57,412 [Thread-6] INFO org.apache.hadoop.conf.Configuration.de
precation - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.
address
2016-03-19 06:33:57,412 [Thread-6] INFO org.apache.hadoop.conf.Configuration.de
precation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-chec
ksum
2016-03-19 06:33:57,413 [Thread-6] INFO org.apache.hadoop.mapreduce.lib.output.
FileOutputCommitter - File Output Committer Algorithm version is 1
2016-03-19 06:33:57,415 [Thread-6] INFO org.apache.hadoop.mapred.LocalJobRunner
- OutputCommitter is org.apache.pig.backend.hadoop.executionengine.mapReduceLay
er.PigOutputCommitter
2016-03-19 06:33:57,494 [Thread-6] INFO org.apache.hadoop.mapred.LocalJobRunner
- Waiting for map tasks
2016-03-19 06:33:57,494 [LocalJobRunner Map Task Executor #0] INFO org.apache.h

adoop.mapred.LocalJobRunner - Starting task: attempt_local1224478176_0002_m_0000

00_0
2016-03-19 06:33:57,535 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapreduce.lib.output.FileOutputCommitter - File Output Committer Algorithm
version is 1
2016-03-19 06:33:57,555 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.Task - Using ResourceCalculatorProcessTree : [ ]
2016-03-19 06:33:57,560 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Processing split: Number of splits :1
Total Length = 237
Input split[0]:
Length = 237
ClassName: org.apache.hadoop.mapreduce.lib.input.FileSplit
Locations:
----------------------2016-03-19 06:33:57,590 [LocalJobRunner Map Task Executor #0] INFO org.apache.p
ig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader - Current split
being processed file:/root/input/EmpEmailId.txt:0+237
2016-03-19 06:33:57,601 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapreduce.lib.output.FileOutputCommitter - File Output Committer Algorithm
version is 1
2016-03-19 06:33:57,644 [LocalJobRunner Map Task Executor #0] INFO org.apache.p
ig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not gener
ate code.
2016-03-19 06:33:57,665 [LocalJobRunner Map Task Executor #0] INFO org.apache.p
ig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map - Aliases being
processed per job phase (AliasName[line,offset]): M: Empemail[2,11],Empemail[-1,
-1] C: R:
2016-03-19 06:33:57,674 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner 2016-03-19 06:33:57,674 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.Task - Task:attempt_local1224478176_0002_m_000000_0 is done. And is
in the process of committing
2016-03-19 06:33:57,683 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner 2016-03-19 06:33:57,683 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.Task - Task attempt_local1224478176_0002_m_000000_0 is allowed to c
ommit now
2016-03-19 06:33:57,685 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapreduce.lib.output.FileOutputCommitter - Saved output of task 'attempt_l
ocal1224478176_0002_m_000000_0' to file:/tmp/temp666787139/tmp1022024822/_tempor
ary/0/task_local1224478176_0002_m_000000
2016-03-19 06:33:57,686 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner - map
2016-03-19 06:33:57,686 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.Task - Task 'attempt_local1224478176_0002_m_000000_0' done.
2016-03-19 06:33:57,686 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner - Finishing task: attempt_local1224478176_0002_m_000
000_0
2016-03-19 06:33:57,686 [Thread-6] INFO org.apache.hadoop.mapred.LocalJobRunner
- map task executor complete.
2016-03-19 06:33:57,869 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 06:33:57,872 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 06:33:57,873 [main] INFO org.apache.hadoop.conf.Configuration.deprec

ation - mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps

2016-03-19 06:33:57,874 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
2016-03-19 06:33:57,874 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 06:33:57,905 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - 100% complete
2016-03-19 06:33:57,911 [main] INFO org.apache.pig.tools.pigstats.mapreduce.Sim
plePigStats - Script Statistics:
HadoopVersion PigVersion
UserId StartedAt
FinishedAt
2.7.1 0.15.0 root
2016-03-19 06:33:56
2016-03-19 06:33:57

Features
UNKNOWN

Success!
Job Stats (time in seconds):
JobId Maps
Reduces MaxMapTime
MinMapTime
AvgMapTime
MedianMa
pTime MaxReduceTime MinReduceTime AvgReduceTimeMedianReducetime Alias Fe
ature Outputs
job_local1224478176_0002
1
0
n/a
n/a
n/a
n/a
0
0
0
0
Empemail
MAP_ONLY
file:/tmp/temp6667
87139/tmp1022024822,
Input(s):
Successfully read 11 records from: "/root/input/EmpEmailId.txt"
Output(s):
Successfully stored 11 records in: "file:/tmp/temp666787139/tmp1022024822"
Counters:
Total records written : 11
Total bytes written : 0
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0
Job DAG:
job_local1224478176_0002
2016-03-19 06:33:57,914 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 06:33:57,915 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 06:33:57,917 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 06:33:57,923 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - Success!
2016-03-19 06:33:57,924 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-03-19 06:33:57,924 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2016-03-19 06:33:57,924 [main] WARN org.apache.pig.data.SchemaTupleBackend - Sc
hemaTupleBackend has already been initialized
2016-03-19 06:33:57,937 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileI
nputFormat - Total input paths to process : 1

2016-03-19 06:33:57,937 [main] INFO org.apache.pig.backend.hadoop.executionengi

ne.util.MapRedUtil - Total input paths to process : 1
(1,[email protected])
(2,[email protected])
(3,[email protected])
(4,[email protected])
(5,[email protected])
(6,[email protected])
(7,[email protected])
(8,[email protected])
(9,[email protected])
(10,[email protected])
(11,[email protected])
grunt> joined = JOIN Emprec by $0 , Empemail $0;
2016-03-19 06:39:57,539 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 12
00: <line 3, column 38> mismatched input '$0' expecting BY
Details at logfile: /root/pig_1458369061737.log
grunt> joined = JOIN Emprec by $0 , Empemail by $0;
2016-03-19 06:40:19,152 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 12
00: Pig script failed to parse:
<line 3, column 14> Undefined alias: Emprec
Details at logfile: /root/pig_1458369061737.log
grunt> joined = JOIN Emprec by $0 , Empemail by $0;
2016-03-19 06:42:14,638 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 12
00: Pig script failed to parse:
<line 3, column 14> Undefined alias: Emprec
Details at logfile: /root/pig_1458369061737.log
grunt> Emprec = LOAD '/root/input/EmpRecord.txt' AS (id:int , name:chararray , s
alary:int , location:chararray , DOJ:chararray);
2016-03-19 06:42:31,010 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-03-19 06:42:31,010 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
grunt> joined = JOIN Emprec by $0 , Empemail by $0;
grunt> dunmp joined
2016-03-19 06:42:44,433 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 10
00: Error during parsing. Encountered " <IDENTIFIER> "dunmp "" at line 3, column
1.
Was expecting one of:
<EOF>
"cat" ...
"clear" ...
"fs" ...
"sh" ...
"cd" ...
"cp" ...
"copyFromLocal" ...
"copyToLocal" ...
"dump" ...
"\\d" ...
"describe" ...
"\\de" ...
"aliases" ...
"explain" ...
"\\e" ...
"help" ...
"history" ...
"kill" ...
"ls" ...
"mv" ...

"mkdir" ...
"pwd" ...
"quit" ...
"\\q" ...
"register" ...
"rm" ...
"rmf" ...
"set" ...
"illustrate" ...
"\\i" ...
"run" ...
"exec" ...
"%default" ...
"%declare" ...
"scriptDone" ...
"" ...
"" ...
<EOL> ...
";" ...
Details at logfile: /root/pig_1458369061737.log
grunt> dump filtered
2016-03-19 06:42:55,113 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 10
03: Unable to find an operator for alias filtered
Details at logfile: /root/pig_1458369061737.log
grunt> dump joined
2016-03-19 06:43:10,258 [main] INFO org.apache.pig.tools.pigstats.ScriptState Pig features used in the script: HASH_JOIN
2016-03-19 06:43:10,320 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-03-19 06:43:10,320 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2016-03-19 06:43:10,320 [main] WARN org.apache.pig.data.SchemaTupleBackend - Sc
hemaTupleBackend has already been initialized
2016-03-19 06:43:10,321 [main] INFO org.apache.pig.newplan.logical.optimizer.Lo
gicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, ConstantCalc
ulator, GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeF
ilter, MergeForEach, PartitionFilterOptimizer, PredicatePushdownOptimizer, PushD
ownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter]}
2016-03-19 06:43:10,328 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? fal
se
2016-03-19 06:43:10,337 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MRCompiler$LastInputStreamingOptimizer - Rewrite: POPackage->P
OForEach to POPackage(JoinPackager)
2016-03-19 06:43:10,337 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
2016-03-19 06:43:10,337 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
2016-03-19 06:43:10,347 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-03-19 06:43:10,347 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2016-03-19 06:43:10,348 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 06:43:10,351 [main] INFO org.apache.pig.tools.pigstats.mapreduce.MRS
criptState - Pig script settings are added to the job
2016-03-19 06:43:10,352 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percen

t is not set, set to default 0.3

2016-03-19 06:43:10,353 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - Reduce phase detected, estimating # of re
quired reducers.
2016-03-19 06:43:10,354 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - Using reducer estimator: org.apache.pig.b
ackend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
2016-03-19 06:43:10,355 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.InputSizeReducerEstimator - BytesPerReducer=1000000000 maxRedu
cers=999 totalInputFileSize=637
2016-03-19 06:43:10,355 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - Setting Parallelism to 1
2016-03-19 06:43:10,375 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - Setting up single store job
2016-03-19 06:43:10,376 [main] INFO org.apache.pig.data.SchemaTupleFrontend - K
ey [pig.schematuple] is false, will not generate code.
2016-03-19 06:43:10,376 [main] INFO org.apache.pig.data.SchemaTupleFrontend - S
tarting process to move generated code to distributed cacche
2016-03-19 06:43:10,376 [main] INFO org.apache.pig.data.SchemaTupleFrontend - D
istributed cache not supported or needed in local mode. Setting key [pig.schemat
uple.local.dir] with code temp directory: /tmp/1458369790376-0
2016-03-19 06:43:10,424 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission
.
2016-03-19 06:43:10,426 [JobControl] INFO org.apache.hadoop.metrics.jvm.JvmMetr
ics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - al
ready initialized
2016-03-19 06:43:10,453 [JobControl] WARN org.apache.hadoop.mapreduce.JobResour
ceUploader - No job jar file set. User classes may not be found. See Job or Job
#setJar(String).
2016-03-19 06:43:10,485 [JobControl] INFO org.apache.hadoop.mapreduce.lib.input
.FileInputFormat - Total input paths to process : 1
2016-03-19 06:43:10,488 [JobControl] INFO org.apache.pig.backend.hadoop.executi
onengine.util.MapRedUtil - Total input paths to process : 1
2016-03-19 06:43:10,488 [JobControl] INFO org.apache.pig.backend.hadoop.executi
onengine.util.MapRedUtil - Total input paths (combined) to process : 1
2016-03-19 06:43:10,493 [JobControl] INFO org.apache.hadoop.mapreduce.lib.input
.FileInputFormat - Total input paths to process : 1
2016-03-19 06:43:10,499 [JobControl] INFO org.apache.pig.backend.hadoop.executi
onengine.util.MapRedUtil - Total input paths to process : 1
2016-03-19 06:43:10,499 [JobControl] INFO org.apache.pig.backend.hadoop.executi
onengine.util.MapRedUtil - Total input paths (combined) to process : 1
2016-03-19 06:43:10,511 [JobControl] INFO org.apache.hadoop.mapreduce.JobSubmit
ter - number of splits:2
2016-03-19 06:43:10,586 [JobControl] INFO org.apache.hadoop.mapreduce.JobSubmit
ter - Submitting tokens for job: job_local16493891_0003
2016-03-19 06:43:10,754 [JobControl] INFO org.apache.hadoop.mapreduce.Job - The
url to track the job: http://localhost:8080/
2016-03-19 06:43:10,754 [Thread-11] INFO org.apache.hadoop.mapred.LocalJobRunne
r - OutputCommitter set in config null
2016-03-19 06:43:10,771 [Thread-11] INFO org.apache.hadoop.conf.Configuration.d
eprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-03-19 06:43:10,771 [Thread-11] INFO org.apache.hadoop.conf.Configuration.d
eprecation - mapred.job.reduce.markreset.buffer.percent is deprecated. Instead,
use mapreduce.reduce.markreset.buffer.percent
2016-03-19 06:43:10,771 [Thread-11] INFO org.apache.hadoop.conf.Configuration.d
eprecation - mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduc
es
2016-03-19 06:43:10,772 [Thread-11] INFO org.apache.hadoop.conf.Configuration.d
eprecation - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker

.address
2016-03-19 06:43:10,772 [Thread-11] INFO org.apache.hadoop.conf.Configuration.d
eprecation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-che
cksum
2016-03-19 06:43:10,772 [Thread-11] INFO org.apache.hadoop.mapreduce.lib.output
.FileOutputCommitter - File Output Committer Algorithm version is 1
2016-03-19 06:43:10,772 [Thread-11] INFO org.apache.hadoop.mapred.LocalJobRunne
r - OutputCommitter is org.apache.pig.backend.hadoop.executionengine.mapReduceLa
yer.PigOutputCommitter
2016-03-19 06:43:10,779 [Thread-11] INFO org.apache.hadoop.mapred.LocalJobRunne
r - Waiting for map tasks
2016-03-19 06:43:10,780 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner - Starting task: attempt_local16493891_0003_m_000000
_0
2016-03-19 06:43:10,787 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapreduce.lib.output.FileOutputCommitter - File Output Committer Algorithm
version is 1
2016-03-19 06:43:10,787 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.Task - Using ResourceCalculatorProcessTree : [ ]
2016-03-19 06:43:10,789 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Processing split: Number of splits :1
Total Length = 400
Input split[0]:
Length = 400
ClassName: org.apache.hadoop.mapreduce.lib.input.FileSplit
Locations:
----------------------2016-03-19 06:43:10,795 [LocalJobRunner Map Task Executor #0] INFO org.apache.p
ig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader - Current split
being processed file:/root/input/EmpRecord.txt:0+400
2016-03-19 06:43:10,912 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - (EQUATOR) 0 kvi 26214396(104857584)
2016-03-19 06:43:10,912 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - mapreduce.task.io.sort.mb: 100
2016-03-19 06:43:10,912 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - soft limit at 83886080
2016-03-19 06:43:10,912 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - bufstart = 0; bufvoid = 104857600
2016-03-19 06:43:10,912 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - kvstart = 26214396; length = 6553600
2016-03-19 06:43:10,916 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Map output collector class = org.apache.hadoop.mapred.Map
Task$MapOutputBuffer
2016-03-19 06:43:10,923 [LocalJobRunner Map Task Executor #0] INFO org.apache.p
ig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not gener
ate code.
2016-03-19 06:43:10,925 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_local16493891_0003
2016-03-19 06:43:10,925 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - Processing aliases Empemail,Emprec,joined
2016-03-19 06:43:10,925 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - detailed locations: M: Emprec[3,9],Emprec[
-1,-1],joined[4,9],Empemail[2,11],Empemail[-1,-1],joined[4,9] C: R:
2016-03-19 06:43:10,929 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - 0% complete
2016-03-19 06:43:10,929 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - Running jobs are [job_local16493891_0003]
2016-03-19 06:43:10,939 [LocalJobRunner Map Task Executor #0] INFO org.apache.p

ig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map - Alias
es being processed per job phase (AliasName[line,offset]): M: Emprec[3,9],Emprec
[-1,-1],joined[4,9],Empemail[2,11],Empemail[-1,-1],joined[4,9] C: R:
2016-03-19 06:43:10,943 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner 2016-03-19 06:43:10,943 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Starting flush of map output
2016-03-19 06:43:10,943 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Spilling map output
2016-03-19 06:43:10,943 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - bufstart = 0; bufend = 514; bufvoid = 104857600
2016-03-19 06:43:10,943 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - kvstart = 26214396(104857584); kvend = 26214356(104857424
); length = 41/6553600
2016-03-19 06:43:10,950 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Finished spill 0
2016-03-19 06:43:10,953 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.Task - Task:attempt_local16493891_0003_m_000000_0 is done. And is i
n the process of committing
2016-03-19 06:43:10,955 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner - map
2016-03-19 06:43:10,955 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.Task - Task 'attempt_local16493891_0003_m_000000_0' done.
2016-03-19 06:43:10,955 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner - Finishing task: attempt_local16493891_0003_m_00000
0_0
2016-03-19 06:43:10,955 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner - Starting task: attempt_local16493891_0003_m_000001
_0
2016-03-19 06:43:10,960 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapreduce.lib.output.FileOutputCommitter - File Output Committer Algorithm
version is 1
2016-03-19 06:43:10,962 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.Task - Using ResourceCalculatorProcessTree : [ ]
2016-03-19 06:43:10,964 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Processing split: Number of splits :1
Total Length = 237
Input split[0]:
Length = 237
ClassName: org.apache.hadoop.mapreduce.lib.input.FileSplit
Locations:
----------------------2016-03-19 06:43:10,966 [LocalJobRunner Map Task Executor #0] INFO org.apache.p
ig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader - Current split
being processed file:/root/input/EmpEmailId.txt:0+237
2016-03-19 06:43:11,072 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - (EQUATOR) 0 kvi 26214396(104857584)
2016-03-19 06:43:11,072 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - mapreduce.task.io.sort.mb: 100
2016-03-19 06:43:11,072 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - soft limit at 83886080
2016-03-19 06:43:11,073 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - bufstart = 0; bufvoid = 104857600
2016-03-19 06:43:11,073 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - kvstart = 26214396; length = 6553600
2016-03-19 06:43:11,074 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Map output collector class = org.apache.hadoop.mapred.Map
Task$MapOutputBuffer

2016-03-19 06:43:11,078 [LocalJobRunner Map Task Executor #0] WARN org.apache.p

ig.data.SchemaTupleBackend - SchemaTupleBackend has already been initialized
2016-03-19 06:43:11,088 [LocalJobRunner Map Task Executor #0] INFO org.apache.p
ig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map - Alias
es being processed per job phase (AliasName[line,offset]): M: Emprec[3,9],Emprec
[-1,-1],joined[4,9],Empemail[2,11],Empemail[-1,-1],joined[4,9] C: R:
2016-03-19 06:43:11,091 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner 2016-03-19 06:43:11,091 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Starting flush of map output
2016-03-19 06:43:11,091 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Spilling map output
2016-03-19 06:43:11,091 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - bufstart = 0; bufend = 325; bufvoid = 104857600
2016-03-19 06:43:11,091 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - kvstart = 26214396(104857584); kvend = 26214356(104857424
); length = 41/6553600
2016-03-19 06:43:11,093 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Finished spill 0
2016-03-19 06:43:11,094 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.Task - Task:attempt_local16493891_0003_m_000001_0 is done. And is i
n the process of committing
2016-03-19 06:43:11,095 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner - map
2016-03-19 06:43:11,095 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.Task - Task 'attempt_local16493891_0003_m_000001_0' done.
2016-03-19 06:43:11,095 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner - Finishing task: attempt_local16493891_0003_m_00000
1_0
2016-03-19 06:43:11,095 [Thread-11] INFO org.apache.hadoop.mapred.LocalJobRunne
r - map task executor complete.
2016-03-19 06:43:11,097 [Thread-11] INFO org.apache.hadoop.mapred.LocalJobRunne
r - Waiting for reduce tasks
2016-03-19 06:43:11,098 [pool-5-thread-1] INFO org.apache.hadoop.mapred.LocalJo
bRunner - Starting task: attempt_local16493891_0003_r_000000_0
2016-03-19 06:43:11,110 [pool-5-thread-1] INFO org.apache.hadoop.mapreduce.lib.
output.FileOutputCommitter - File Output Committer Algorithm version is 1
2016-03-19 06:43:11,113 [pool-5-thread-1] INFO org.apache.hadoop.mapred.Task Using ResourceCalculatorProcessTree : [ ]
2016-03-19 06:43:11,120 [pool-5-thread-1] INFO org.apache.hadoop.mapred.ReduceT
ask - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuff
le@2953cd26
2016-03-19 06:43:11,139 [pool-5-thread-1] INFO org.apache.hadoop.mapreduce.task
.reduce.MergeManagerImpl - MergerManager: memoryLimit=709551680, maxSingleShuffl
eLimit=177387920, mergeThreshold=468304128, ioSortFactor=10, memToMemMergeOutput
sThreshold=10
2016-03-19 06:43:11,145 [EventFetcher for fetching Map Completion Events] INFO
org.apache.hadoop.mapreduce.task.reduce.EventFetcher - attempt_local16493891_000
3_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2016-03-19 06:43:11,235 [localfetcher#1] INFO org.apache.hadoop.mapreduce.task.
reduce.LocalFetcher - localfetcher#1 about to shuffle output of map attempt_loca
l16493891_0003_m_000000_0 decomp: 538 len: 542 to MEMORY
2016-03-19 06:43:11,249 [localfetcher#1] INFO org.apache.hadoop.mapreduce.task.
reduce.InMemoryMapOutput - Read 538 bytes from map-output for attempt_local16493
891_0003_m_000000_0
2016-03-19 06:43:11,252 [localfetcher#1] INFO org.apache.hadoop.mapreduce.task.
reduce.MergeManagerImpl - closeInMemoryFile -> map-output of size: 538, inMemory
MapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->538
2016-03-19 06:43:11,257 [localfetcher#1] INFO org.apache.hadoop.mapreduce.task.
reduce.LocalFetcher - localfetcher#1 about to shuffle output of map attempt_loca

l16493891_0003_m_000001_0 decomp: 349 len: 353 to MEMORY

2016-03-19 06:43:11,262 [localfetcher#1] INFO org.apache.hadoop.mapreduce.task.
reduce.InMemoryMapOutput - Read 349 bytes from map-output for attempt_local16493
891_0003_m_000001_0
2016-03-19 06:43:11,262 [localfetcher#1] INFO org.apache.hadoop.mapreduce.task.
reduce.MergeManagerImpl - closeInMemoryFile -> map-output of size: 349, inMemory
MapOutputs.size() -> 2, commitMemory -> 538, usedMemory ->887
2016-03-19 06:43:11,263 [EventFetcher for fetching Map Completion Events] INFO
org.apache.hadoop.mapreduce.task.reduce.EventFetcher - EventFetcher is interrupt
ed.. Returning
2016-03-19 06:43:11,264 [pool-5-thread-1] INFO org.apache.hadoop.mapred.LocalJo
bRunner - 2 / 2 copied.
2016-03-19 06:43:11,264 [pool-5-thread-1] INFO org.apache.hadoop.mapreduce.task
.reduce.MergeManagerImpl - finalMerge called with 2 in-memory map-outputs and 0
on-disk map-outputs
2016-03-19 06:43:11,269 [Readahead Thread #1] WARN org.apache.hadoop.io.Readahe
adPool - Failed readahead on ifile
EBADF: Bad file descriptor
at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(Native Met
hod)
at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(N
ativeIO.java:267)
at org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFa
dviseIfPossible(NativeIO.java:146)
at org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(Readahead
Pool.java:206)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-03-19 06:43:11,284 [pool-5-thread-1] INFO org.apache.hadoop.mapred.Merger
- Merging 2 sorted segments
2016-03-19 06:43:11,285 [pool-5-thread-1] INFO org.apache.hadoop.mapred.Merger
- Down to the last merge-pass, with 2 segments left of total size: 871 bytes
2016-03-19 06:43:11,294 [pool-5-thread-1] INFO org.apache.hadoop.mapreduce.task
.reduce.MergeManagerImpl - Merged 2 segments, 887 bytes to disk to satisfy reduc
e memory limit
2016-03-19 06:43:11,295 [pool-5-thread-1] INFO org.apache.hadoop.mapreduce.task
.reduce.MergeManagerImpl - Merging 1 files, 889 bytes from disk
2016-03-19 06:43:11,295 [pool-5-thread-1] INFO org.apache.hadoop.mapreduce.task
.reduce.MergeManagerImpl - Merging 0 segments, 0 bytes from memory into reduce
2016-03-19 06:43:11,296 [pool-5-thread-1] INFO org.apache.hadoop.mapred.Merger
- Merging 1 sorted segments
2016-03-19 06:43:11,298 [pool-5-thread-1] INFO org.apache.hadoop.mapred.Merger
- Down to the last merge-pass, with 1 segments left of total size: 877 bytes
2016-03-19 06:43:11,299 [pool-5-thread-1] INFO org.apache.hadoop.mapred.LocalJo
bRunner - 2 / 2 copied.
2016-03-19 06:43:11,315 [pool-5-thread-1] INFO org.apache.hadoop.mapreduce.lib.
output.FileOutputCommitter - File Output Committer Algorithm version is 1
2016-03-19 06:43:11,324 [pool-5-thread-1] INFO org.apache.hadoop.conf.Configura
tion.deprecation - mapred.skip.on is deprecated. Instead, use mapreduce.job.skip
records
2016-03-19 06:43:11,341 [pool-5-thread-1] WARN org.apache.pig.data.SchemaTupleB
ackend - SchemaTupleBackend has already been initialized
2016-03-19 06:43:11,356 [pool-5-thread-1] INFO org.apache.pig.backend.hadoop.ex
ecutionengine.mapReduceLayer.PigMapReduce$Reduce - Aliases being processed per j
ob phase (AliasName[line,offset]): M: Emprec[3,9],Emprec[-1,-1],joined[4,9],Empe
mail[2,11],Empemail[-1,-1],joined[4,9] C: R:
2016-03-19 06:43:11,386 [pool-5-thread-1] INFO org.apache.hadoop.mapred.Task -

Task:attempt_local16493891_0003_r_000000_0 is done. And is in the process of com

mitting
2016-03-19 06:43:11,400 [pool-5-thread-1] INFO org.apache.hadoop.mapred.LocalJo
bRunner - 2 / 2 copied.
2016-03-19 06:43:11,400 [pool-5-thread-1] INFO org.apache.hadoop.mapred.Task Task attempt_local16493891_0003_r_000000_0 is allowed to commit now
2016-03-19 06:43:11,413 [pool-5-thread-1] INFO org.apache.hadoop.mapreduce.lib.
output.FileOutputCommitter - Saved output of task 'attempt_local16493891_0003_r_
000000_0' to file:/tmp/temp666787139/tmp568016623/_temporary/0/task_local1649389
1_0003_r_000000
2016-03-19 06:43:11,414 [pool-5-thread-1] INFO org.apache.hadoop.mapred.LocalJo
bRunner - reduce > reduce
2016-03-19 06:43:11,414 [pool-5-thread-1] INFO org.apache.hadoop.mapred.Task Task 'attempt_local16493891_0003_r_000000_0' done.
2016-03-19 06:43:11,414 [pool-5-thread-1] INFO org.apache.hadoop.mapred.LocalJo
bRunner - Finishing task: attempt_local16493891_0003_r_000000_0
2016-03-19 06:43:11,418 [Thread-11] INFO org.apache.hadoop.mapred.LocalJobRunne
r - reduce task executor complete.
2016-03-19 06:43:11,430 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - Running jobs are [job_local16493891_0003]
2016-03-19 06:43:11,563 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 06:43:11,567 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 06:43:11,574 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 06:43:11,596 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - 100% complete
2016-03-19 06:43:11,598 [main] INFO org.apache.pig.tools.pigstats.mapreduce.Sim
plePigStats - Script Statistics:
HadoopVersion PigVersion
UserId StartedAt
FinishedAt
2.7.1 0.15.0 root
2016-03-19 06:43:10
2016-03-19 06:43:11
N

Features
HASH_JOI

Success!
Job Stats (time in seconds):
JobId Maps
Reduces MaxMapTime
MinMapTime
AvgMapTime
MedianMa
pTime MaxReduceTime MinReduceTime AvgReduceTimeMedianReducetime Alias Fe
ature Outputs
job_local16493891_0003 2
1
n/a
n/a
n/a
n/a
n/a
n/a
n/a
n/a
Empemail,Emprec,joined HASH_JOIN
file:/tmp/temp6667
87139/tmp568016623,
Input(s):
Successfully read 11 records from: "/root/input/EmpRecord.txt"
Successfully read 11 records from: "/root/input/EmpEmailId.txt"
Output(s):
Successfully stored 11 records in: "file:/tmp/temp666787139/tmp568016623"
Counters:
Total records written : 11
Total bytes written : 0
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0

Total records proactively spilled: 0

Job DAG:
job_local16493891_0003
2016-03-19 06:43:11,600 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 06:43:11,605 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 06:43:11,611 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 06:43:11,628 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - Success!
2016-03-19 06:43:11,635 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-03-19 06:43:11,638 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2016-03-19 06:43:11,639 [main] WARN org.apache.pig.data.SchemaTupleBackend - Sc
hemaTupleBackend has already been initialized
2016-03-19 06:43:11,678 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileI
nputFormat - Total input paths to process : 1
2016-03-19 06:43:11,678 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.util.MapRedUtil - Total input paths to process : 1
(1,Abraham,11054,New York,10-Jan-20012,1,[email protected])
(2,Jacob,21541,New York,16-Feb-2011,2,[email protected])
(3,John,32541,Washington,22-Mar-2010,3,[email protected])
(4,Marley,43658,Detroit,28-Apr-2009,4,[email protected])
(5,Bob,57854,Michigan,4-May-2008,5,[email protected])
(6,Wincent,65241,California,9-Jun-2007,6,[email protected])
(7,Steve,79854,California,15-Jul-2006,7,[email protected])
(8,Michael,85412,Mexico,21-Aug-2006,8,[email protected])
(9,Stewart,92789,Brazil,29-Sep-2005,9,[email protected])
(10,Daniel,102545,New York,5-Oct-2004,10,[email protected])
(11,Jack,0,Washington,11-Nov-2003,11,[email protected])
grunt> joined1 = COGROUP Emprec by $0 , Empemail by $0;
grunt> dump joined1
2016-03-19 06:44:12,763 [main] INFO org.apache.pig.tools.pigstats.ScriptState Pig features used in the script: COGROUP
2016-03-19 06:44:12,782 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-03-19 06:44:12,782 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2016-03-19 06:44:12,782 [main] WARN org.apache.pig.data.SchemaTupleBackend - Sc
hemaTupleBackend has already been initialized
2016-03-19 06:44:12,783 [main] INFO org.apache.pig.newplan.logical.optimizer.Lo
gicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, ConstantCalc
ulator, GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeF
ilter, MergeForEach, PartitionFilterOptimizer, PredicatePushdownOptimizer, PushD
ownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter]}
2016-03-19 06:44:12,788 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? fal
se
2016-03-19 06:44:12,791 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
2016-03-19 06:44:12,791 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1

2016-03-19 06:44:12,801 [main] INFO org.apache.hadoop.conf.Configuration.deprec

ation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-03-19 06:44:12,801 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2016-03-19 06:44:12,802 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 06:44:12,805 [main] INFO org.apache.pig.tools.pigstats.mapreduce.MRS
criptState - Pig script settings are added to the job
2016-03-19 06:44:12,806 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percen
t is not set, set to default 0.3
2016-03-19 06:44:12,807 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - Reduce phase detected, estimating # of re
quired reducers.
2016-03-19 06:44:12,807 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - Using reducer estimator: org.apache.pig.b
ackend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
2016-03-19 06:44:12,808 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.InputSizeReducerEstimator - BytesPerReducer=1000000000 maxRedu
cers=999 totalInputFileSize=637
2016-03-19 06:44:12,808 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - Setting Parallelism to 1
2016-03-19 06:44:12,818 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - Setting up single store job
2016-03-19 06:44:12,819 [main] INFO org.apache.pig.data.SchemaTupleFrontend - K
ey [pig.schematuple] is false, will not generate code.
2016-03-19 06:44:12,819 [main] INFO org.apache.pig.data.SchemaTupleFrontend - S
tarting process to move generated code to distributed cacche
2016-03-19 06:44:12,819 [main] INFO org.apache.pig.data.SchemaTupleFrontend - D
istributed cache not supported or needed in local mode. Setting key [pig.schemat
uple.local.dir] with code temp directory: /tmp/1458369852818-0
2016-03-19 06:44:12,841 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission
.
2016-03-19 06:44:12,842 [JobControl] INFO org.apache.hadoop.metrics.jvm.JvmMetr
ics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - al
ready initialized
2016-03-19 06:44:12,851 [JobControl] WARN org.apache.hadoop.mapreduce.JobResour
ceUploader - No job jar file set. User classes may not be found. See Job or Job
#setJar(String).
2016-03-19 06:44:12,861 [JobControl] INFO org.apache.hadoop.mapreduce.lib.input
.FileInputFormat - Total input paths to process : 1
2016-03-19 06:44:12,861 [JobControl] INFO org.apache.pig.backend.hadoop.executi
onengine.util.MapRedUtil - Total input paths to process : 1
2016-03-19 06:44:12,862 [JobControl] INFO org.apache.pig.backend.hadoop.executi
onengine.util.MapRedUtil - Total input paths (combined) to process : 1
2016-03-19 06:44:12,869 [JobControl] INFO org.apache.hadoop.mapreduce.lib.input
.FileInputFormat - Total input paths to process : 1
2016-03-19 06:44:12,870 [JobControl] INFO org.apache.pig.backend.hadoop.executi
onengine.util.MapRedUtil - Total input paths to process : 1
2016-03-19 06:44:12,870 [JobControl] INFO org.apache.pig.backend.hadoop.executi
onengine.util.MapRedUtil - Total input paths (combined) to process : 1
2016-03-19 06:44:12,872 [JobControl] INFO org.apache.hadoop.mapreduce.JobSubmit
ter - number of splits:2
2016-03-19 06:44:12,958 [JobControl] INFO org.apache.hadoop.mapreduce.JobSubmit
ter - Submitting tokens for job: job_local1236981714_0004
2016-03-19 06:44:13,047 [JobControl] INFO org.apache.hadoop.mapreduce.Job - The
url to track the job: http://localhost:8080/
2016-03-19 06:44:13,050 [Thread-21] INFO org.apache.hadoop.mapred.LocalJobRunne

r - OutputCommitter set in config null

2016-03-19 06:44:13,062 [Thread-21] INFO org.apache.hadoop.conf.Configuration.d
eprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-03-19 06:44:13,062 [Thread-21] INFO org.apache.hadoop.conf.Configuration.d
eprecation - mapred.job.reduce.markreset.buffer.percent is deprecated. Instead,
use mapreduce.reduce.markreset.buffer.percent
2016-03-19 06:44:13,062 [Thread-21] INFO org.apache.hadoop.conf.Configuration.d
eprecation - mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduc
es
2016-03-19 06:44:13,062 [Thread-21] INFO org.apache.hadoop.conf.Configuration.d
eprecation - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker
.address
2016-03-19 06:44:13,063 [Thread-21] INFO org.apache.hadoop.conf.Configuration.d
eprecation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-che
cksum
2016-03-19 06:44:13,063 [Thread-21] INFO org.apache.hadoop.mapreduce.lib.output
.FileOutputCommitter - File Output Committer Algorithm version is 1
2016-03-19 06:44:13,063 [Thread-21] INFO org.apache.hadoop.mapred.LocalJobRunne
r - OutputCommitter is org.apache.pig.backend.hadoop.executionengine.mapReduceLa
yer.PigOutputCommitter
2016-03-19 06:44:13,067 [Thread-21] INFO org.apache.hadoop.mapred.LocalJobRunne
r - Waiting for map tasks
2016-03-19 06:44:13,068 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner - Starting task: attempt_local1236981714_0004_m_0000
00_0
2016-03-19 06:44:13,078 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapreduce.lib.output.FileOutputCommitter - File Output Committer Algorithm
version is 1
2016-03-19 06:44:13,079 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.Task - Using ResourceCalculatorProcessTree : [ ]
2016-03-19 06:44:13,080 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Processing split: Number of splits :1
Total Length = 400
Input split[0]:
Length = 400
ClassName: org.apache.hadoop.mapreduce.lib.input.FileSplit
Locations:
----------------------2016-03-19 06:44:13,085 [LocalJobRunner Map Task Executor #0] INFO org.apache.p
ig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader - Current split
being processed file:/root/input/EmpRecord.txt:0+400
2016-03-19 06:44:13,162 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - (EQUATOR) 0 kvi 26214396(104857584)
2016-03-19 06:44:13,163 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - mapreduce.task.io.sort.mb: 100
2016-03-19 06:44:13,163 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - soft limit at 83886080
2016-03-19 06:44:13,163 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - bufstart = 0; bufvoid = 104857600
2016-03-19 06:44:13,163 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - kvstart = 26214396; length = 6553600
2016-03-19 06:44:13,164 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Map output collector class = org.apache.hadoop.mapred.Map
Task$MapOutputBuffer
2016-03-19 06:44:13,171 [LocalJobRunner Map Task Executor #0] INFO org.apache.p
ig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not gener
ate code.
2016-03-19 06:44:13,182 [LocalJobRunner Map Task Executor #0] INFO org.apache.p

ig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map - Alias
es being processed per job phase (AliasName[line,offset]): M: Empemail[2,11],Emp
email[-1,-1],joined1[5,10],Emprec[3,9],Emprec[-1,-1],joined1[5,10] C: R:
2016-03-19 06:44:13,185 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner 2016-03-19 06:44:13,185 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Starting flush of map output
2016-03-19 06:44:13,186 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Spilling map output
2016-03-19 06:44:13,186 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - bufstart = 0; bufend = 514; bufvoid = 104857600
2016-03-19 06:44:13,186 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - kvstart = 26214396(104857584); kvend = 26214356(104857424
); length = 41/6553600
2016-03-19 06:44:13,188 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Finished spill 0
2016-03-19 06:44:13,189 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.Task - Task:attempt_local1236981714_0004_m_000000_0 is done. And is
in the process of committing
2016-03-19 06:44:13,190 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner - map
2016-03-19 06:44:13,190 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.Task - Task 'attempt_local1236981714_0004_m_000000_0' done.
2016-03-19 06:44:13,190 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner - Finishing task: attempt_local1236981714_0004_m_000
000_0
2016-03-19 06:44:13,190 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner - Starting task: attempt_local1236981714_0004_m_0000
01_0
2016-03-19 06:44:13,194 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapreduce.lib.output.FileOutputCommitter - File Output Committer Algorithm
version is 1
2016-03-19 06:44:13,195 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.Task - Using ResourceCalculatorProcessTree : [ ]
2016-03-19 06:44:13,196 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Processing split: Number of splits :1
Total Length = 237
Input split[0]:
Length = 237
ClassName: org.apache.hadoop.mapreduce.lib.input.FileSplit
Locations:
----------------------2016-03-19 06:44:13,198 [LocalJobRunner Map Task Executor #0] INFO org.apache.p
ig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader - Current split
being processed file:/root/input/EmpEmailId.txt:0+237
2016-03-19 06:44:13,232 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - (EQUATOR) 0 kvi 26214396(104857584)
2016-03-19 06:44:13,232 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - mapreduce.task.io.sort.mb: 100
2016-03-19 06:44:13,232 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - soft limit at 83886080
2016-03-19 06:44:13,232 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - bufstart = 0; bufvoid = 104857600
2016-03-19 06:44:13,233 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - kvstart = 26214396; length = 6553600
2016-03-19 06:44:13,233 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Map output collector class = org.apache.hadoop.mapred.Map
Task$MapOutputBuffer

2016-03-19 06:44:13,238 [LocalJobRunner Map Task Executor #0] WARN org.apache.p

ig.data.SchemaTupleBackend - SchemaTupleBackend has already been initialized
2016-03-19 06:44:13,246 [LocalJobRunner Map Task Executor #0] INFO org.apache.p
ig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map - Alias
es being processed per job phase (AliasName[line,offset]): M: Empemail[2,11],Emp
email[-1,-1],joined1[5,10],Emprec[3,9],Emprec[-1,-1],joined1[5,10] C: R:
2016-03-19 06:44:13,249 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner 2016-03-19 06:44:13,249 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Starting flush of map output
2016-03-19 06:44:13,249 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Spilling map output
2016-03-19 06:44:13,249 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - bufstart = 0; bufend = 325; bufvoid = 104857600
2016-03-19 06:44:13,250 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - kvstart = 26214396(104857584); kvend = 26214356(104857424
); length = 41/6553600
2016-03-19 06:44:13,251 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Finished spill 0
2016-03-19 06:44:13,252 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.Task - Task:attempt_local1236981714_0004_m_000001_0 is done. And is
in the process of committing
2016-03-19 06:44:13,253 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner - map
2016-03-19 06:44:13,253 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.Task - Task 'attempt_local1236981714_0004_m_000001_0' done.
2016-03-19 06:44:13,253 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner - Finishing task: attempt_local1236981714_0004_m_000
001_0
2016-03-19 06:44:13,254 [Thread-21] INFO org.apache.hadoop.mapred.LocalJobRunne
r - map task executor complete.
2016-03-19 06:44:13,254 [Thread-21] INFO org.apache.hadoop.mapred.LocalJobRunne
r - Waiting for reduce tasks
2016-03-19 06:44:13,254 [pool-10-thread-1] INFO org.apache.hadoop.mapred.LocalJ
obRunner - Starting task: attempt_local1236981714_0004_r_000000_0
2016-03-19 06:44:13,261 [pool-10-thread-1] INFO org.apache.hadoop.mapreduce.lib
.output.FileOutputCommitter - File Output Committer Algorithm version is 1
2016-03-19 06:44:13,265 [pool-10-thread-1] INFO org.apache.hadoop.mapred.Task Using ResourceCalculatorProcessTree : [ ]
2016-03-19 06:44:13,265 [pool-10-thread-1] INFO org.apache.hadoop.mapred.Reduce
Task - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuf
fle@2828e2c5
2016-03-19 06:44:13,267 [pool-10-thread-1] INFO org.apache.hadoop.mapreduce.tas
k.reduce.MergeManagerImpl - MergerManager: memoryLimit=709551680, maxSingleShuff
leLimit=177387920, mergeThreshold=468304128, ioSortFactor=10, memToMemMergeOutpu
tsThreshold=10
2016-03-19 06:44:13,268 [EventFetcher for fetching Map Completion Events] INFO
org.apache.hadoop.mapreduce.task.reduce.EventFetcher - attempt_local1236981714_0
004_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2016-03-19 06:44:13,269 [localfetcher#2] INFO org.apache.hadoop.mapreduce.task.
reduce.LocalFetcher - localfetcher#2 about to shuffle output of map attempt_loca
l1236981714_0004_m_000001_0 decomp: 349 len: 353 to MEMORY
2016-03-19 06:44:13,269 [localfetcher#2] INFO org.apache.hadoop.mapreduce.task.
reduce.InMemoryMapOutput - Read 349 bytes from map-output for attempt_local12369
81714_0004_m_000001_0
2016-03-19 06:44:13,269 [localfetcher#2] INFO org.apache.hadoop.mapreduce.task.
reduce.MergeManagerImpl - closeInMemoryFile -> map-output of size: 349, inMemory
MapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->349
2016-03-19 06:44:13,272 [localfetcher#2] INFO org.apache.hadoop.mapreduce.task.
reduce.LocalFetcher - localfetcher#2 about to shuffle output of map attempt_loca

l1236981714_0004_m_000000_0 decomp: 538 len: 542 to MEMORY

2016-03-19 06:44:13,273 [localfetcher#2] INFO org.apache.hadoop.mapreduce.task.
reduce.InMemoryMapOutput - Read 538 bytes from map-output for attempt_local12369
81714_0004_m_000000_0
2016-03-19 06:44:13,273 [localfetcher#2] INFO org.apache.hadoop.mapreduce.task.
reduce.MergeManagerImpl - closeInMemoryFile -> map-output of size: 538, inMemory
MapOutputs.size() -> 2, commitMemory -> 349, usedMemory ->887
2016-03-19 06:44:13,273 [EventFetcher for fetching Map Completion Events] INFO
org.apache.hadoop.mapreduce.task.reduce.EventFetcher - EventFetcher is interrupt
ed.. Returning
2016-03-19 06:44:13,273 [pool-10-thread-1] INFO org.apache.hadoop.mapred.LocalJ
obRunner - 2 / 2 copied.
2016-03-19 06:44:13,274 [pool-10-thread-1] INFO org.apache.hadoop.mapreduce.tas
k.reduce.MergeManagerImpl - finalMerge called with 2 in-memory map-outputs and 0
on-disk map-outputs
2016-03-19 06:44:13,274 [pool-10-thread-1] INFO org.apache.hadoop.mapred.Merger
- Merging 2 sorted segments
2016-03-19 06:44:13,274 [pool-10-thread-1] INFO org.apache.hadoop.mapred.Merger
- Down to the last merge-pass, with 2 segments left of total size: 871 bytes
2016-03-19 06:44:13,276 [pool-10-thread-1] INFO org.apache.hadoop.mapreduce.tas
k.reduce.MergeManagerImpl - Merged 2 segments, 887 bytes to disk to satisfy redu
ce memory limit
2016-03-19 06:44:13,276 [pool-10-thread-1] INFO org.apache.hadoop.mapreduce.tas
k.reduce.MergeManagerImpl - Merging 1 files, 889 bytes from disk
2016-03-19 06:44:13,276 [pool-10-thread-1] INFO org.apache.hadoop.mapreduce.tas
k.reduce.MergeManagerImpl - Merging 0 segments, 0 bytes from memory into reduce
2016-03-19 06:44:13,276 [pool-10-thread-1] INFO org.apache.hadoop.mapred.Merger
- Merging 1 sorted segments
2016-03-19 06:44:13,276 [pool-10-thread-1] INFO org.apache.hadoop.mapred.Merger
- Down to the last merge-pass, with 1 segments left of total size: 877 bytes
2016-03-19 06:44:13,282 [pool-10-thread-1] INFO org.apache.hadoop.mapred.LocalJ
obRunner - 2 / 2 copied.
2016-03-19 06:44:13,282 [Readahead Thread #3] WARN org.apache.hadoop.io.Readahe
adPool - Failed readahead on ifile
EBADF: Bad file descriptor
at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(Native Met
hod)
at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(N
ativeIO.java:267)
at org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFa
dviseIfPossible(NativeIO.java:146)
at org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(Readahead
Pool.java:206)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-03-19 06:44:13,285 [pool-10-thread-1] INFO org.apache.hadoop.mapreduce.lib
.output.FileOutputCommitter - File Output Committer Algorithm version is 1
2016-03-19 06:44:13,289 [pool-10-thread-1] WARN org.apache.pig.data.SchemaTuple
Backend - SchemaTupleBackend has already been initialized
2016-03-19 06:44:13,316 [pool-10-thread-1] INFO org.apache.pig.backend.hadoop.e
xecutionengine.mapReduceLayer.PigMapReduce$Reduce - Aliases being processed per
job phase (AliasName[line,offset]): M: Empemail[2,11],Empemail[-1,-1],joined1[5,
10],Emprec[3,9],Emprec[-1,-1],joined1[5,10] C: R:
2016-03-19 06:44:13,326 [pool-10-thread-1] INFO org.apache.hadoop.mapred.Task Task:attempt_local1236981714_0004_r_000000_0 is done. And is in the process of
committing
2016-03-19 06:44:13,334 [pool-10-thread-1] INFO org.apache.hadoop.mapred.LocalJ

obRunner - 2 / 2 copied.
2016-03-19 06:44:13,334 [pool-10-thread-1] INFO org.apache.hadoop.mapred.Task Task attempt_local1236981714_0004_r_000000_0 is allowed to commit now
2016-03-19 06:44:13,339 [pool-10-thread-1] INFO org.apache.hadoop.mapreduce.lib
.output.FileOutputCommitter - Saved output of task 'attempt_local1236981714_0004
_r_000000_0' to file:/tmp/temp666787139/tmp-278931343/_temporary/0/task_local123
6981714_0004_r_000000
2016-03-19 06:44:13,340 [pool-10-thread-1] INFO org.apache.hadoop.mapred.LocalJ
obRunner - reduce > reduce
2016-03-19 06:44:13,340 [pool-10-thread-1] INFO org.apache.hadoop.mapred.Task Task 'attempt_local1236981714_0004_r_000000_0' done.
2016-03-19 06:44:13,340 [pool-10-thread-1] INFO org.apache.hadoop.mapred.LocalJ
obRunner - Finishing task: attempt_local1236981714_0004_r_000000_0
2016-03-19 06:44:13,340 [Thread-21] INFO org.apache.hadoop.mapred.LocalJobRunne
r - reduce task executor complete.
2016-03-19 06:44:13,341 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_local1236981714_0004
2016-03-19 06:44:13,341 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - Processing aliases Empemail,Emprec,joined1
2016-03-19 06:44:13,341 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - detailed locations: M: Empemail[2,11],Empe
mail[-1,-1],joined1[5,10],Emprec[3,9],Emprec[-1,-1],joined1[5,10] C: R:
2016-03-19 06:44:13,347 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - Running jobs are [job_local1236981714_0004
]
2016-03-19 06:44:13,551 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 06:44:13,553 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 06:44:13,554 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 06:44:13,561 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - 100% complete
2016-03-19 06:44:13,561 [main] INFO org.apache.pig.tools.pigstats.mapreduce.Sim
plePigStats - Script Statistics:
HadoopVersion PigVersion
UserId StartedAt
FinishedAt
2.7.1 0.15.0 root
2016-03-19 06:44:12
2016-03-19 06:44:13

Features
COGROUP

Success!
Job Stats (time in seconds):
JobId Maps
Reduces MaxMapTime
MinMapTime
AvgMapTime
MedianMa
pTime MaxReduceTime MinReduceTime AvgReduceTimeMedianReducetime Alias Fe
ature Outputs
job_local1236981714_0004
2
1
n/a
n/a
n/a
n/a
n/a
n/a
n/a
n/a
Empemail,Emprec,joined1
COGROUP file:/tmp/
temp666787139/tmp-278931343,
Input(s):
Successfully read 11 records from: "/root/input/EmpEmailId.txt"
Successfully read 11 records from: "/root/input/EmpRecord.txt"
Output(s):
Successfully stored 11 records in: "file:/tmp/temp666787139/tmp-278931343"
Counters:

Total records written : 11

Total bytes written : 0
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0
Job DAG:
job_local1236981714_0004
2016-03-19 06:44:13,563 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 06:44:13,564 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 06:44:13,566 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 06:44:13,573 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - Success!
2016-03-19 06:44:13,574 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-03-19 06:44:13,574 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2016-03-19 06:44:13,574 [main] WARN org.apache.pig.data.SchemaTupleBackend - Sc
hemaTupleBackend has already been initialized
2016-03-19 06:44:13,589 [main] INFO org.apache.hadoop.mapreduce.lib.input.FileI
nputFormat - Total input paths to process : 1
2016-03-19 06:44:13,589 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.util.MapRedUtil - Total input paths to process : 1
(1,{(1,Abraham,11054,New York,10-Jan-20012)},{(1,[email protected])})
(2,{(2,Jacob,21541,New York,16-Feb-2011)},{(2,[email protected])})
(3,{(3,John,32541,Washington,22-Mar-2010)},{(3,[email protected])})
(4,{(4,Marley,43658,Detroit,28-Apr-2009)},{(4,[email protected])})
(5,{(5,Bob,57854,Michigan,4-May-2008)},{(5,[email protected])})
(6,{(6,Wincent,65241,California,9-Jun-2007)},{(6,[email protected])})
(7,{(7,Steve,79854,California,15-Jul-2006)},{(7,[email protected])})
(8,{(8,Michael,85412,Mexico,21-Aug-2006)},{(8,[email protected])})
(9,{(9,Stewart,92789,Brazil,29-Sep-2005)},{(9,[email protected])})
(10,{(10,Daniel,102545,New York,5-Oct-2004)},{(10,[email protected])})
(11,{(11,Jack,0,Washington,11-Nov-2003)},{(11,[email protected])})
grunt>
grunt>
grunt>
grunt>
grunt>
grunt>
grunt>
grunt> dump joined1 $1.$1;
2016-03-19 06:49:20,662 [main] INFO org.apache.pig.tools.pigstats.ScriptState Pig features used in the script: COGROUP
2016-03-19 06:49:20,679 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-03-19 06:49:20,680 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2016-03-19 06:49:20,680 [main] WARN org.apache.pig.data.SchemaTupleBackend - Sc
hemaTupleBackend has already been initialized
2016-03-19 06:49:20,680 [main] INFO org.apache.pig.newplan.logical.optimizer.Lo
gicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, ConstantCalc

ulator, GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeF

ilter, MergeForEach, PartitionFilterOptimizer, PredicatePushdownOptimizer, PushD
ownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter]}
2016-03-19 06:49:20,687 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? fal
se
2016-03-19 06:49:20,698 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
2016-03-19 06:49:20,699 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
2016-03-19 06:49:20,723 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-03-19 06:49:20,724 [main] INFO org.apache.hadoop.conf.Configuration.deprec
ation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2016-03-19 06:49:20,725 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already
initialized
2016-03-19 06:49:20,728 [main] INFO org.apache.pig.tools.pigstats.mapreduce.MRS
criptState - Pig script settings are added to the job
2016-03-19 06:49:20,729 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percen
t is not set, set to default 0.3
2016-03-19 06:49:20,729 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - Reduce phase detected, estimating # of re
quired reducers.
2016-03-19 06:49:20,729 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - Using reducer estimator: org.apache.pig.b
ackend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
2016-03-19 06:49:20,731 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.InputSizeReducerEstimator - BytesPerReducer=1000000000 maxRedu
cers=999 totalInputFileSize=637
2016-03-19 06:49:20,731 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - Setting Parallelism to 1
2016-03-19 06:49:20,753 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.JobControlCompiler - Setting up single store job
2016-03-19 06:49:20,755 [main] INFO org.apache.pig.data.SchemaTupleFrontend - K
ey [pig.schematuple] is false, will not generate code.
2016-03-19 06:49:20,756 [main] INFO org.apache.pig.data.SchemaTupleFrontend - S
tarting process to move generated code to distributed cacche
2016-03-19 06:49:20,756 [main] INFO org.apache.pig.data.SchemaTupleFrontend - D
istributed cache not supported or needed in local mode. Setting key [pig.schemat
uple.local.dir] with code temp directory: /tmp/1458370160755-0
2016-03-19 06:49:20,791 [main] INFO org.apache.pig.backend.hadoop.executionengi
ne.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission
.
2016-03-19 06:49:20,799 [JobControl] INFO org.apache.hadoop.metrics.jvm.JvmMetr
ics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - al
ready initialized
2016-03-19 06:49:20,812 [JobControl] WARN org.apache.hadoop.mapreduce.JobResour
ceUploader - No job jar file set. User classes may not be found. See Job or Job
#setJar(String).
2016-03-19 06:49:20,821 [JobControl] INFO org.apache.hadoop.mapreduce.lib.input
.FileInputFormat - Total input paths to process : 1
2016-03-19 06:49:20,821 [JobControl] INFO org.apache.pig.backend.hadoop.executi
onengine.util.MapRedUtil - Total input paths to process : 1
2016-03-19 06:49:20,821 [JobControl] INFO org.apache.pig.backend.hadoop.executi
onengine.util.MapRedUtil - Total input paths (combined) to process : 1
2016-03-19 06:49:20,825 [JobControl] INFO org.apache.hadoop.mapreduce.lib.input
.FileInputFormat - Total input paths to process : 1
2016-03-19 06:49:20,825 [JobControl] INFO org.apache.pig.backend.hadoop.executi

onengine.util.MapRedUtil - Total input paths to process : 1

2016-03-19 06:49:20,825 [JobControl] INFO org.apache.pig.backend.hadoop.executi
onengine.util.MapRedUtil - Total input paths (combined) to process : 1
2016-03-19 06:49:20,827 [JobControl] INFO org.apache.hadoop.mapreduce.JobSubmit
ter - number of splits:2
2016-03-19 06:49:20,850 [JobControl] INFO org.apache.hadoop.mapreduce.JobSubmit
ter - Submitting tokens for job: job_local2109207482_0005
2016-03-19 06:49:20,946 [JobControl] INFO org.apache.hadoop.mapreduce.Job - The
url to track the job: http://localhost:8080/
2016-03-19 06:49:20,946 [Thread-31] INFO org.apache.hadoop.mapred.LocalJobRunne
r - OutputCommitter set in config null
2016-03-19 06:49:20,952 [Thread-31] INFO org.apache.hadoop.conf.Configuration.d
eprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
2016-03-19 06:49:20,952 [Thread-31] INFO org.apache.hadoop.conf.Configuration.d
eprecation - mapred.job.reduce.markreset.buffer.percent is deprecated. Instead,
use mapreduce.reduce.markreset.buffer.percent
2016-03-19 06:49:20,953 [Thread-31] INFO org.apache.hadoop.conf.Configuration.d
eprecation - mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduc
es
2016-03-19 06:49:20,953 [Thread-31] INFO org.apache.hadoop.conf.Configuration.d
eprecation - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker
.address
2016-03-19 06:49:20,953 [Thread-31] INFO org.apache.hadoop.conf.Configuration.d
eprecation - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-che
cksum
2016-03-19 06:49:20,953 [Thread-31] INFO org.apache.hadoop.mapreduce.lib.output
.FileOutputCommitter - File Output Committer Algorithm version is 1
2016-03-19 06:49:20,953 [Thread-31] INFO org.apache.hadoop.mapred.LocalJobRunne
r - OutputCommitter is org.apache.pig.backend.hadoop.executionengine.mapReduceLa
yer.PigOutputCommitter
2016-03-19 06:49:20,957 [Thread-31] INFO org.apache.hadoop.mapred.LocalJobRunne
r - Waiting for map tasks
2016-03-19 06:49:20,958 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner - Starting task: attempt_local2109207482_0005_m_0000
00_0
2016-03-19 06:49:20,965 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapreduce.lib.output.FileOutputCommitter - File Output Committer Algorithm
version is 1
2016-03-19 06:49:20,965 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.Task - Using ResourceCalculatorProcessTree : [ ]
2016-03-19 06:49:20,967 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Processing split: Number of splits :1
Total Length = 400
Input split[0]:
Length = 400
ClassName: org.apache.hadoop.mapreduce.lib.input.FileSplit
Locations:
----------------------2016-03-19 06:49:20,972 [LocalJobRunner Map Task Executor #0] INFO
ig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader being processed file:/root/input/EmpRecord.txt:0+400
2016-03-19 06:49:20,993 [LocalJobRunner Map Task Executor #0] INFO
adoop.mapred.MapTask - (EQUATOR) 0 kvi 26214396(104857584)
2016-03-19 06:49:20,993 [LocalJobRunner Map Task Executor #0] INFO
adoop.mapred.MapTask - mapreduce.task.io.sort.mb: 100
2016-03-19 06:49:20,993 [LocalJobRunner Map Task Executor #0] INFO
adoop.mapred.MapTask - soft limit at 83886080
2016-03-19 06:49:20,993 [LocalJobRunner Map Task Executor #0] INFO

org.apache.p
Current split
org.apache.h
org.apache.h
org.apache.h
org.apache.h

adoop.mapred.MapTask - bufstart = 0; bufvoid = 104857600

2016-03-19 06:49:20,993 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - kvstart = 26214396; length = 6553600
2016-03-19 06:49:20,994 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Map output collector class = org.apache.hadoop.mapred.Map
Task$MapOutputBuffer
2016-03-19 06:49:21,000 [LocalJobRunner Map Task Executor #0] INFO org.apache.p
ig.data.SchemaTupleBackend - Key [pig.schematuple] was not set... will not gener
ate code.
2016-03-19 06:49:21,013 [LocalJobRunner Map Task Executor #0] INFO org.apache.p
ig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map - Alias
es being processed per job phase (AliasName[line,offset]): M: Empemail[2,11],Emp
email[-1,-1],joined1[5,10],Emprec[3,9],Emprec[-1,-1],joined1[5,10] C: R:
2016-03-19 06:49:21,023 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner 2016-03-19 06:49:21,023 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Starting flush of map output
2016-03-19 06:49:21,023 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Spilling map output
2016-03-19 06:49:21,023 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - bufstart = 0; bufend = 514; bufvoid = 104857600
2016-03-19 06:49:21,023 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - kvstart = 26214396(104857584); kvend = 26214356(104857424
); length = 41/6553600
2016-03-19 06:49:21,026 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Finished spill 0
2016-03-19 06:49:21,027 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.Task - Task:attempt_local2109207482_0005_m_000000_0 is done. And is
in the process of committing
2016-03-19 06:49:21,028 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner - map
2016-03-19 06:49:21,030 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.Task - Task 'attempt_local2109207482_0005_m_000000_0' done.
2016-03-19 06:49:21,030 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner - Finishing task: attempt_local2109207482_0005_m_000
000_0
2016-03-19 06:49:21,030 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.LocalJobRunner - Starting task: attempt_local2109207482_0005_m_0000
01_0
2016-03-19 06:49:21,036 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapreduce.lib.output.FileOutputCommitter - File Output Committer Algorithm
version is 1
2016-03-19 06:49:21,042 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.Task - Using ResourceCalculatorProcessTree : [ ]
2016-03-19 06:49:21,044 [LocalJobRunner Map Task Executor #0] INFO org.apache.h
adoop.mapred.MapTask - Processing split: Number of splits :1
Total Length = 237
Input split[0]:
Length = 237
ClassName: org.apache.hadoop.mapreduce.lib.input.FileSplit
Locations:
----------------------2016-03-19 06:49:21,045 [LocalJobRunner Map Task Executor #0] INFO org.apache.p
ig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader - Current split
being processed file:/root/input/EmpEmailId.txt:0+237
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000dd8fa000, 10
4861696, 0) failed; error='Cannot allocate memory' (errno=12)
#

# There is insufficient memory for the Java Runtime Environment to continue.

# Native memory allocation (malloc) failed to allocate 104861696 bytes for commi
tting reserved memory.
# An error report file with more information is saved as:
# /root/hs_err_pid3506.log
root@ip-172-31-37-176:~#