java - Hadoop Mapper is failing because of "Container killed by the ApplicationMaster" -

-

i trying execute map reduce program on hadoop.

when submit job hadoop single node cluster. job getting created failing message

"container killed applicationmaster"

the input used of size 10 mb.

when used same script of input file 400 kb, got succeded. failing the input file of size 10 mb.

the complete log displayed in terminal follows.

    15/05/29 09:52:16 warn util.nativecodeloader: unable `load native-  hadoop library platform... using builtin-java classes      applicable submitting job on cluster... 15/05/29 09:52:17 info client.rmproxy: connecting resourcemanager @ /0.0.0.0:8032 15/05/29 09:52:18 info input.fileinputformat: total input paths process : 1 15/05/29 09:52:18 info mapreduce.jobsubmitter: number of splits:1 15/05/29 09:52:19 info mapreduce.jobsubmitter: submitting tokens job: job_1432910768528_0001 15/05/29 09:52:19 info impl.yarnclientimpl: submitted application application_1432910768528_0001 15/05/29 09:52:19 info mapreduce.job: url track job: http://localhost:8088/proxy/application_1432910768528_0001/ 15/05/29 09:52:19 info mapreduce.job: running job: job_1432910768528_0001 15/05/29 09:52:29 info mapreduce.job: job job_1432910768528_0001 running in uber mode : false 15/05/29 09:52:29 info mapreduce.job:  map 0% reduce 0% 15/05/29 09:52:41 info mapreduce.job:  map 100% reduce 0% 15/05/29 10:03:01 info mapreduce.job:  map 0% reduce 0% 15/05/29 10:03:01 info mapreduce.job: task id : attempt_1432910768528_0001_m_000000_0, status : failed attemptid:attempt_1432910768528_0001_m_000000_0 timed out after 600 secs container killed applicationmaster. container killed on request. exit code 143 container exited non-zero exit code 143

my mapper here triggering other program going process input file here. program getting trigger mapper consumes lots of memory.

so please me in regard.

include below properties in yarn-site.xml , restart vm,

<property>    <name>yarn.nodemanager.vmem-check-enabled</name>    <value>false</value>    <description>whether virtual memory limits enforced containers</description> </property>  <property>    <name>yarn.nodemanager.vmem-pmem-ratio</name>    <value>4</value>    <description>ratio between virtual memory physical memory when setting memory limits containers</description> </property>

Comments

Post a Comment

Popular posts from this blog

Java 3D LWJGL collision -

-
i making 3d java game lwjgl library, , wondering how add collision detection, player not go through models. i using obj models. here objloader class, loads models: package renderengine; import java.io.bufferedreader; import java.io.file; import java.io.filenotfoundexception; import java.io.filereader; import java.util.arraylist; import java.util.list; import models.rawmodel; import org.lwjgl.util.vector.vector2f; import org.lwjgl.util.vector.vector3f; public class objloader { public static rawmodel loadobjmodel(string filename, loader loader){ filereader fr = null; try { fr = new filereader(new file("res/"+filename+".obj")); } catch (filenotfoundexception e) { system.err.println("couldn't load file!"); e.printstacktrace(); } bufferedreader reader = new bufferedreader(fr); string line; list<vector3f> vertices = new arraylist<vector3f...
Read more

spring - SubProtocolWebSocketHandler - No handlers -

-
i have ugly error during deploying spring app on jboss. 18:11:16,025 error [org.apache.catalina.core.containerbase.[jboss.web].[default-host].[/consumer]] (msc service thread 1-7) exception sending context initialized event listener instance of class org.springframework.web.context.contextloaderlistener: org.springframework.context.applicationcontextexception: failed start bean 'subprotocolwebsockethandler'; nested exception java.lang.illegalargumentexception: no handlers @ org.springframework.context.support.defaultlifecycleprocessor.dostart(defaultlifecycleprocessor.java:176) [spring-context-4.1.3.release.jar:4.1.3.release] @ org.springframework.context.support.defaultlifecycleprocessor.access$200(defaultlifecycleprocessor.java:51) [spring-context-4.1.3.release.jar:4.1.3.release] @ org.springframework.context.support.defaultlifecycleprocessor$lifecyclegroup.start(defaultlifecycleprocessor.java:346) [spring-context-4.1.3.release.jar:4.1.3.release] @ org.s...
Read more

methods - python can't use function in submodule -

-
my folder structure this: pythonstuff/ program.py modulea/ __init__.py foo.py bar.py here's code bar.py : def hello(): print("hello, world!") here's code program.py : #!/usr/bin/env python3 modulea import bar bar.hello() i run $ python3 program.py somehow error: file "program.py", line 3, in <module> bar.hello() attributeerror: 'module' object has no attribute 'hello' edit: __init__.py file empty. edit2: after trying realized had bar.py in root directory contained hello() method. bar.py in modulea/ directory empty. add it import os __all__ = [] module in os.listdir(os.path.dirname(__file__)): if module != '__init__.py' , module[-3:] == '.py': __all__.append(module[:-3]) the init .py files required make python treat directories containing packages; done prevent directories common name, such string, unintentionally hiding v...
Read more