Hadoop - resource manager doesn't start on docker container with stack deploy
0
Hadoop resourcemanger can't connect to namenode. I've deployed hadoop to docker container with docker stack deploy. But when I run containers with docker-compose up it works fine. I want to manage with swarm, that's why need to run with docker stack deploy
docker.compose.yml:
version: "3.3"
services:
namenode:
image: hadoop-namenode:2.7.7
hostname: namenode
volumes:
- ./data/namenode:/hadoop/dfs/name
environment:
- CLUSTER_NAME=test
env_file:
- ./hadoop.env
ports:
- 50070:50070
- 8020:8020
deploy:
replicas: 1
restart_policy:
condition: on-failure
networks:
- hadoopnet
resourcemanager:
image: hadoop-resourcemanager:2.7.7
hostname: resourcemanager
depends_on:
- namenode
env_file:
- ./hadoop.env
ports:
- 8088:8088
deploy:
replicas: 1
restart_policy:
condition: on-failure
networks:
- hadoopnet
networks:
hadoopnet:
external:
name: receiptnet
Error log:
2019-02-01T05:04:52.511935227Z 19/02/01 05:04:52 INFO resourcemanager.ResourceManager: Transitioning to active state
2019-02-01T05:04:52.672344032Z 19/02/01 05:04:52 INFO service.AbstractService: Service org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore failed in state STARTED; cause: java.lang.IllegalArgumentException: java.net.UnknownHostException: namenode
2019-02-01T05:04:52.672363778Z java.lang.IllegalArgumentException: java.net.UnknownHostException: namenode
2019-02-01T05:04:52.672367234Z at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:377)
2019-02-01T05:04:52.672370066Z at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:320)
2019-02-01T05:04:52.672375508Z at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176)
2019-02-01T05:04:52.672378205Z at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:687)
2019-02-01T05:04:52.672381153Z at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:628)
2019-02-01T05:04:52.672383878Z at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149)
2019-02-01T05:04:52.672386707Z at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2667)
2019-02-01T05:04:52.672389357Z at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:93)
2019-02-01T05:04:52.672392026Z at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2701)
2019-02-01T05:04:52.672394684Z at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2683)
2019-02-01T05:04:52.672397232Z at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:372)
2019-02-01T05:04:52.672399864Z at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:171)
2019-02-01T05:04:52.672402470Z at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:356)
2019-02-01T05:04:52.672415836Z at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
2019-02-01T05:04:52.672418809Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore.startInternal(FileSystemRMStateStore.java:141)
2019-02-01T05:04:52.672421594Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.serviceStart(RMStateStore.java:562)
2019-02-01T05:04:52.672424281Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
2019-02-01T05:04:52.672426994Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceStart(ResourceManager.java:564)
2019-02-01T05:04:52.672429918Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
2019-02-01T05:04:52.672432556Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startActiveServices(ResourceManager.java:974)
2019-02-01T05:04:52.672435381Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1015)
2019-02-01T05:04:52.672438983Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1011)
2019-02-01T05:04:52.672441831Z at java.security.AccessController.doPrivileged(Native Method)
2019-02-01T05:04:52.672444365Z at javax.security.auth.Subject.doAs(Subject.java:422)
2019-02-01T05:04:52.672447066Z at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
2019-02-01T05:04:52.672449849Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:1011)
2019-02-01T05:04:52.672452619Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1051)
2019-02-01T05:04:52.672455440Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
2019-02-01T05:04:52.672458129Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1188)
2019-02-01T05:04:52.672460913Z Caused by: java.net.UnknownHostException: namenode
2019-02-01T05:04:52.672463551Z ... 29 more
2019-02-01T05:04:52.672676001Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.686973256Z 19/02/01 05:04:52 WARN service.AbstractService: When stopping the service org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore : java.lang.NullPointerException
2019-02-01T05:04:52.686991741Z java.lang.NullPointerException
2019-02-01T05:04:52.686994730Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore$13.run(FileSystemRMStateStore.java:692)
2019-02-01T05:04:52.686997647Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore$13.run(FileSystemRMStateStore.java:689)
2019-02-01T05:04:52.687000863Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore$FSAction.runWithRetries(FileSystemRMStateStore.java:705)
2019-02-01T05:04:52.687003644Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore.closeWithRetries(FileSystemRMStateStore.java:689)
2019-02-01T05:04:52.687030092Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore.closeInternal(FileSystemRMStateStore.java:149)
2019-02-01T05:04:52.687033922Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.serviceStop(RMStateStore.java:580)
2019-02-01T05:04:52.687036528Z at org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
2019-02-01T05:04:52.687038942Z at org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
2019-02-01T05:04:52.687041285Z at org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:80)
2019-02-01T05:04:52.687043724Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:203)
2019-02-01T05:04:52.687046140Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceStart(ResourceManager.java:564)
2019-02-01T05:04:52.687048649Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
2019-02-01T05:04:52.687051072Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startActiveServices(ResourceManager.java:974)
2019-02-01T05:04:52.687053591Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1015)
2019-02-01T05:04:52.687056124Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1011)
2019-02-01T05:04:52.687058648Z at java.security.AccessController.doPrivileged(Native Method)
2019-02-01T05:04:52.687061014Z at javax.security.auth.Subject.doAs(Subject.java:422)
2019-02-01T05:04:52.687063539Z at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
2019-02-01T05:04:52.687066991Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:1011)
2019-02-01T05:04:52.687069777Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1051)
2019-02-01T05:04:52.687074082Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
2019-02-01T05:04:52.687076615Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1188)
2019-02-01T05:04:52.687154597Z 19/02/01 05:04:52 INFO service.AbstractService: Service RMActiveServices failed in state STARTED; cause: java.lang.IllegalArgumentException: java.net.UnknownHostException: namenode
2019-02-01T05:04:52.687254023Z Caused by: java.net.UnknownHostException: namenode
2019-02-01T05:04:52.687258435Z ... 29 more
2019-02-01T05:04:52.687441964Z 19/02/01 05:04:52 INFO impl.MetricsSystemImpl: Stopping ResourceManager metrics system...
2019-02-01T05:04:52.687731949Z 19/02/01 05:04:52 INFO impl.MetricsSystemImpl: ResourceManager metrics system stopped.
2019-02-01T05:04:52.687835386Z 19/02/01 05:04:52 INFO impl.MetricsSystemImpl: ResourceManager metrics system shutdown complete.
2019-02-01T05:04:52.688442258Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.RMFatalEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMFatalEventDispatcher
2019-02-01T05:04:52.688821697Z 19/02/01 05:04:52 INFO security.NMTokenSecretManagerInRM: NMTokenKeyRollingInterval: 86400000ms and NMTokenKeyActivationDelay: 900000ms
2019-02-01T05:04:52.688998639Z 19/02/01 05:04:52 INFO security.RMContainerTokenSecretManager: ContainerTokenKeyRollingInterval: 86400000ms and ContainerTokenKeyActivationDelay: 900000ms
2019-02-01T05:04:52.691041177Z 19/02/01 05:04:52 INFO security.AMRMTokenSecretManager: AMRMTokenKeyRollingInterval: 86400000ms and AMRMTokenKeyActivationDelay: 900000 ms
2019-02-01T05:04:52.691140559Z 19/02/01 05:04:52 INFO recovery.RMStateStoreFactory: Using RMStateStore implementation - class org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore
2019-02-01T05:04:52.691222539Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreEventType for class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler
2019-02-01T05:04:52.691424692Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.NodesListManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.NodesListManager
2019-02-01T05:04:52.691459962Z 19/02/01 05:04:52 INFO resourcemanager.ResourceManager: Using Scheduler: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler
2019-02-01T05:04:52.691593345Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher
2019-02-01T05:04:52.691620389Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher
2019-02-01T05:04:52.691650575Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationAttemptEventDispatcher
2019-02-01T05:04:52.691681755Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$NodeEventDispatcher
2019-02-01T05:04:52.692718629Z 19/02/01 05:04:52 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2019-02-01T05:04:52.693481063Z 19/02/01 05:04:52 INFO impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2019-02-01T05:04:52.693506892Z 19/02/01 05:04:52 INFO impl.MetricsSystemImpl: ResourceManager metrics system started
2019-02-01T05:04:52.708320956Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.RMAppManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.RMAppManager
2019-02-01T05:04:52.708418732Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEventType for class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher
2019-02-01T05:04:52.708612991Z 19/02/01 05:04:52 WARN util.MBeans: Failed to register MBean "Hadoop:service=ResourceManager,name=RMNMInfo": Instance already exists.
2019-02-01T05:04:52.708630958Z 19/02/01 05:04:52 INFO resourcemanager.RMNMInfo: Registered RMNMInfo MBean
2019-02-01T05:04:52.708727767Z 19/02/01 05:04:52 INFO util.HostsFileReader: Refreshing hosts (include/exclude) list
2019-02-01T05:04:52.709327466Z 19/02/01 05:04:52 INFO conf.Configuration: found resource capacity-scheduler.xml at file:/opt/hadoop-2.7.7/etc/hadoop/capacity-scheduler.xml
2019-02-01T05:04:52.726958313Z 19/02/01 05:04:52 INFO capacity.CapacitySchedulerConfiguration: max alloc mb per queue for root is undefined
2019-02-01T05:04:52.726978624Z 19/02/01 05:04:52 INFO capacity.CapacitySchedulerConfiguration: max alloc vcore per queue for root is undefined
2019-02-01T05:04:52.727081097Z 19/02/01 05:04:52 INFO capacity.ParentQueue: root, capacity=1.0, asboluteCapacity=1.0, maxCapacity=1.0, asboluteMaxCapacity=1.0, state=RUNNING, acls=ADMINISTER_QUEUE:*SUBMIT_APP:*, labels=*,
2019-02-01T05:04:52.727088073Z , reservationsContinueLooking=true
2019-02-01T05:04:52.727104981Z 19/02/01 05:04:52 INFO capacity.ParentQueue: Initialized parent-queue root name=root, fullname=root
2019-02-01T05:04:52.728900615Z 19/02/01 05:04:52 INFO capacity.CapacitySchedulerConfiguration: max alloc mb per queue for root.default is undefined
2019-02-01T05:04:52.728917761Z 19/02/01 05:04:52 INFO capacity.CapacitySchedulerConfiguration: max alloc vcore per queue for root.default is undefined
2019-02-01T05:04:52.729060099Z 19/02/01 05:04:52 INFO capacity.LeafQueue: Initializing default
2019-02-01T05:04:52.729066915Z capacity = 1.0 [= (float) configuredCapacity / 100 ]
2019-02-01T05:04:52.729069853Z asboluteCapacity = 1.0 [= parentAbsoluteCapacity * capacity ]
2019-02-01T05:04:52.729072357Z maxCapacity = 1.0 [= configuredMaxCapacity ]
2019-02-01T05:04:52.729074780Z absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ]
2019-02-01T05:04:52.729077499Z userLimit = 100 [= configuredUserLimit ]
2019-02-01T05:04:52.729079928Z userLimitFactor = 1.0 [= configuredUserLimitFactor ]
2019-02-01T05:04:52.729082468Z maxApplications = 10000 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)]
2019-02-01T05:04:52.729085342Z maxApplicationsPerUser = 10000 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ]
2019-02-01T05:04:52.729097025Z usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)]
2019-02-01T05:04:52.729100156Z absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory]
2019-02-01T05:04:52.729102684Z maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ]
2019-02-01T05:04:52.729105142Z minimumAllocationFactor = 0.875 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ]
2019-02-01T05:04:52.729107582Z maximumAllocation = <memory:8192, vCores:32> [= configuredMaxAllocation ]
2019-02-01T05:04:52.729110196Z numContainers = 0 [= currentNumContainers ]
2019-02-01T05:04:52.729112529Z state = RUNNING [= configuredState ]
2019-02-01T05:04:52.729114912Z acls = ADMINISTER_QUEUE:*SUBMIT_APP:* [= configuredAcls ]
2019-02-01T05:04:52.729117397Z nodeLocalityDelay = 40
2019-02-01T05:04:52.729119763Z labels=*,
2019-02-01T05:04:52.729122044Z nodeLocalityDelay = 40
2019-02-01T05:04:52.729124520Z reservationsContinueLooking = true
2019-02-01T05:04:52.729126850Z preemptionDisabled = true
2019-02-01T05:04:52.729129257Z
2019-02-01T05:04:52.729132934Z 19/02/01 05:04:52 INFO capacity.CapacityScheduler: Initialized queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=<memory:0, vCores:0>, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0
2019-02-01T05:04:52.729189879Z 19/02/01 05:04:52 INFO capacity.CapacityScheduler: Initialized queue: root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=<memory:0, vCores:0>usedCapacity=0.0, numApps=0, numContainers=0
2019-02-01T05:04:52.729256580Z 19/02/01 05:04:52 INFO capacity.CapacityScheduler: Initialized root queue root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=<memory:0, vCores:0>usedCapacity=0.0, numApps=0, numContainers=0
2019-02-01T05:04:52.729281241Z 19/02/01 05:04:52 INFO capacity.CapacityScheduler: Initialized queue mappings, override: false
2019-02-01T05:04:52.729530679Z 19/02/01 05:04:52 INFO capacity.CapacityScheduler: Initialized CapacityScheduler with calculator=class org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator, minimumAllocation=<<memory:1024, vCores:1>>, maximumAllocation=<<memory:8192, vCores:32>>, asynchronousScheduling=false, asyncScheduleInterval=5ms
2019-02-01T05:04:52.730297879Z 19/02/01 05:04:52 INFO service.AbstractService: Service ResourceManager failed in state STARTED; cause: java.lang.IllegalArgumentException: java.net.UnknownHostException: namenode
...
2019-02-01T05:04:52.730415589Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730462887Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730490199Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730530613Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730570231Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730618909Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730654117Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730683974Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730719284Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730754134Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730828723Z 19/02/01 05:04:52 INFO resourcemanager.ResourceManager: Transitioning to standby state
2019-02-01T05:04:52.730860759Z 19/02/01 05:04:52 INFO resourcemanager.ResourceManager: Transitioned to standby state
2019-02-01T05:04:52.731128521Z 19/02/01 05:04:52 FATAL resourcemanager.ResourceManager: Error starting ResourceManager
2019-02-01T05:04:52.731134799Z java.lang.IllegalArgumentException: java.net.UnknownHostException: namenode
2019-02-01T05:04:52.731137638Z at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:377)
2019-02-01T05:04:52.731140203Z at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:320)
2019-02-01T05:04:52.731142680Z at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176)
2019-02-01T05:04:52.731145223Z at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:687)
2019-02-01T05:04:52.731148005Z at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:628)
2019-02-01T05:04:52.731150679Z at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149)
2019-02-01T05:04:52.731153277Z at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2667)
2019-02-01T05:04:52.731155733Z at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:93)
2019-02-01T05:04:52.731158209Z at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2701)
2019-02-01T05:04:52.731160792Z at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2683)
2019-02-01T05:04:52.731167909Z at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:372)
2019-02-01T05:04:52.731170782Z at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:171)
2019-02-01T05:04:52.731173188Z at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:356)
2019-02-01T05:04:52.731175525Z at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
2019-02-01T05:04:52.731177905Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore.startInternal(FileSystemRMStateStore.java:141)
2019-02-01T05:04:52.731180622Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.serviceStart(RMStateStore.java:562)
2019-02-01T05:04:52.731183338Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
2019-02-01T05:04:52.731185899Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceStart(ResourceManager.java:564)
2019-02-01T05:04:52.731188416Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
2019-02-01T05:04:52.731190825Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startActiveServices(ResourceManager.java:974)
2019-02-01T05:04:52.731193465Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1015)
2019-02-01T05:04:52.731196301Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1011)
2019-02-01T05:04:52.731199731Z at java.security.AccessController.doPrivileged(Native Method)
2019-02-01T05:04:52.731202464Z at javax.security.auth.Subject.doAs(Subject.java:422)
2019-02-01T05:04:52.731205034Z at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
2019-02-01T05:04:52.731207624Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:1011)
2019-02-01T05:04:52.731210144Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1051)
2019-02-01T05:04:52.731212614Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
2019-02-01T05:04:52.731215010Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1188)
2019-02-01T05:04:52.731217684Z Caused by: java.net.UnknownHostException: namenode
2019-02-01T05:04:52.731220264Z ... 29 more
2019-02-01T05:04:52.732002339Z 19/02/01 05:04:52 INFO resourcemanager.ResourceManager: SHUTDOWN_MSG:
2019-02-01T05:04:52.732014614Z /************************************************************
2019-02-01T05:04:52.732018144Z SHUTDOWN_MSG: Shutting down ResourceManager at resourcemanager/10.0.5.203
2019-02-01T05:04:52.732020590Z ************************************************************/
docker hadoop stack docker-swarm
asked Feb 1 at 6:48
BehruzBehruz
13
add a comment |
0
Hadoop resourcemanger can't connect to namenode. I've deployed hadoop to docker container with docker stack deploy. But when I run containers with docker-compose up it works fine. I want to manage with swarm, that's why need to run with docker stack deploy
docker.compose.yml:
version: "3.3"
services:
namenode:
image: hadoop-namenode:2.7.7
hostname: namenode
volumes:
- ./data/namenode:/hadoop/dfs/name
environment:
- CLUSTER_NAME=test
env_file:
- ./hadoop.env
ports:
- 50070:50070
- 8020:8020
deploy:
replicas: 1
restart_policy:
condition: on-failure
networks:
- hadoopnet
resourcemanager:
image: hadoop-resourcemanager:2.7.7
hostname: resourcemanager
depends_on:
- namenode
env_file:
- ./hadoop.env
ports:
- 8088:8088
deploy:
replicas: 1
restart_policy:
condition: on-failure
networks:
- hadoopnet
networks:
hadoopnet:
external:
name: receiptnet
Error log:
2019-02-01T05:04:52.511935227Z 19/02/01 05:04:52 INFO resourcemanager.ResourceManager: Transitioning to active state
2019-02-01T05:04:52.672344032Z 19/02/01 05:04:52 INFO service.AbstractService: Service org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore failed in state STARTED; cause: java.lang.IllegalArgumentException: java.net.UnknownHostException: namenode
2019-02-01T05:04:52.672363778Z java.lang.IllegalArgumentException: java.net.UnknownHostException: namenode
2019-02-01T05:04:52.672367234Z at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:377)
2019-02-01T05:04:52.672370066Z at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:320)
2019-02-01T05:04:52.672375508Z at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176)
2019-02-01T05:04:52.672378205Z at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:687)
2019-02-01T05:04:52.672381153Z at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:628)
2019-02-01T05:04:52.672383878Z at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149)
2019-02-01T05:04:52.672386707Z at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2667)
2019-02-01T05:04:52.672389357Z at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:93)
2019-02-01T05:04:52.672392026Z at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2701)
2019-02-01T05:04:52.672394684Z at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2683)
2019-02-01T05:04:52.672397232Z at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:372)
2019-02-01T05:04:52.672399864Z at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:171)
2019-02-01T05:04:52.672402470Z at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:356)
2019-02-01T05:04:52.672415836Z at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
2019-02-01T05:04:52.672418809Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore.startInternal(FileSystemRMStateStore.java:141)
2019-02-01T05:04:52.672421594Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.serviceStart(RMStateStore.java:562)
2019-02-01T05:04:52.672424281Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
2019-02-01T05:04:52.672426994Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceStart(ResourceManager.java:564)
2019-02-01T05:04:52.672429918Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
2019-02-01T05:04:52.672432556Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startActiveServices(ResourceManager.java:974)
2019-02-01T05:04:52.672435381Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1015)
2019-02-01T05:04:52.672438983Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1011)
2019-02-01T05:04:52.672441831Z at java.security.AccessController.doPrivileged(Native Method)
2019-02-01T05:04:52.672444365Z at javax.security.auth.Subject.doAs(Subject.java:422)
2019-02-01T05:04:52.672447066Z at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
2019-02-01T05:04:52.672449849Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:1011)
2019-02-01T05:04:52.672452619Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1051)
2019-02-01T05:04:52.672455440Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
2019-02-01T05:04:52.672458129Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1188)
2019-02-01T05:04:52.672460913Z Caused by: java.net.UnknownHostException: namenode
2019-02-01T05:04:52.672463551Z ... 29 more
2019-02-01T05:04:52.672676001Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.686973256Z 19/02/01 05:04:52 WARN service.AbstractService: When stopping the service org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore : java.lang.NullPointerException
2019-02-01T05:04:52.686991741Z java.lang.NullPointerException
2019-02-01T05:04:52.686994730Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore$13.run(FileSystemRMStateStore.java:692)
2019-02-01T05:04:52.686997647Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore$13.run(FileSystemRMStateStore.java:689)
2019-02-01T05:04:52.687000863Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore$FSAction.runWithRetries(FileSystemRMStateStore.java:705)
2019-02-01T05:04:52.687003644Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore.closeWithRetries(FileSystemRMStateStore.java:689)
2019-02-01T05:04:52.687030092Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore.closeInternal(FileSystemRMStateStore.java:149)
2019-02-01T05:04:52.687033922Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.serviceStop(RMStateStore.java:580)
2019-02-01T05:04:52.687036528Z at org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
2019-02-01T05:04:52.687038942Z at org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
2019-02-01T05:04:52.687041285Z at org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:80)
2019-02-01T05:04:52.687043724Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:203)
2019-02-01T05:04:52.687046140Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceStart(ResourceManager.java:564)
2019-02-01T05:04:52.687048649Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
2019-02-01T05:04:52.687051072Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startActiveServices(ResourceManager.java:974)
2019-02-01T05:04:52.687053591Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1015)
2019-02-01T05:04:52.687056124Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1011)
2019-02-01T05:04:52.687058648Z at java.security.AccessController.doPrivileged(Native Method)
2019-02-01T05:04:52.687061014Z at javax.security.auth.Subject.doAs(Subject.java:422)
2019-02-01T05:04:52.687063539Z at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
2019-02-01T05:04:52.687066991Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:1011)
2019-02-01T05:04:52.687069777Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1051)
2019-02-01T05:04:52.687074082Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
2019-02-01T05:04:52.687076615Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1188)
2019-02-01T05:04:52.687154597Z 19/02/01 05:04:52 INFO service.AbstractService: Service RMActiveServices failed in state STARTED; cause: java.lang.IllegalArgumentException: java.net.UnknownHostException: namenode
2019-02-01T05:04:52.687254023Z Caused by: java.net.UnknownHostException: namenode
2019-02-01T05:04:52.687258435Z ... 29 more
2019-02-01T05:04:52.687441964Z 19/02/01 05:04:52 INFO impl.MetricsSystemImpl: Stopping ResourceManager metrics system...
2019-02-01T05:04:52.687731949Z 19/02/01 05:04:52 INFO impl.MetricsSystemImpl: ResourceManager metrics system stopped.
2019-02-01T05:04:52.687835386Z 19/02/01 05:04:52 INFO impl.MetricsSystemImpl: ResourceManager metrics system shutdown complete.
2019-02-01T05:04:52.688442258Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.RMFatalEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMFatalEventDispatcher
2019-02-01T05:04:52.688821697Z 19/02/01 05:04:52 INFO security.NMTokenSecretManagerInRM: NMTokenKeyRollingInterval: 86400000ms and NMTokenKeyActivationDelay: 900000ms
2019-02-01T05:04:52.688998639Z 19/02/01 05:04:52 INFO security.RMContainerTokenSecretManager: ContainerTokenKeyRollingInterval: 86400000ms and ContainerTokenKeyActivationDelay: 900000ms
2019-02-01T05:04:52.691041177Z 19/02/01 05:04:52 INFO security.AMRMTokenSecretManager: AMRMTokenKeyRollingInterval: 86400000ms and AMRMTokenKeyActivationDelay: 900000 ms
2019-02-01T05:04:52.691140559Z 19/02/01 05:04:52 INFO recovery.RMStateStoreFactory: Using RMStateStore implementation - class org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore
2019-02-01T05:04:52.691222539Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreEventType for class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler
2019-02-01T05:04:52.691424692Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.NodesListManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.NodesListManager
2019-02-01T05:04:52.691459962Z 19/02/01 05:04:52 INFO resourcemanager.ResourceManager: Using Scheduler: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler
2019-02-01T05:04:52.691593345Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher
2019-02-01T05:04:52.691620389Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher
2019-02-01T05:04:52.691650575Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationAttemptEventDispatcher
2019-02-01T05:04:52.691681755Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$NodeEventDispatcher
2019-02-01T05:04:52.692718629Z 19/02/01 05:04:52 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2019-02-01T05:04:52.693481063Z 19/02/01 05:04:52 INFO impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2019-02-01T05:04:52.693506892Z 19/02/01 05:04:52 INFO impl.MetricsSystemImpl: ResourceManager metrics system started
2019-02-01T05:04:52.708320956Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.RMAppManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.RMAppManager
2019-02-01T05:04:52.708418732Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEventType for class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher
2019-02-01T05:04:52.708612991Z 19/02/01 05:04:52 WARN util.MBeans: Failed to register MBean "Hadoop:service=ResourceManager,name=RMNMInfo": Instance already exists.
2019-02-01T05:04:52.708630958Z 19/02/01 05:04:52 INFO resourcemanager.RMNMInfo: Registered RMNMInfo MBean
2019-02-01T05:04:52.708727767Z 19/02/01 05:04:52 INFO util.HostsFileReader: Refreshing hosts (include/exclude) list
2019-02-01T05:04:52.709327466Z 19/02/01 05:04:52 INFO conf.Configuration: found resource capacity-scheduler.xml at file:/opt/hadoop-2.7.7/etc/hadoop/capacity-scheduler.xml
2019-02-01T05:04:52.726958313Z 19/02/01 05:04:52 INFO capacity.CapacitySchedulerConfiguration: max alloc mb per queue for root is undefined
2019-02-01T05:04:52.726978624Z 19/02/01 05:04:52 INFO capacity.CapacitySchedulerConfiguration: max alloc vcore per queue for root is undefined
2019-02-01T05:04:52.727081097Z 19/02/01 05:04:52 INFO capacity.ParentQueue: root, capacity=1.0, asboluteCapacity=1.0, maxCapacity=1.0, asboluteMaxCapacity=1.0, state=RUNNING, acls=ADMINISTER_QUEUE:*SUBMIT_APP:*, labels=*,
2019-02-01T05:04:52.727088073Z , reservationsContinueLooking=true
2019-02-01T05:04:52.727104981Z 19/02/01 05:04:52 INFO capacity.ParentQueue: Initialized parent-queue root name=root, fullname=root
2019-02-01T05:04:52.728900615Z 19/02/01 05:04:52 INFO capacity.CapacitySchedulerConfiguration: max alloc mb per queue for root.default is undefined
2019-02-01T05:04:52.728917761Z 19/02/01 05:04:52 INFO capacity.CapacitySchedulerConfiguration: max alloc vcore per queue for root.default is undefined
2019-02-01T05:04:52.729060099Z 19/02/01 05:04:52 INFO capacity.LeafQueue: Initializing default
2019-02-01T05:04:52.729066915Z capacity = 1.0 [= (float) configuredCapacity / 100 ]
2019-02-01T05:04:52.729069853Z asboluteCapacity = 1.0 [= parentAbsoluteCapacity * capacity ]
2019-02-01T05:04:52.729072357Z maxCapacity = 1.0 [= configuredMaxCapacity ]
2019-02-01T05:04:52.729074780Z absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ]
2019-02-01T05:04:52.729077499Z userLimit = 100 [= configuredUserLimit ]
2019-02-01T05:04:52.729079928Z userLimitFactor = 1.0 [= configuredUserLimitFactor ]
2019-02-01T05:04:52.729082468Z maxApplications = 10000 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)]
2019-02-01T05:04:52.729085342Z maxApplicationsPerUser = 10000 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ]
2019-02-01T05:04:52.729097025Z usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)]
2019-02-01T05:04:52.729100156Z absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory]
2019-02-01T05:04:52.729102684Z maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ]
2019-02-01T05:04:52.729105142Z minimumAllocationFactor = 0.875 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ]
2019-02-01T05:04:52.729107582Z maximumAllocation = <memory:8192, vCores:32> [= configuredMaxAllocation ]
2019-02-01T05:04:52.729110196Z numContainers = 0 [= currentNumContainers ]
2019-02-01T05:04:52.729112529Z state = RUNNING [= configuredState ]
2019-02-01T05:04:52.729114912Z acls = ADMINISTER_QUEUE:*SUBMIT_APP:* [= configuredAcls ]
2019-02-01T05:04:52.729117397Z nodeLocalityDelay = 40
2019-02-01T05:04:52.729119763Z labels=*,
2019-02-01T05:04:52.729122044Z nodeLocalityDelay = 40
2019-02-01T05:04:52.729124520Z reservationsContinueLooking = true
2019-02-01T05:04:52.729126850Z preemptionDisabled = true
2019-02-01T05:04:52.729129257Z
2019-02-01T05:04:52.729132934Z 19/02/01 05:04:52 INFO capacity.CapacityScheduler: Initialized queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=<memory:0, vCores:0>, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0
2019-02-01T05:04:52.729189879Z 19/02/01 05:04:52 INFO capacity.CapacityScheduler: Initialized queue: root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=<memory:0, vCores:0>usedCapacity=0.0, numApps=0, numContainers=0
2019-02-01T05:04:52.729256580Z 19/02/01 05:04:52 INFO capacity.CapacityScheduler: Initialized root queue root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=<memory:0, vCores:0>usedCapacity=0.0, numApps=0, numContainers=0
2019-02-01T05:04:52.729281241Z 19/02/01 05:04:52 INFO capacity.CapacityScheduler: Initialized queue mappings, override: false
2019-02-01T05:04:52.729530679Z 19/02/01 05:04:52 INFO capacity.CapacityScheduler: Initialized CapacityScheduler with calculator=class org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator, minimumAllocation=<<memory:1024, vCores:1>>, maximumAllocation=<<memory:8192, vCores:32>>, asynchronousScheduling=false, asyncScheduleInterval=5ms
2019-02-01T05:04:52.730297879Z 19/02/01 05:04:52 INFO service.AbstractService: Service ResourceManager failed in state STARTED; cause: java.lang.IllegalArgumentException: java.net.UnknownHostException: namenode
...
2019-02-01T05:04:52.730415589Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730462887Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730490199Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730530613Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730570231Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730618909Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730654117Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730683974Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730719284Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730754134Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730828723Z 19/02/01 05:04:52 INFO resourcemanager.ResourceManager: Transitioning to standby state
2019-02-01T05:04:52.730860759Z 19/02/01 05:04:52 INFO resourcemanager.ResourceManager: Transitioned to standby state
2019-02-01T05:04:52.731128521Z 19/02/01 05:04:52 FATAL resourcemanager.ResourceManager: Error starting ResourceManager
2019-02-01T05:04:52.731134799Z java.lang.IllegalArgumentException: java.net.UnknownHostException: namenode
2019-02-01T05:04:52.731137638Z at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:377)
2019-02-01T05:04:52.731140203Z at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:320)
2019-02-01T05:04:52.731142680Z at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176)
2019-02-01T05:04:52.731145223Z at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:687)
2019-02-01T05:04:52.731148005Z at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:628)
2019-02-01T05:04:52.731150679Z at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149)
2019-02-01T05:04:52.731153277Z at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2667)
2019-02-01T05:04:52.731155733Z at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:93)
2019-02-01T05:04:52.731158209Z at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2701)
2019-02-01T05:04:52.731160792Z at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2683)
2019-02-01T05:04:52.731167909Z at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:372)
2019-02-01T05:04:52.731170782Z at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:171)
2019-02-01T05:04:52.731173188Z at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:356)
2019-02-01T05:04:52.731175525Z at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
2019-02-01T05:04:52.731177905Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore.startInternal(FileSystemRMStateStore.java:141)
2019-02-01T05:04:52.731180622Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.serviceStart(RMStateStore.java:562)
2019-02-01T05:04:52.731183338Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
2019-02-01T05:04:52.731185899Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceStart(ResourceManager.java:564)
2019-02-01T05:04:52.731188416Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
2019-02-01T05:04:52.731190825Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startActiveServices(ResourceManager.java:974)
2019-02-01T05:04:52.731193465Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1015)
2019-02-01T05:04:52.731196301Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1011)
2019-02-01T05:04:52.731199731Z at java.security.AccessController.doPrivileged(Native Method)
2019-02-01T05:04:52.731202464Z at javax.security.auth.Subject.doAs(Subject.java:422)
2019-02-01T05:04:52.731205034Z at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
2019-02-01T05:04:52.731207624Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:1011)
2019-02-01T05:04:52.731210144Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1051)
2019-02-01T05:04:52.731212614Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
2019-02-01T05:04:52.731215010Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1188)
2019-02-01T05:04:52.731217684Z Caused by: java.net.UnknownHostException: namenode
2019-02-01T05:04:52.731220264Z ... 29 more
2019-02-01T05:04:52.732002339Z 19/02/01 05:04:52 INFO resourcemanager.ResourceManager: SHUTDOWN_MSG:
2019-02-01T05:04:52.732014614Z /************************************************************
2019-02-01T05:04:52.732018144Z SHUTDOWN_MSG: Shutting down ResourceManager at resourcemanager/10.0.5.203
2019-02-01T05:04:52.732020590Z ************************************************************/
docker hadoop stack docker-swarm
asked Feb 1 at 6:48
BehruzBehruz
13
add a comment |
0
0
0
Hadoop resourcemanger can't connect to namenode. I've deployed hadoop to docker container with docker stack deploy. But when I run containers with docker-compose up it works fine. I want to manage with swarm, that's why need to run with docker stack deploy
docker.compose.yml:
version: "3.3"
services:
namenode:
image: hadoop-namenode:2.7.7
hostname: namenode
volumes:
- ./data/namenode:/hadoop/dfs/name
environment:
- CLUSTER_NAME=test
env_file:
- ./hadoop.env
ports:
- 50070:50070
- 8020:8020
deploy:
replicas: 1
restart_policy:
condition: on-failure
networks:
- hadoopnet
resourcemanager:
image: hadoop-resourcemanager:2.7.7
hostname: resourcemanager
depends_on:
- namenode
env_file:
- ./hadoop.env
ports:
- 8088:8088
deploy:
replicas: 1
restart_policy:
condition: on-failure
networks:
- hadoopnet
networks:
hadoopnet:
external:
name: receiptnet
Error log:
2019-02-01T05:04:52.511935227Z 19/02/01 05:04:52 INFO resourcemanager.ResourceManager: Transitioning to active state
2019-02-01T05:04:52.672344032Z 19/02/01 05:04:52 INFO service.AbstractService: Service org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore failed in state STARTED; cause: java.lang.IllegalArgumentException: java.net.UnknownHostException: namenode
2019-02-01T05:04:52.672363778Z java.lang.IllegalArgumentException: java.net.UnknownHostException: namenode
2019-02-01T05:04:52.672367234Z at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:377)
2019-02-01T05:04:52.672370066Z at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:320)
2019-02-01T05:04:52.672375508Z at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176)
2019-02-01T05:04:52.672378205Z at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:687)
2019-02-01T05:04:52.672381153Z at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:628)
2019-02-01T05:04:52.672383878Z at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149)
2019-02-01T05:04:52.672386707Z at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2667)
2019-02-01T05:04:52.672389357Z at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:93)
2019-02-01T05:04:52.672392026Z at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2701)
2019-02-01T05:04:52.672394684Z at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2683)
2019-02-01T05:04:52.672397232Z at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:372)
2019-02-01T05:04:52.672399864Z at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:171)
2019-02-01T05:04:52.672402470Z at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:356)
2019-02-01T05:04:52.672415836Z at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
2019-02-01T05:04:52.672418809Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore.startInternal(FileSystemRMStateStore.java:141)
2019-02-01T05:04:52.672421594Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.serviceStart(RMStateStore.java:562)
2019-02-01T05:04:52.672424281Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
2019-02-01T05:04:52.672426994Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceStart(ResourceManager.java:564)
2019-02-01T05:04:52.672429918Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
2019-02-01T05:04:52.672432556Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startActiveServices(ResourceManager.java:974)
2019-02-01T05:04:52.672435381Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1015)
2019-02-01T05:04:52.672438983Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1011)
2019-02-01T05:04:52.672441831Z at java.security.AccessController.doPrivileged(Native Method)
2019-02-01T05:04:52.672444365Z at javax.security.auth.Subject.doAs(Subject.java:422)
2019-02-01T05:04:52.672447066Z at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
2019-02-01T05:04:52.672449849Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:1011)
2019-02-01T05:04:52.672452619Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1051)
2019-02-01T05:04:52.672455440Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
2019-02-01T05:04:52.672458129Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1188)
2019-02-01T05:04:52.672460913Z Caused by: java.net.UnknownHostException: namenode
2019-02-01T05:04:52.672463551Z ... 29 more
2019-02-01T05:04:52.672676001Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.686973256Z 19/02/01 05:04:52 WARN service.AbstractService: When stopping the service org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore : java.lang.NullPointerException
2019-02-01T05:04:52.686991741Z java.lang.NullPointerException
2019-02-01T05:04:52.686994730Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore$13.run(FileSystemRMStateStore.java:692)
2019-02-01T05:04:52.686997647Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore$13.run(FileSystemRMStateStore.java:689)
2019-02-01T05:04:52.687000863Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore$FSAction.runWithRetries(FileSystemRMStateStore.java:705)
2019-02-01T05:04:52.687003644Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore.closeWithRetries(FileSystemRMStateStore.java:689)
2019-02-01T05:04:52.687030092Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore.closeInternal(FileSystemRMStateStore.java:149)
2019-02-01T05:04:52.687033922Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.serviceStop(RMStateStore.java:580)
2019-02-01T05:04:52.687036528Z at org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
2019-02-01T05:04:52.687038942Z at org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
2019-02-01T05:04:52.687041285Z at org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:80)
2019-02-01T05:04:52.687043724Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:203)
2019-02-01T05:04:52.687046140Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceStart(ResourceManager.java:564)
2019-02-01T05:04:52.687048649Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
2019-02-01T05:04:52.687051072Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startActiveServices(ResourceManager.java:974)
2019-02-01T05:04:52.687053591Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1015)
2019-02-01T05:04:52.687056124Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1011)
2019-02-01T05:04:52.687058648Z at java.security.AccessController.doPrivileged(Native Method)
2019-02-01T05:04:52.687061014Z at javax.security.auth.Subject.doAs(Subject.java:422)
2019-02-01T05:04:52.687063539Z at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
2019-02-01T05:04:52.687066991Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:1011)
2019-02-01T05:04:52.687069777Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1051)
2019-02-01T05:04:52.687074082Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
2019-02-01T05:04:52.687076615Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1188)
2019-02-01T05:04:52.687154597Z 19/02/01 05:04:52 INFO service.AbstractService: Service RMActiveServices failed in state STARTED; cause: java.lang.IllegalArgumentException: java.net.UnknownHostException: namenode
2019-02-01T05:04:52.687254023Z Caused by: java.net.UnknownHostException: namenode
2019-02-01T05:04:52.687258435Z ... 29 more
2019-02-01T05:04:52.687441964Z 19/02/01 05:04:52 INFO impl.MetricsSystemImpl: Stopping ResourceManager metrics system...
2019-02-01T05:04:52.687731949Z 19/02/01 05:04:52 INFO impl.MetricsSystemImpl: ResourceManager metrics system stopped.
2019-02-01T05:04:52.687835386Z 19/02/01 05:04:52 INFO impl.MetricsSystemImpl: ResourceManager metrics system shutdown complete.
2019-02-01T05:04:52.688442258Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.RMFatalEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMFatalEventDispatcher
2019-02-01T05:04:52.688821697Z 19/02/01 05:04:52 INFO security.NMTokenSecretManagerInRM: NMTokenKeyRollingInterval: 86400000ms and NMTokenKeyActivationDelay: 900000ms
2019-02-01T05:04:52.688998639Z 19/02/01 05:04:52 INFO security.RMContainerTokenSecretManager: ContainerTokenKeyRollingInterval: 86400000ms and ContainerTokenKeyActivationDelay: 900000ms
2019-02-01T05:04:52.691041177Z 19/02/01 05:04:52 INFO security.AMRMTokenSecretManager: AMRMTokenKeyRollingInterval: 86400000ms and AMRMTokenKeyActivationDelay: 900000 ms
2019-02-01T05:04:52.691140559Z 19/02/01 05:04:52 INFO recovery.RMStateStoreFactory: Using RMStateStore implementation - class org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore
2019-02-01T05:04:52.691222539Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreEventType for class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler
2019-02-01T05:04:52.691424692Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.NodesListManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.NodesListManager
2019-02-01T05:04:52.691459962Z 19/02/01 05:04:52 INFO resourcemanager.ResourceManager: Using Scheduler: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler
2019-02-01T05:04:52.691593345Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher
2019-02-01T05:04:52.691620389Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher
2019-02-01T05:04:52.691650575Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationAttemptEventDispatcher
2019-02-01T05:04:52.691681755Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$NodeEventDispatcher
2019-02-01T05:04:52.692718629Z 19/02/01 05:04:52 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2019-02-01T05:04:52.693481063Z 19/02/01 05:04:52 INFO impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2019-02-01T05:04:52.693506892Z 19/02/01 05:04:52 INFO impl.MetricsSystemImpl: ResourceManager metrics system started
2019-02-01T05:04:52.708320956Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.RMAppManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.RMAppManager
2019-02-01T05:04:52.708418732Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEventType for class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher
2019-02-01T05:04:52.708612991Z 19/02/01 05:04:52 WARN util.MBeans: Failed to register MBean "Hadoop:service=ResourceManager,name=RMNMInfo": Instance already exists.
2019-02-01T05:04:52.708630958Z 19/02/01 05:04:52 INFO resourcemanager.RMNMInfo: Registered RMNMInfo MBean
2019-02-01T05:04:52.708727767Z 19/02/01 05:04:52 INFO util.HostsFileReader: Refreshing hosts (include/exclude) list
2019-02-01T05:04:52.709327466Z 19/02/01 05:04:52 INFO conf.Configuration: found resource capacity-scheduler.xml at file:/opt/hadoop-2.7.7/etc/hadoop/capacity-scheduler.xml
2019-02-01T05:04:52.726958313Z 19/02/01 05:04:52 INFO capacity.CapacitySchedulerConfiguration: max alloc mb per queue for root is undefined
2019-02-01T05:04:52.726978624Z 19/02/01 05:04:52 INFO capacity.CapacitySchedulerConfiguration: max alloc vcore per queue for root is undefined
2019-02-01T05:04:52.727081097Z 19/02/01 05:04:52 INFO capacity.ParentQueue: root, capacity=1.0, asboluteCapacity=1.0, maxCapacity=1.0, asboluteMaxCapacity=1.0, state=RUNNING, acls=ADMINISTER_QUEUE:*SUBMIT_APP:*, labels=*,
2019-02-01T05:04:52.727088073Z , reservationsContinueLooking=true
2019-02-01T05:04:52.727104981Z 19/02/01 05:04:52 INFO capacity.ParentQueue: Initialized parent-queue root name=root, fullname=root
2019-02-01T05:04:52.728900615Z 19/02/01 05:04:52 INFO capacity.CapacitySchedulerConfiguration: max alloc mb per queue for root.default is undefined
2019-02-01T05:04:52.728917761Z 19/02/01 05:04:52 INFO capacity.CapacitySchedulerConfiguration: max alloc vcore per queue for root.default is undefined
2019-02-01T05:04:52.729060099Z 19/02/01 05:04:52 INFO capacity.LeafQueue: Initializing default
2019-02-01T05:04:52.729066915Z capacity = 1.0 [= (float) configuredCapacity / 100 ]
2019-02-01T05:04:52.729069853Z asboluteCapacity = 1.0 [= parentAbsoluteCapacity * capacity ]
2019-02-01T05:04:52.729072357Z maxCapacity = 1.0 [= configuredMaxCapacity ]
2019-02-01T05:04:52.729074780Z absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ]
2019-02-01T05:04:52.729077499Z userLimit = 100 [= configuredUserLimit ]
2019-02-01T05:04:52.729079928Z userLimitFactor = 1.0 [= configuredUserLimitFactor ]
2019-02-01T05:04:52.729082468Z maxApplications = 10000 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)]
2019-02-01T05:04:52.729085342Z maxApplicationsPerUser = 10000 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ]
2019-02-01T05:04:52.729097025Z usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)]
2019-02-01T05:04:52.729100156Z absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory]
2019-02-01T05:04:52.729102684Z maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ]
2019-02-01T05:04:52.729105142Z minimumAllocationFactor = 0.875 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ]
2019-02-01T05:04:52.729107582Z maximumAllocation = <memory:8192, vCores:32> [= configuredMaxAllocation ]
2019-02-01T05:04:52.729110196Z numContainers = 0 [= currentNumContainers ]
2019-02-01T05:04:52.729112529Z state = RUNNING [= configuredState ]
2019-02-01T05:04:52.729114912Z acls = ADMINISTER_QUEUE:*SUBMIT_APP:* [= configuredAcls ]
2019-02-01T05:04:52.729117397Z nodeLocalityDelay = 40
2019-02-01T05:04:52.729119763Z labels=*,
2019-02-01T05:04:52.729122044Z nodeLocalityDelay = 40
2019-02-01T05:04:52.729124520Z reservationsContinueLooking = true
2019-02-01T05:04:52.729126850Z preemptionDisabled = true
2019-02-01T05:04:52.729129257Z
2019-02-01T05:04:52.729132934Z 19/02/01 05:04:52 INFO capacity.CapacityScheduler: Initialized queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=<memory:0, vCores:0>, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0
2019-02-01T05:04:52.729189879Z 19/02/01 05:04:52 INFO capacity.CapacityScheduler: Initialized queue: root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=<memory:0, vCores:0>usedCapacity=0.0, numApps=0, numContainers=0
2019-02-01T05:04:52.729256580Z 19/02/01 05:04:52 INFO capacity.CapacityScheduler: Initialized root queue root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=<memory:0, vCores:0>usedCapacity=0.0, numApps=0, numContainers=0
2019-02-01T05:04:52.729281241Z 19/02/01 05:04:52 INFO capacity.CapacityScheduler: Initialized queue mappings, override: false
2019-02-01T05:04:52.729530679Z 19/02/01 05:04:52 INFO capacity.CapacityScheduler: Initialized CapacityScheduler with calculator=class org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator, minimumAllocation=<<memory:1024, vCores:1>>, maximumAllocation=<<memory:8192, vCores:32>>, asynchronousScheduling=false, asyncScheduleInterval=5ms
2019-02-01T05:04:52.730297879Z 19/02/01 05:04:52 INFO service.AbstractService: Service ResourceManager failed in state STARTED; cause: java.lang.IllegalArgumentException: java.net.UnknownHostException: namenode
...
2019-02-01T05:04:52.730415589Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730462887Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730490199Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730530613Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730570231Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730618909Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730654117Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730683974Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730719284Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730754134Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730828723Z 19/02/01 05:04:52 INFO resourcemanager.ResourceManager: Transitioning to standby state
2019-02-01T05:04:52.730860759Z 19/02/01 05:04:52 INFO resourcemanager.ResourceManager: Transitioned to standby state
2019-02-01T05:04:52.731128521Z 19/02/01 05:04:52 FATAL resourcemanager.ResourceManager: Error starting ResourceManager
2019-02-01T05:04:52.731134799Z java.lang.IllegalArgumentException: java.net.UnknownHostException: namenode
2019-02-01T05:04:52.731137638Z at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:377)
2019-02-01T05:04:52.731140203Z at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:320)
2019-02-01T05:04:52.731142680Z at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176)
2019-02-01T05:04:52.731145223Z at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:687)
2019-02-01T05:04:52.731148005Z at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:628)
2019-02-01T05:04:52.731150679Z at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149)
2019-02-01T05:04:52.731153277Z at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2667)
2019-02-01T05:04:52.731155733Z at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:93)
2019-02-01T05:04:52.731158209Z at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2701)
2019-02-01T05:04:52.731160792Z at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2683)
2019-02-01T05:04:52.731167909Z at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:372)
2019-02-01T05:04:52.731170782Z at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:171)
2019-02-01T05:04:52.731173188Z at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:356)
2019-02-01T05:04:52.731175525Z at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
2019-02-01T05:04:52.731177905Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore.startInternal(FileSystemRMStateStore.java:141)
2019-02-01T05:04:52.731180622Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.serviceStart(RMStateStore.java:562)
2019-02-01T05:04:52.731183338Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
2019-02-01T05:04:52.731185899Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceStart(ResourceManager.java:564)
2019-02-01T05:04:52.731188416Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
2019-02-01T05:04:52.731190825Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startActiveServices(ResourceManager.java:974)
2019-02-01T05:04:52.731193465Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1015)
2019-02-01T05:04:52.731196301Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1011)
2019-02-01T05:04:52.731199731Z at java.security.AccessController.doPrivileged(Native Method)
2019-02-01T05:04:52.731202464Z at javax.security.auth.Subject.doAs(Subject.java:422)
2019-02-01T05:04:52.731205034Z at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
2019-02-01T05:04:52.731207624Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:1011)
2019-02-01T05:04:52.731210144Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1051)
2019-02-01T05:04:52.731212614Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
2019-02-01T05:04:52.731215010Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1188)
2019-02-01T05:04:52.731217684Z Caused by: java.net.UnknownHostException: namenode
2019-02-01T05:04:52.731220264Z ... 29 more
2019-02-01T05:04:52.732002339Z 19/02/01 05:04:52 INFO resourcemanager.ResourceManager: SHUTDOWN_MSG:
2019-02-01T05:04:52.732014614Z /************************************************************
2019-02-01T05:04:52.732018144Z SHUTDOWN_MSG: Shutting down ResourceManager at resourcemanager/10.0.5.203
2019-02-01T05:04:52.732020590Z ************************************************************/
docker hadoop stack docker-swarm
asked Feb 1 at 6:48
BehruzBehruz
13
Hadoop resourcemanger can't connect to namenode. I've deployed hadoop to docker container with docker stack deploy. But when I run containers with docker-compose up it works fine. I want to manage with swarm, that's why need to run with docker stack deploy
docker.compose.yml:
version: "3.3"
services:
namenode:
image: hadoop-namenode:2.7.7
hostname: namenode
volumes:
- ./data/namenode:/hadoop/dfs/name
environment:
- CLUSTER_NAME=test
env_file:
- ./hadoop.env
ports:
- 50070:50070
- 8020:8020
deploy:
replicas: 1
restart_policy:
condition: on-failure
networks:
- hadoopnet
resourcemanager:
image: hadoop-resourcemanager:2.7.7
hostname: resourcemanager
depends_on:
- namenode
env_file:
- ./hadoop.env
ports:
- 8088:8088
deploy:
replicas: 1
restart_policy:
condition: on-failure
networks:
- hadoopnet
networks:
hadoopnet:
external:
name: receiptnet
Error log:
2019-02-01T05:04:52.511935227Z 19/02/01 05:04:52 INFO resourcemanager.ResourceManager: Transitioning to active state
2019-02-01T05:04:52.672344032Z 19/02/01 05:04:52 INFO service.AbstractService: Service org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore failed in state STARTED; cause: java.lang.IllegalArgumentException: java.net.UnknownHostException: namenode
2019-02-01T05:04:52.672363778Z java.lang.IllegalArgumentException: java.net.UnknownHostException: namenode
2019-02-01T05:04:52.672367234Z at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:377)
2019-02-01T05:04:52.672370066Z at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:320)
2019-02-01T05:04:52.672375508Z at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176)
2019-02-01T05:04:52.672378205Z at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:687)
2019-02-01T05:04:52.672381153Z at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:628)
2019-02-01T05:04:52.672383878Z at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149)
2019-02-01T05:04:52.672386707Z at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2667)
2019-02-01T05:04:52.672389357Z at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:93)
2019-02-01T05:04:52.672392026Z at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2701)
2019-02-01T05:04:52.672394684Z at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2683)
2019-02-01T05:04:52.672397232Z at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:372)
2019-02-01T05:04:52.672399864Z at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:171)
2019-02-01T05:04:52.672402470Z at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:356)
2019-02-01T05:04:52.672415836Z at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
2019-02-01T05:04:52.672418809Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore.startInternal(FileSystemRMStateStore.java:141)
2019-02-01T05:04:52.672421594Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.serviceStart(RMStateStore.java:562)
2019-02-01T05:04:52.672424281Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
2019-02-01T05:04:52.672426994Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceStart(ResourceManager.java:564)
2019-02-01T05:04:52.672429918Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
2019-02-01T05:04:52.672432556Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startActiveServices(ResourceManager.java:974)
2019-02-01T05:04:52.672435381Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1015)
2019-02-01T05:04:52.672438983Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1011)
2019-02-01T05:04:52.672441831Z at java.security.AccessController.doPrivileged(Native Method)
2019-02-01T05:04:52.672444365Z at javax.security.auth.Subject.doAs(Subject.java:422)
2019-02-01T05:04:52.672447066Z at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
2019-02-01T05:04:52.672449849Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:1011)
2019-02-01T05:04:52.672452619Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1051)
2019-02-01T05:04:52.672455440Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
2019-02-01T05:04:52.672458129Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1188)
2019-02-01T05:04:52.672460913Z Caused by: java.net.UnknownHostException: namenode
2019-02-01T05:04:52.672463551Z ... 29 more
2019-02-01T05:04:52.672676001Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.686973256Z 19/02/01 05:04:52 WARN service.AbstractService: When stopping the service org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore : java.lang.NullPointerException
2019-02-01T05:04:52.686991741Z java.lang.NullPointerException
2019-02-01T05:04:52.686994730Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore$13.run(FileSystemRMStateStore.java:692)
2019-02-01T05:04:52.686997647Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore$13.run(FileSystemRMStateStore.java:689)
2019-02-01T05:04:52.687000863Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore$FSAction.runWithRetries(FileSystemRMStateStore.java:705)
2019-02-01T05:04:52.687003644Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore.closeWithRetries(FileSystemRMStateStore.java:689)
2019-02-01T05:04:52.687030092Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore.closeInternal(FileSystemRMStateStore.java:149)
2019-02-01T05:04:52.687033922Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.serviceStop(RMStateStore.java:580)
2019-02-01T05:04:52.687036528Z at org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
2019-02-01T05:04:52.687038942Z at org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
2019-02-01T05:04:52.687041285Z at org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:80)
2019-02-01T05:04:52.687043724Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:203)
2019-02-01T05:04:52.687046140Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceStart(ResourceManager.java:564)
2019-02-01T05:04:52.687048649Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
2019-02-01T05:04:52.687051072Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startActiveServices(ResourceManager.java:974)
2019-02-01T05:04:52.687053591Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1015)
2019-02-01T05:04:52.687056124Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1011)
2019-02-01T05:04:52.687058648Z at java.security.AccessController.doPrivileged(Native Method)
2019-02-01T05:04:52.687061014Z at javax.security.auth.Subject.doAs(Subject.java:422)
2019-02-01T05:04:52.687063539Z at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
2019-02-01T05:04:52.687066991Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:1011)
2019-02-01T05:04:52.687069777Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1051)
2019-02-01T05:04:52.687074082Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
2019-02-01T05:04:52.687076615Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1188)
2019-02-01T05:04:52.687154597Z 19/02/01 05:04:52 INFO service.AbstractService: Service RMActiveServices failed in state STARTED; cause: java.lang.IllegalArgumentException: java.net.UnknownHostException: namenode
2019-02-01T05:04:52.687254023Z Caused by: java.net.UnknownHostException: namenode
2019-02-01T05:04:52.687258435Z ... 29 more
2019-02-01T05:04:52.687441964Z 19/02/01 05:04:52 INFO impl.MetricsSystemImpl: Stopping ResourceManager metrics system...
2019-02-01T05:04:52.687731949Z 19/02/01 05:04:52 INFO impl.MetricsSystemImpl: ResourceManager metrics system stopped.
2019-02-01T05:04:52.687835386Z 19/02/01 05:04:52 INFO impl.MetricsSystemImpl: ResourceManager metrics system shutdown complete.
2019-02-01T05:04:52.688442258Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.RMFatalEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMFatalEventDispatcher
2019-02-01T05:04:52.688821697Z 19/02/01 05:04:52 INFO security.NMTokenSecretManagerInRM: NMTokenKeyRollingInterval: 86400000ms and NMTokenKeyActivationDelay: 900000ms
2019-02-01T05:04:52.688998639Z 19/02/01 05:04:52 INFO security.RMContainerTokenSecretManager: ContainerTokenKeyRollingInterval: 86400000ms and ContainerTokenKeyActivationDelay: 900000ms
2019-02-01T05:04:52.691041177Z 19/02/01 05:04:52 INFO security.AMRMTokenSecretManager: AMRMTokenKeyRollingInterval: 86400000ms and AMRMTokenKeyActivationDelay: 900000 ms
2019-02-01T05:04:52.691140559Z 19/02/01 05:04:52 INFO recovery.RMStateStoreFactory: Using RMStateStore implementation - class org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore
2019-02-01T05:04:52.691222539Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStoreEventType for class org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$ForwardingEventHandler
2019-02-01T05:04:52.691424692Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.NodesListManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.NodesListManager
2019-02-01T05:04:52.691459962Z 19/02/01 05:04:52 INFO resourcemanager.ResourceManager: Using Scheduler: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler
2019-02-01T05:04:52.691593345Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher
2019-02-01T05:04:52.691620389Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationEventDispatcher
2019-02-01T05:04:52.691650575Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$ApplicationAttemptEventDispatcher
2019-02-01T05:04:52.691681755Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$NodeEventDispatcher
2019-02-01T05:04:52.692718629Z 19/02/01 05:04:52 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2019-02-01T05:04:52.693481063Z 19/02/01 05:04:52 INFO impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2019-02-01T05:04:52.693506892Z 19/02/01 05:04:52 INFO impl.MetricsSystemImpl: ResourceManager metrics system started
2019-02-01T05:04:52.708320956Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.RMAppManagerEventType for class org.apache.hadoop.yarn.server.resourcemanager.RMAppManager
2019-02-01T05:04:52.708418732Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEventType for class org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher
2019-02-01T05:04:52.708612991Z 19/02/01 05:04:52 WARN util.MBeans: Failed to register MBean "Hadoop:service=ResourceManager,name=RMNMInfo": Instance already exists.
2019-02-01T05:04:52.708630958Z 19/02/01 05:04:52 INFO resourcemanager.RMNMInfo: Registered RMNMInfo MBean
2019-02-01T05:04:52.708727767Z 19/02/01 05:04:52 INFO util.HostsFileReader: Refreshing hosts (include/exclude) list
2019-02-01T05:04:52.709327466Z 19/02/01 05:04:52 INFO conf.Configuration: found resource capacity-scheduler.xml at file:/opt/hadoop-2.7.7/etc/hadoop/capacity-scheduler.xml
2019-02-01T05:04:52.726958313Z 19/02/01 05:04:52 INFO capacity.CapacitySchedulerConfiguration: max alloc mb per queue for root is undefined
2019-02-01T05:04:52.726978624Z 19/02/01 05:04:52 INFO capacity.CapacitySchedulerConfiguration: max alloc vcore per queue for root is undefined
2019-02-01T05:04:52.727081097Z 19/02/01 05:04:52 INFO capacity.ParentQueue: root, capacity=1.0, asboluteCapacity=1.0, maxCapacity=1.0, asboluteMaxCapacity=1.0, state=RUNNING, acls=ADMINISTER_QUEUE:*SUBMIT_APP:*, labels=*,
2019-02-01T05:04:52.727088073Z , reservationsContinueLooking=true
2019-02-01T05:04:52.727104981Z 19/02/01 05:04:52 INFO capacity.ParentQueue: Initialized parent-queue root name=root, fullname=root
2019-02-01T05:04:52.728900615Z 19/02/01 05:04:52 INFO capacity.CapacitySchedulerConfiguration: max alloc mb per queue for root.default is undefined
2019-02-01T05:04:52.728917761Z 19/02/01 05:04:52 INFO capacity.CapacitySchedulerConfiguration: max alloc vcore per queue for root.default is undefined
2019-02-01T05:04:52.729060099Z 19/02/01 05:04:52 INFO capacity.LeafQueue: Initializing default
2019-02-01T05:04:52.729066915Z capacity = 1.0 [= (float) configuredCapacity / 100 ]
2019-02-01T05:04:52.729069853Z asboluteCapacity = 1.0 [= parentAbsoluteCapacity * capacity ]
2019-02-01T05:04:52.729072357Z maxCapacity = 1.0 [= configuredMaxCapacity ]
2019-02-01T05:04:52.729074780Z absoluteMaxCapacity = 1.0 [= 1.0 maximumCapacity undefined, (parentAbsoluteMaxCapacity * maximumCapacity) / 100 otherwise ]
2019-02-01T05:04:52.729077499Z userLimit = 100 [= configuredUserLimit ]
2019-02-01T05:04:52.729079928Z userLimitFactor = 1.0 [= configuredUserLimitFactor ]
2019-02-01T05:04:52.729082468Z maxApplications = 10000 [= configuredMaximumSystemApplicationsPerQueue or (int)(configuredMaximumSystemApplications * absoluteCapacity)]
2019-02-01T05:04:52.729085342Z maxApplicationsPerUser = 10000 [= (int)(maxApplications * (userLimit / 100.0f) * userLimitFactor) ]
2019-02-01T05:04:52.729097025Z usedCapacity = 0.0 [= usedResourcesMemory / (clusterResourceMemory * absoluteCapacity)]
2019-02-01T05:04:52.729100156Z absoluteUsedCapacity = 0.0 [= usedResourcesMemory / clusterResourceMemory]
2019-02-01T05:04:52.729102684Z maxAMResourcePerQueuePercent = 0.1 [= configuredMaximumAMResourcePercent ]
2019-02-01T05:04:52.729105142Z minimumAllocationFactor = 0.875 [= (float)(maximumAllocationMemory - minimumAllocationMemory) / maximumAllocationMemory ]
2019-02-01T05:04:52.729107582Z maximumAllocation = <memory:8192, vCores:32> [= configuredMaxAllocation ]
2019-02-01T05:04:52.729110196Z numContainers = 0 [= currentNumContainers ]
2019-02-01T05:04:52.729112529Z state = RUNNING [= configuredState ]
2019-02-01T05:04:52.729114912Z acls = ADMINISTER_QUEUE:*SUBMIT_APP:* [= configuredAcls ]
2019-02-01T05:04:52.729117397Z nodeLocalityDelay = 40
2019-02-01T05:04:52.729119763Z labels=*,
2019-02-01T05:04:52.729122044Z nodeLocalityDelay = 40
2019-02-01T05:04:52.729124520Z reservationsContinueLooking = true
2019-02-01T05:04:52.729126850Z preemptionDisabled = true
2019-02-01T05:04:52.729129257Z
2019-02-01T05:04:52.729132934Z 19/02/01 05:04:52 INFO capacity.CapacityScheduler: Initialized queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=<memory:0, vCores:0>, usedCapacity=0.0, absoluteUsedCapacity=0.0, numApps=0, numContainers=0
2019-02-01T05:04:52.729189879Z 19/02/01 05:04:52 INFO capacity.CapacityScheduler: Initialized queue: root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=<memory:0, vCores:0>usedCapacity=0.0, numApps=0, numContainers=0
2019-02-01T05:04:52.729256580Z 19/02/01 05:04:52 INFO capacity.CapacityScheduler: Initialized root queue root: numChildQueue= 1, capacity=1.0, absoluteCapacity=1.0, usedResources=<memory:0, vCores:0>usedCapacity=0.0, numApps=0, numContainers=0
2019-02-01T05:04:52.729281241Z 19/02/01 05:04:52 INFO capacity.CapacityScheduler: Initialized queue mappings, override: false
2019-02-01T05:04:52.729530679Z 19/02/01 05:04:52 INFO capacity.CapacityScheduler: Initialized CapacityScheduler with calculator=class org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator, minimumAllocation=<<memory:1024, vCores:1>>, maximumAllocation=<<memory:8192, vCores:32>>, asynchronousScheduling=false, asyncScheduleInterval=5ms
2019-02-01T05:04:52.730297879Z 19/02/01 05:04:52 INFO service.AbstractService: Service ResourceManager failed in state STARTED; cause: java.lang.IllegalArgumentException: java.net.UnknownHostException: namenode
...
2019-02-01T05:04:52.730415589Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730462887Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730490199Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730530613Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730570231Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730618909Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730654117Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730683974Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730719284Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730754134Z 19/02/01 05:04:52 INFO event.AsyncDispatcher: AsyncDispatcher is draining to stop, igonring any new events.
2019-02-01T05:04:52.730828723Z 19/02/01 05:04:52 INFO resourcemanager.ResourceManager: Transitioning to standby state
2019-02-01T05:04:52.730860759Z 19/02/01 05:04:52 INFO resourcemanager.ResourceManager: Transitioned to standby state
2019-02-01T05:04:52.731128521Z 19/02/01 05:04:52 FATAL resourcemanager.ResourceManager: Error starting ResourceManager
2019-02-01T05:04:52.731134799Z java.lang.IllegalArgumentException: java.net.UnknownHostException: namenode
2019-02-01T05:04:52.731137638Z at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:377)
2019-02-01T05:04:52.731140203Z at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:320)
2019-02-01T05:04:52.731142680Z at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176)
2019-02-01T05:04:52.731145223Z at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:687)
2019-02-01T05:04:52.731148005Z at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:628)
2019-02-01T05:04:52.731150679Z at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149)
2019-02-01T05:04:52.731153277Z at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2667)
2019-02-01T05:04:52.731155733Z at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:93)
2019-02-01T05:04:52.731158209Z at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2701)
2019-02-01T05:04:52.731160792Z at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2683)
2019-02-01T05:04:52.731167909Z at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:372)
2019-02-01T05:04:52.731170782Z at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:171)
2019-02-01T05:04:52.731173188Z at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:356)
2019-02-01T05:04:52.731175525Z at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
2019-02-01T05:04:52.731177905Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore.startInternal(FileSystemRMStateStore.java:141)
2019-02-01T05:04:52.731180622Z at org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore.serviceStart(RMStateStore.java:562)
2019-02-01T05:04:52.731183338Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
2019-02-01T05:04:52.731185899Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceStart(ResourceManager.java:564)
2019-02-01T05:04:52.731188416Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
2019-02-01T05:04:52.731190825Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startActiveServices(ResourceManager.java:974)
2019-02-01T05:04:52.731193465Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1015)
2019-02-01T05:04:52.731196301Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$1.run(ResourceManager.java:1011)
2019-02-01T05:04:52.731199731Z at java.security.AccessController.doPrivileged(Native Method)
2019-02-01T05:04:52.731202464Z at javax.security.auth.Subject.doAs(Subject.java:422)
2019-02-01T05:04:52.731205034Z at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
2019-02-01T05:04:52.731207624Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.transitionToActive(ResourceManager.java:1011)
2019-02-01T05:04:52.731210144Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1051)
2019-02-01T05:04:52.731212614Z at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
2019-02-01T05:04:52.731215010Z at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1188)
2019-02-01T05:04:52.731217684Z Caused by: java.net.UnknownHostException: namenode
2019-02-01T05:04:52.731220264Z ... 29 more
2019-02-01T05:04:52.732002339Z 19/02/01 05:04:52 INFO resourcemanager.ResourceManager: SHUTDOWN_MSG:
2019-02-01T05:04:52.732014614Z /************************************************************
2019-02-01T05:04:52.732018144Z SHUTDOWN_MSG: Shutting down ResourceManager at resourcemanager/10.0.5.203
2019-02-01T05:04:52.732020590Z ************************************************************/
docker hadoop stack docker-swarm
docker hadoop stack docker-swarm
asked Feb 1 at 6:48
BehruzBehruz
13
asked Feb 1 at 6:48
BehruzBehruz
13
edited Feb 1 at 6:54
Behruz
asked Feb 1 at 6:48
BehruzBehruz
13
asked Feb 1 at 6:48
BehruzBehruz
13
asked Feb 1 at 6:48
BehruzBehruz
13
13
add a comment |
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "3"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1400873%2fhadoop-resource-manager-doesnt-start-on-docker-container-with-stack-deploy%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
draft saved
draft discarded
Thanks for contributing an answer to Super User!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1400873%2fhadoop-resource-manager-doesnt-start-on-docker-container-with-stack-deploy%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
