eggroll.properties: Eggroll's Main Configuration File

Configuration File Style

eggroll.properties uses typical k=v style as inferred by its filename extension.

Naming Conventions

The key of each configuration item follows this naming convention: eggroll.${module}.${conf_key_seperated_by_dot}

For example, eggroll.session.processors.per.node can be explained as this:

fixed: eggroll

module: session

conf_key: processors.per.node

The value of each configuration item is in free style. Actual data type will be explained and converted in usage. The default value can be found in conf_keys.py and ConfKeys.scala. If the default values from conf_keys.py and ConfKeys.scala differs, the actual value depends of the actual usage: whether it is used in python or jvm. If it happens, please give us a feedback in issue zone.

Common Configurations:

#	key	default value	mandatory (Y/N)	description	comments
1	eggroll.logs.dir	(N/A)	Y	Eggroll's logs dir, a path where logs generates
2	eggroll.data.dir	(N/A)	Y	Eggroll's data dir, a path where data locates
3	eggroll.core.security.ca.crt.path	(N/A)	N	The path of CA certificate to verify client certificate	secure channel
4	eggroll.core.security.crt.path	(N/A)	N	The path of server certificate (used when accepting an RPC request)	secure channel
5	eggroll.core.security.key.path	(N/A)	N	The path of server private key	secure channel
6	eggroll.core.security.client.ca.crt.path	(N/A)	N	The path of CA certificate to verify server certificate	secure channel
7	eggroll.core.security.client.crt.path	(N/A)	N	The path of client certificate (used when initiating an RPC request)	secure channel
8	eggroll.core.security.key.path	(N/A)	N	The path of client private key	secure channel
9	eggroll.resourcemanager.clustermanager.host	(N/A)	Y	Host of Cluster Manager
10	eggroll.resourcemanager.clustermanager.port	(N/A)	Y	Port of Cluster Manager
11	eggroll.resourcemanager.nodemanager.port	(N/A)	Y	Port of Node Manager
12	eggroll.resourcemanager.process.tag	(N/A)	N	A tag in command line to distinguish different instances of CM / NM / RS
13	eggroll.session.start.timeout.ms	20_000	N	Timeout in ms to get or create a new session	overridden in default conf file with 180_000
14	eggroll.session.stop.timeout.ms	20_000	N	Timeout in ms to stop a session
15	eggroll.session.processors.per.node	(N/A)	Y	Number of processors started of each node for a session	explicitly written to 4 in default conf file
16	eggroll.rollpair.transferpair.sendbuf.size	250_000	N	Batch size of data transfer WITHIN A CLUSTER
17	eggroll.rollpair.eggpair.server.executor.pool.max.size	5_000	N	Max size of egg_pair's command server executor pool
18	eggroll.rollpair.eggpair.data.server.executor.pool.max.size	5_000	N	Max size of egg_pair's data server executor pool
19	eggroll.rollpair.transferpair.executor.pool.max.size	5_000	N	Max size of egg_pair's shuffle executor pool
20	eggroll.rollpair.default.store.type	ROLLPAIR_LMDB	N	Default store type of RollPair
21	eggroll.rollsite.pull.max.retry	720	N	Max retries of pulling data. Actual waiting time is this value * eggroll.rollsite.pull.interval.sec. Transfer time should not be greater than this time period
22	eggroll.rollsite.pull.interval.sec	600	N	Interval of each pulling attempt. Also acts as timeout of one pulling attempt
23	eggroll.rollsite.pull.header.timeout.sec	432_000 (720 * 600)	N	Timeout of pulling the first batch of data. Computing time in other parties should not be greater than this value
24	eggroll.rollsite.pull.header.interval.sec	300	N	Interval of each pulling header attempt
25	eggroll.rollsite.push.batches.per.stream	10	N	Stream batch size when pushing data
26	eggroll.rollsite.push.max.retry	3	N	Max retries in logic layer when pushing data
27	eggroll.rollsite.push.overall.timeout.sec	600	N	Timeout of waiting push to complete and session exits
28	eggroll.rollsite.push.per.stream.timeout.sec	300	N	Timeout of pushing a single stream
29	eggroll.rollsite.polling.server.enabled	false	N	Whether the RS runs in polling server mode	polling mode
30	eggroll.rollsite.polling.client.enabled	false	N	Whether the RS runs in polling client mode	polling mode
31	eggroll.rollsite.polling.concurrency	50	N	Number of polling threads	polling mode
32	eggroll.rollsite.polling.no.data.timeout.sec	300	N	Timeout of polling but no data arrives	polling mode
33	eggroll.rollsite.polling.exchanger.data.op.timeout.sec	240	N	Timeout of offering / polling data to / from q	polling mode
34	eggroll.rollsite.host	127.0.0.1	N	Host of rollsite
35	eggroll.rollsite.port	9370	N	Port of rollsite
36	eggroll.rollsite.party.id	10001	N	Rollsite party ID of myself
37	eggroll.rollsite.route.table.path	conf/route_table.json	N	file path of route table
38	eggroll.rollsite.route.table.key	123	N	Key to verify when updating route table	overridden in default conf file
39	eggroll.rollsite.route.table.whitelist	127.0.0.1	N	Ip white list seperated by comma where route table update is allowed
40	eggroll.rollsite.jvm.options	(N/A)	N	jvm options used when rollsite starts	overridden in default conf file
41	eggroll.rollsite.adapter.sendbuf.size	1_048_576	N	Batch size when sending data TO ANOTHER SITE / AMONG CLUSTERS
42	eggroll.rollsite.receive.executor.pool.max.size	5_000	N	Max executor size of roll site client