Skip to content

Commit 775def1

Browse files
yodengyodeng
committed
update readme
1 parent 9aea072 commit 775def1

File tree

1 file changed

+77
-120
lines changed

1 file changed

+77
-120
lines changed

‎README.md

+77-120
Original file line numberDiff line numberDiff line change
@@ -50,196 +50,153 @@ You can get the quick help like this:
5050
##### runjob/runflow：
5151

5252
$ runjob --help
53-
Usage: runjob [-h] [-v] [-j [<jobfile>]] [-n <int>] [-s <int>] [-e <int>]
54-
[-w <workdir>] [-d] [-l <file>] [-r <int>] [-f] [-R <int>] [-C]
55-
[-M {local,localhost,sge,slurm,batchcompute}] [--ini <configfile>]
56-
[--dag] [--dag-extend] [--strict] [--quiet] [--max-check <float>]
57-
[--max-submit <float>] [--max-queue-time <float/str>]
58-
[--max-run-time <float/str>] [--max-wait-time <float/str>]
59-
[--max-timeout-retry <int>]
60-
[--local | --localhost | --sge | --slurm | --batchcompute]
61-
[-i [<str> ...]] [-L <logdir>]
53+
Usage: runjob [-h] [-v] [-j [<jobfile>]] [-n <int>] [-s <int>] [-e <int>] [-w <workdir>] [-d] [-l <file>] [-f]
54+
[-M {local,localhost,sge,slurm}] [--config <configfile>] [--dag] [--dag-extend] [--strict] [--quiet] [--show-config]
55+
[-r <int>] [-R <int>] [--max-check <float>] [--max-submit <float>] [--max-queue-time <float/str>]
56+
[--max-run-time <float/str>] [--max-wait-time <float/str>] [--max-timeout-retry <int>]
57+
[--local | --localhost | --sge | --slurm] [-i [<str> ...]] [-L <logdir>] [-q [<queue> ...]] [-c <int>] [-m <int>]
58+
[--node [<node> ...]] [--round-node]
6259

63-
runjob is a tool for managing parallel tasks from a specific job file running
64-
in localhost, sge, slurm, batchcompute.
60+
runjob is a tool for managing parallel tasks from a specific job file running in localhost, sge, slurm.
6561

6662
Optional Arguments:
6763
-h, --help show this help message and exit
6864
--local submit your jobs to local, same as '--mode local'.
6965
--localhost submit your jobs to localhost, same as '--mode localhost'.
7066
--sge submit your jobs to sge, same as '--mode sge'.
7167
--slurm submit your jobs to slurm, same as '--mode slurm'.
72-
--batchcompute submit your jobs to batchcompute, same as '--mode
73-
batchcompute'.
7468
-i, --injname [<str> ...]
75-
job names you need to run. (default: all job names of the
76-
jobfile)
69+
job names you need to run. (default: all job names of the jobfile)
7770
-L, --logdir <logdir>
7871
the output log dir. (default: join(workdir, "logs"))
7972

8073
Base Arguments:
8174
-v, --version show program's version number and exit
8275
-j, --jobfile [<jobfile>]
8376
input jobfile, if empty, stdin is used. (required)
84-
-n, --num <int> the max job number runing at the same time. (default: all
85-
of the jobfile, max 1000)
86-
-s, --start <int> which line number(1-base) be used for the first job.
87-
(default: 1)
88-
-e, --end <int> which line number (include) be used for the last job.
89-
(default: last line of the jobfile)
77+
-n, --num <int> the max job number runing at the same time. (default: all of the jobfile, max 1000)
78+
-s, --start <int> which line number(1-base) be used for the first job. (default: 1)
79+
-e, --end <int> which line number (include) be used for the last job. (default: last line of the jobfile)
9080
-w, --workdir <workdir>
9181
work directory. (default: /home/dengyong/soft/git/runjob)
9282
-d, --debug log debug info.
9383
-l, --log <file> append log info to file. (default: stdout)
94-
-r, --retry <int> retry N times of the error job, 0 or minus means do not
95-
re-submit. (default: 0)
9684
-f, --force force to submit jobs even already successed.
85+
-M, --mode {local,localhost,sge,slurm}
86+
the mode to submit your jobs, if no sge installed, always localhost. (default: sge)
87+
--config <configfile>
88+
input configfile for configurations search.
89+
--dag do not execute anything and print the directed acyclic graph of jobs in the dot language.
90+
--dag-extend do not execute anything and print the extend directed acyclic graph of jobs in the dot language.
91+
--strict use strict to run, means if any errors, clean all jobs and exit.
92+
--quiet suppress all output and logging.
93+
--show-config show configurations and exit.
94+
95+
Rate Arguments:
96+
-r, --retry <int> retry N times of the error job, 0 or minus means do not re-submit. (default: 0)
9797
-R, --retry-sec <int>
9898
retry the error job after N seconds. (default: 2)
99-
-C, --config show configurations and exit.
100-
-M, --mode {local,localhost,sge,slurm,batchcompute}
101-
the mode to submit your jobs, if no sge installed, always
102-
localhost. (default: sge)
103-
--ini <configfile> input configfile for configurations search.
104-
--dag do not execute anything and print the directed acyclic
105-
graph of jobs in the dot language.
106-
--dag-extend do not execute anything and print the extend directed
107-
acyclic graph of jobs in the dot language.
108-
--strict use strict to run, means if any errors, clean all jobs and
109-
exit.
110-
--quiet suppress all output and logging.
111-
--max-check <float> maximal number of job status checks per second, fractions
112-
allowed. (default: 5)
113-
--max-submit <float> maximal number of jobs submited per second, fractions
114-
allowed. (default: 20)
99+
--max-check <float> maximal number of job status checks per second, fractions allowed. (default: 5)
100+
--max-submit <float> maximal number of jobs submited per second, fractions allowed. (default: 20)
115101

116-
Time Control Arguments:
102+
Time Arguments:
117103
--max-queue-time <float/str>
118-
maximal time (d/h/m/s) between submit and running per job.
119-
(default: no-limiting)
104+
maximal time (d/h/m/s) between submit and running per job. (default: no-limiting)
120105
--max-run-time <float/str>
121-
maximal time (d/h/m/s) start from running per job.
122-
(default: no-limiting)
106+
maximal time (d/h/m/s) start from running per job. (default: no-limiting)
123107
--max-wait-time <float/str>
124-
maximal time (d/h/m/s) start from submit per job. (default:
125-
no-limiting)
108+
maximal time (d/h/m/s) start from submit per job. (default: no-limiting)
126109
--max-timeout-retry <int>
127-
retry N times for the timeout error job, 0 or minus means
128-
do not re-submit. (default: 0)
110+
retry N times for the timeout error job, 0 or minus means do not re-submit. (default: 0)
111+
112+
Resource Arguments:
113+
-q, --queue [<queue> ...]
114+
queue/partition for running, multi-queue can be sepreated by whitespace. (default: all accessed)
115+
-c, --cpu <int> max cpu number used. (default: 1)
116+
-m, --memory <int> max memory used (GB). (default: 1)
117+
--node [<node> ...] node for running, multi-node can be sepreated by whitespace. (default: all accessed)
118+
--round-node round all define node per job for load balance
129119

130120
##### runsge/runshell/runbatch:
131121

132122
```
133123
$ runsge --help
134-
Usage: runsge [-h] [-v] [-j [<jobfile>]] [-n <int>] [-s <int>] [-e <int>]
135-
[-w <workdir>] [-d] [-l <file>] [-r <int>] [-f] [-R <int>] [-C]
136-
[-M {local,localhost,sge,slurm,batchcompute}] [--ini <configfile>]
137-
[--dag] [--dag-extend] [--strict] [--quiet] [--max-check <float>]
138-
[--max-submit <float>] [--max-queue-time <float/str>]
139-
[--max-run-time <float/str>] [--max-wait-time <float/str>]
140-
[--max-timeout-retry <int>]
141-
[--local | --localhost | --sge | --slurm | --batchcompute]
142-
[-N <jobname>] [-L <logdir>] [-g <int>] [--init <cmd>]
143-
[--call-back <cmd>] [-q [<queue> ...]] [-m <int>] [-c <int>]
144-
[--out-maping <dir>] [--access-key-id <str>]
145-
[--access-key-secret <str>]
146-
[--region {beijing,hangzhou,huhehaote,shanghai,zhangjiakou,chengdu,hongkong,qingdao,shenzhen}]
147-
148-
runsge is a tool for managing parallel tasks from a specific shell file
149-
runing in localhost, sge, slurm, batchcompute.
124+
Usage: runsge [-h] [-v] [-j [<jobfile>]] [-n <int>] [-s <int>] [-e <int>] [-w <workdir>] [-d] [-l <file>] [-f]
125+
[-M {local,localhost,sge,slurm}] [--config <configfile>] [--dag] [--dag-extend] [--strict] [--quiet] [--show-config]
126+
[-r <int>] [-R <int>] [--max-check <float>] [--max-submit <float>] [--max-queue-time <float/str>]
127+
[--max-run-time <float/str>] [--max-wait-time <float/str>] [--max-timeout-retry <int>]
128+
[--local | --localhost | --sge | --slurm] [-N <jobname>] [-L <logdir>] [-g <int>] [--init <cmd>] [--call-back <cmd>]
129+
[-q [<queue> ...]] [-c <int>] [-m <int>] [--node [<node> ...]] [--round-node]
130+
131+
runsge is a tool for managing parallel tasks from a specific shell file runing in localhost, sge, slurm.
150132
151133
Optional Arguments:
152134
-h, --help show this help message and exit
153135
--local submit your jobs to local, same as '--mode local'.
154136
--localhost submit your jobs to localhost, same as '--mode localhost'.
155137
--sge submit your jobs to sge, same as '--mode sge'.
156138
--slurm submit your jobs to slurm, same as '--mode slurm'.
157-
--batchcompute submit your jobs to batchcompute, same as '--mode
158-
batchcompute'.
159139
-N, --jobname <jobname>
160140
job name. (default: basename of the jobfile)
161141
-L, --logdir <logdir>
162-
the output log dir. (default:
163-
"/home/dengyong/soft/git/runjob/runsge_*_log_dir")
142+
the output log dir. (default: "/home/dengyong/soft/git/runjob/runsge_*_log_dir")
164143
-g, --groups <int> N lines to consume a new job group. (default: 1)
165144
--init <cmd> command before all jobs, will be running in localhost.
166-
--call-back <cmd> command after all jobs finished, will be running in
167-
localhost.
145+
--call-back <cmd> command after all jobs finished, will be running in localhost.
168146
169147
Base Arguments:
170148
-v, --version show program's version number and exit
171149
-j, --jobfile [<jobfile>]
172150
input jobfile, if empty, stdin is used. (required)
173-
-n, --num <int> the max job number runing at the same time. (default: all
174-
of the jobfile, max 1000)
175-
-s, --start <int> which line number(1-base) be used for the first job.
176-
(default: 1)
177-
-e, --end <int> which line number (include) be used for the last job.
178-
(default: last line of the jobfile)
151+
-n, --num <int> the max job number runing at the same time. (default: all of the jobfile, max 1000)
152+
-s, --start <int> which line number(1-base) be used for the first job. (default: 1)
153+
-e, --end <int> which line number (include) be used for the last job. (default: last line of the jobfile)
179154
-w, --workdir <workdir>
180155
work directory. (default: /home/dengyong/soft/git/runjob)
181156
-d, --debug log debug info.
182157
-l, --log <file> append log info to file. (default: stdout)
183-
-r, --retry <int> retry N times of the error job, 0 or minus means do not
184-
re-submit. (default: 0)
185158
-f, --force force to submit jobs even already successed.
159+
-M, --mode {local,localhost,sge,slurm}
160+
the mode to submit your jobs, if no sge installed, always localhost. (default: sge)
161+
--config <configfile>
162+
input configfile for configurations search.
163+
--dag do not execute anything and print the directed acyclic graph of jobs in the dot language.
164+
--dag-extend do not execute anything and print the extend directed acyclic graph of jobs in the dot language.
165+
--strict use strict to run, means if any errors, clean all jobs and exit.
166+
--quiet suppress all output and logging.
167+
--show-config show configurations and exit.
168+
169+
Rate Arguments:
170+
-r, --retry <int> retry N times of the error job, 0 or minus means do not re-submit. (default: 0)
186171
-R, --retry-sec <int>
187172
retry the error job after N seconds. (default: 2)
188-
-C, --config show configurations and exit.
189-
-M, --mode {local,localhost,sge,slurm,batchcompute}
190-
the mode to submit your jobs, if no sge installed, always
191-
localhost. (default: sge)
192-
--ini <configfile> input configfile for configurations search.
193-
--dag do not execute anything and print the directed acyclic
194-
graph of jobs in the dot language.
195-
--dag-extend do not execute anything and print the extend directed
196-
acyclic graph of jobs in the dot language.
197-
--strict use strict to run, means if any errors, clean all jobs and
198-
exit.
199-
--quiet suppress all output and logging.
200-
--max-check <float> maximal number of job status checks per second, fractions
201-
allowed. (default: 5)
202-
--max-submit <float> maximal number of jobs submited per second, fractions
203-
allowed. (default: 20)
173+
--max-check <float> maximal number of job status checks per second, fractions allowed. (default: 5)
174+
--max-submit <float> maximal number of jobs submited per second, fractions allowed. (default: 20)
204175
205-
Time Control Arguments:
176+
Time Arguments:
206177
--max-queue-time <float/str>
207-
maximal time (d/h/m/s) between submit and running per job.
208-
(default: no-limiting)
178+
maximal time (d/h/m/s) between submit and running per job. (default: no-limiting)
209179
--max-run-time <float/str>
210-
maximal time (d/h/m/s) start from running per job.
211-
(default: no-limiting)
180+
maximal time (d/h/m/s) start from running per job. (default: no-limiting)
212181
--max-wait-time <float/str>
213-
maximal time (d/h/m/s) start from submit per job. (default:
214-
no-limiting)
182+
maximal time (d/h/m/s) start from submit per job. (default: no-limiting)
215183
--max-timeout-retry <int>
216-
retry N times for the timeout error job, 0 or minus means
217-
do not re-submit. (default: 0)
184+
retry N times for the timeout error job, 0 or minus means do not re-submit. (default: 0)
218185
219-
Sge/Slurm Arguments:
186+
Resource Arguments:
220187
-q, --queue [<queue> ...]
221-
the queue/partition your job running, multi queue can be
222-
sepreated by whitespace. (default: all accessed queue)
223-
-m, --memory <int> the memory used per command (GB). (default: 1)
224-
-c, --cpu <int> the cpu numbers you job used. (default: 1)
225-
226-
Batchcompute Arguments:
227-
--out-maping <dir> the oss output directory if your mode is "batchcompute",
228-
all output file will be mapping to you OSS://BUCKET-NAME.
229-
if not set, any output will be reserved.
230-
--access-key-id <str>
231-
AccessKeyID while access oss.
232-
--access-key-secret <str>
233-
AccessKeySecret while access oss.
234-
--region {beijing,hangzhou,huhehaote,shanghai,zhangjiakou,chengdu,hongkong,qingdao,shenzhen}
235-
batch compute region. (default: beijing)
188+
queue/partition for running, multi-queue can be sepreated by whitespace. (default: all accessed)
189+
-c, --cpu <int> max cpu number used. (default: 1)
190+
-m, --memory <int> max memory used (GB). (default: 1)
191+
--node [<node> ...] node for running, multi-node can be sepreated by whitespace. (default: all accessed)
192+
--round-node round all define node per job for load balance
236193
```
237194

238195
##### qs/qcs:
239196

240197
```
241198
$ qs --help
242-
For summary all jobs
199+
For summary jobs
243200
Usage: qs [jobfile|logdir|logfile]
244201
qcs --help
245202
qslurm

0 commit comments

Comments
 (0)