Trace number 3317591

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-randomUNSAT 942.118 362.887

General information on the benchmark

Namerandom/medium/
unif-k5-r21.3-v100-c2130-S1741857700-014.UNKNOWN.cnf
MD5SUM68bf08be908d1e2b9826bc30706a7365
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark349.016
Satisfiable
(Un)Satisfiability was proved
Number of variables100
Number of clauses2130
Sum of the clauses size10650
Maximum clause length5
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 52130
Number of clauses of size over 50

Solver Data 

0.35/0.63	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.35/0.63	c INFO: solving HOME/instance-3317591-1304270255.cnf
0.35/0.63	c INFO: computing task features
0.35/0.64	c INFO: computed variable-clause graph statistics
0.35/0.64	c INFO: computed clause balance statistics
0.43/0.70	c INFO: computed variable balance statistics
0.43/0.73	c INFO: computed small-clause counts
0.43/0.73	c INFO: computed Horn-clause counts
0.43/0.73	c INFO: computed variable graph statistics
0.43/0.73	c INFO: collected features for HOME/instance-3317591-1304270255.cnf in 0.10 s
0.43/0.76	c INFO: running gnovelty+2@0 for 97 with 4799 remaining (b = 0.62)
0.43/0.79	c INFO: running march_hi@0 for 194 with 4799 remaining (b = 0.19)
0.54/0.82	c INFO: running gnovelty+2@0 for 97 with 4799 remaining (b = 0.07)
0.54/0.85	c INFO: running march_hi@0 for 388 with 4799 remaining (b = 0.29)
348.93/124.99	c INFO: running march_hi@0 for 485 with 4675 remaining (b = 0.30)
529.42/197.30	c INFO: running march_hi@0 for 485 with 4603 remaining (b = 0.27)
540.11/201.63	c INFO: running march_hi@0 for 291 with 4599 remaining (b = 0.18)
941.80/362.86	s UNSATISFIABLE

Verifier Data 

No possible verification on an UNSAT instance

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3317591-1304270255/watcher-3317591-1304270255 -o /tmp/evaluation-result-3317591-1304270255/solver-3317591-1304270255 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3317591-1304270255.cnf -seed 1493208630 -budget 4800 -cores 4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.98 9.30 9.03 8/256 21435
/proc/meminfo: memFree=23467128/32951124 swapFree=67111524/67111528
[pid=21435] ppid=21433 vsize=15184 CPUtime=0 cores=1,3,5,7
/proc/21435/stat : 21435 (python) S 21433 21435 20058 0 -1 4202496 733 0 0 0 0 0 0 0 25 0 1 0 538623298 15548416 622 33554432000 4194304 5683644 140734838671248 18446744073709551615 212687673344 0 2147483391 16781312 2 18446744071699199166 0 0 17 1 0 0 0
/proc/21435/statm: 3796 622 322 364 0 283 0

[startup+0.0817611 s]
/proc/loadavg: 8.98 9.30 9.03 8/256 21435
/proc/meminfo: memFree=23467128/32951124 swapFree=67111524/67111528
[pid=21435] ppid=21433 vsize=50016 CPUtime=0.04 cores=1,3,5,7
/proc/21435/stat : 21435 (python) S 21433 21435 20058 0 -1 4202496 1975 0 0 0 3 1 0 0 25 0 1 0 538623298 51216384 1737 33554432000 4194304 5683644 140734838671248 18446744073709551615 212687673344 0 2147483391 16781312 2 18446744071699199166 0 0 17 1 0 0 0
/proc/21435/statm: 12504 1737 480 364 0 1204 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 50016

[startup+0.100761 s]
/proc/loadavg: 8.98 9.30 9.03 8/256 21435
/proc/meminfo: memFree=23467128/32951124 swapFree=67111524/67111528
[pid=21435] ppid=21433 vsize=89772 CPUtime=0.05 cores=1,3,5,7
/proc/21435/stat : 21435 (python) R 21433 21435 20058 0 -1 4202496 2672 0 0 0 4 1 0 0 17 0 1 0 538623298 91926528 2388 33554432000 4194304 5683644 140734838671248 18446744073709551615 212687673344 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/21435/statm: 22443 2388 750 364 0 1623 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 89772

[startup+0.300751 s]
/proc/loadavg: 8.98 9.30 9.03 8/256 21435
/proc/meminfo: memFree=23467128/32951124 swapFree=67111524/67111528
[pid=21435] ppid=21433 vsize=135752 CPUtime=0.17 cores=1,3,5,7
/proc/21435/stat : 21435 (python) S 21433 21435 20058 0 -1 4202496 5961 934 1 0 14 3 0 0 18 0 1 0 538623298 139010048 4151 33554432000 4194304 5683644 140734838671248 18446744073709551615 212687671301 0 2147483391 16781312 2 18446744071699199166 0 0 17 1 0 0 0
/proc/21435/statm: 33938 4151 1032 364 0 3124 0
Current children cumulated CPU time (s) 0.17
Current children cumulated vsize (KiB) 135752

[startup+0.700651 s]
/proc/loadavg: 8.98 9.30 9.03 8/256 21435
/proc/meminfo: memFree=23467128/32951124 swapFree=67111524/67111528
[pid=21435] ppid=21433 vsize=278432 CPUtime=0.43 cores=1,3,5,7
/proc/21435/stat : 21435 (python) R 21433 21435 20058 0 -1 4202496 9329 934 1 0 35 8 0 0 17 0 1 0 538623298 285114368 7000 33554432000 4194304 5683644 140734838671248 18446744073709551615 4602978 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/21435/statm: 69608 7000 1826 364 0 5532 0
Current children cumulated CPU time (s) 0.43
Current children cumulated vsize (KiB) 278432

[startup+1.50047 s]
/proc/loadavg: 8.98 9.30 9.03 11/269 21452
/proc/meminfo: memFree=23427964/32951124 swapFree=67111524/67111528
[pid=21435] ppid=21433 vsize=320168 CPUtime=0.59 cores=1,3,5,7
/proc/21435/stat : 21435 (python) S 21433 21435 20058 0 -1 4202496 12264 934 1 0 50 9 0 0 25 0 5 0 538623298 327852032 7238 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699502027 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/21435/statm: 80042 7238 1881 364 0 15951 0
[pid=21441] ppid=21435 vsize=278456 CPUtime=0 cores=1,3,5,7
/proc/21441/stat : 21441 (python) S 21435 21435 20058 0 -1 4202560 737 0 0 0 0 0 0 0 18 0 1 0 538623375 285138944 5550 33554432000 4194304 5683644 140734838671248 18446744073709551615 212687695375 0 0 16781312 514 4499150732741574655 0 0 17 1 0 0 0
/proc/21441/statm: 69614 5550 354 364 0 5532 0
[pid=21443] ppid=21441 vsize=1017800 CPUtime=0.41 cores=1,3,5,7
/proc/21443/stat : 21443 (gnovelty+2) R 21441 21443 21443 0 -1 4202496 436 0 0 0 41 0 0 0 20 0 1 0 538623375 1042227200 171 33554432000 134512640 135663704 4289825280 18446744073709551615 134525520 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/21443/statm: 254450 171 113 282 0 254166 0
[pid=21444] ppid=21435 vsize=289280 CPUtime=0 cores=1,3,5,7
/proc/21444/stat : 21444 (python) S 21435 21435 20058 0 -1 4202560 752 0 0 0 0 0 0 0 15 0 1 0 538623378 296222720 5676 33554432000 4194304 5683644 140734838671248 18446744073709551615 212687695430 0 0 16781312 514 6094830451568902408 0 0 17 7 0 0 0
/proc/21444/statm: 72320 5676 348 364 0 8235 0
[pid=21446] ppid=21444 vsize=1560 CPUtime=0.64 cores=1,3,5,7
/proc/21446/stat : 21446 (march_hi) R 21444 21446 21446 0 -1 4202496 502 0 0 0 64 0 0 0 25 0 1 0 538623380 1597440 240 33554432000 4194304 4955518 140737151146336 18446744073709551615 4208667 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/21446/statm: 390 240 106 186 0 203 0
[pid=21447] ppid=21435 vsize=299536 CPUtime=0 cores=1,3,5,7
/proc/21447/stat : 21447 (python) S 21435 21435 20058 0 -1 4202560 745 0 0 0 0 0 0 0 22 0 1 0 538623381 306724864 5680 33554432000 4194304 5683644 140734838671248 18446744073709551615 212687695430 0 0 16781312 514 4465196439286317055 0 0 17 7 0 0 0
/proc/21447/statm: 74884 5680 348 364 0 10796 0
[pid=21449] ppid=21447 vsize=1017800 CPUtime=0.32 cores=1,3,5,7
/proc/21449/stat : 21449 (gnovelty+2) R 21447 21449 21449 0 -1 4202496 440 0 0 0 32 0 0 0 25 0 1 0 538623383 1042227200 172 33554432000 134512640 135663704 4293665680 18446744073709551615 134546746 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/21449/statm: 254450 172 113 282 0 254166 0
[pid=21450] ppid=21435 vsize=309924 CPUtime=0 cores=1,3,5,7
/proc/21450/stat : 21450 (python) S 21435 21435 20058 0 -1 4202560 753 0 0 0 0 0 0 0 21 0 1 0 538623384 317362176 5701 33554432000 4194304 5683644 140734838671248 18446744073709551615 212687695430 0 0 16781312 514 1106601283046834438 0 0 17 7 0 0 0
/proc/21450/statm: 77481 5701 348 364 0 13390 0
[pid=21452] ppid=21450 vsize=1560 CPUtime=0.31 cores=1,3,5,7
/proc/21452/stat : 21452 (march_hi) R 21450 21452 21452 0 -1 4202496 505 0 0 0 31 0 0 0 25 0 1 0 538623384 1597440 239 33554432000 4194304 4955518 140734487367072 18446744073709551615 4235627 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/21452/statm: 390 239 106 186 0 203 0
[pid=21435/tid=21442] ppid=21433 vsize=320168 CPUtime=0 cores=1,3,5,7
/proc/21435/task/21442/stat : 21442 (python) S 21433 21435 20058 0 -1 4202560 27 934 0 0 0 0 0 0 19 0 5 0 538623375 327852032 7238 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 2 18446604467105998912 0 0 -1 1 0 0 0
[pid=21435/tid=21445] ppid=21433 vsize=320168 CPUtime=0 cores=1,3,5,7
/proc/21435/task/21445/stat : 21445 (python) S 21433 21435 20058 0 -1 4202560 25 934 0 0 0 0 0 0 21 0 5 0 538623378 327852032 7238 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 2 18446604467105998912 0 0 -1 7 0 0 0
[pid=21435/tid=21448] ppid=21433 vsize=320168 CPUtime=0 cores=1,3,5,7
/proc/21435/task/21448/stat : 21448 (python) S 21433 21435 20058 0 -1 4202560 23 934 0 0 0 0 0 0 23 0 5 0 538623381 327852032 7238 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 2 18446604467105998912 0 0 -1 7 0 0 0
[pid=21435/tid=21451] ppid=21433 vsize=320168 CPUtime=0 cores=1,3,5,7
/proc/21435/task/21451/stat : 21451 (python) S 21433 21435 20058 0 -1 4202560 24 934 0 0 0 0 0 0 25 0 5 0 538623384 327852032 7238 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 2 18446604467105998912 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 2.27
Current children cumulated vsize (KiB) 3536084

[startup+3.10118 s]
/proc/loadavg: 8.98 9.30 9.03 11/269 21452
/proc/meminfo: memFree=23427220/32951124 swapFree=67111524/67111528
[pid=21435] ppid=21433 vsize=320168 CPUtime=0.59 cores=1,3,5,7
/proc/21435/stat : 21435 (python) S 21433 21435 20058 0 -1 4202496 12264 934 1 0 50 9 0 0 25 0 5 0 538623298 327852032 7238 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699502027 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/21435/statm: 80042 7238 1881 364 0 15951 0
[pid=21441] ppid=21435 vsize=278456 CPUtime=0 cores=1,3,5,7
/proc/21441/stat : 21441 (python) S 21435 21435 20058 0 -1 4202560 737 0 0 0 0 0 0 0 18 0 1 0 538623375 285138944 5550 33554432000 4194304 5683644 140734838671248 18446744073709551615 212687695375 0 0 16781312 514 4499150732741542151 0 0 17 1 0 0 0
/proc/21441/statm: 69614 5550 354 364 0 5532 0
[pid=21443] ppid=21441 vsize=1017800 CPUtime=1.21 cores=1,3,5,7
/proc/21443/stat : 21443 (gnovelty+2) R 21441 21443 21443 0 -1 4202496 436 0 0 0 121 0 0 0 22 0 1 0 538623375 1042227200 171 33554432000 134512640 135663704 4289825280 18446744073709551615 134544186 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/21443/statm: 254450 171 113 282 0 254166 0
[pid=21444] ppid=21435 vsize=289280 CPUtime=0 cores=1,3,5,7
/proc/21444/stat : 21444 (python) S 21435 21435 20058 0 -1 4202560 752 0 0 0 0 0 0 0 15 0 1 0 538623378 296222720 5676 33554432000 4194304 5683644 140734838671248 18446744073709551615 212687695430 0 0 16781312 514 6094830451568902408 0 0 17 7 0 0 0
/proc/21444/statm: 72320 5676 348 364 0 8235 0
[pid=21446] ppid=21444 vsize=1560 CPUtime=2.23 cores=1,3,5,7
/proc/21446/stat : 21446 (march_hi) R 21444 21446 21446 0 -1 4202496 507 0 0 0 223 0 0 0 25 0 1 0 538623380 1597440 245 33554432000 4194304 4955518 140737151146336 18446744073709551615 4224642 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/21446/statm: 390 245 106 186 0 203 0
[pid=21447] ppid=21435 vsize=299536 CPUtime=0 cores=1,3,5,7
/proc/21447/stat : 21447 (python) S 21435 21435 20058 0 -1 4202560 745 0 0 0 0 0 0 0 18 0 1 0 538623381 306724864 5680 33554432000 4194304 5683644 140734838671248 18446744073709551615 212687695430 0 0 16781312 514 4465196439286284551 0 0 17 7 0 0 0
/proc/21447/statm: 74884 5680 348 364 0 10796 0
[pid=21449] ppid=21447 vsize=1017800 CPUtime=1.12 cores=1,3,5,7
/proc/21449/stat : 21449 (gnovelty+2) R 21447 21449 21449 0 -1 4202496 440 0 0 0 112 0 0 0 25 0 1 0 538623383 1042227200 172 33554432000 134512640 135663704 4293665680 18446744073709551615 134524561 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/21449/statm: 254450 172 113 282 0 254166 0
[pid=21450] ppid=21435 vsize=309924 CPUtime=0 cores=1,3,5,7
/proc/21450/stat : 21450 (python) S 21435 21435 20058 0 -1 4202560 761 0 0 0 0 0 0 0 15 0 1 0 538623384 317362176 5701 33554432000 4194304 5683644 140734838671248 18446744073709551615 212687695430 0 0 16781312 514 1106601283046834438 0 0 17 7 0 0 0
/proc/21450/statm: 77481 5701 348 364 0 13390 0
[pid=21452] ppid=21450 vsize=1560 CPUtime=1.11 cores=1,3,5,7
/proc/21452/stat : 21452 (march_hi) R 21450 21452 21452 0 -1 4202496 505 0 0 0 111 0 0 0 25 0 1 0 538623384 1597440 239 33554432000 4194304 4955518 140734487367072 18446744073709551615 4297490 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/21452/statm: 390 239 106 186 0 203 0
[pid=21435/tid=21442] ppid=21433 vsize=320168 CPUtime=0 cores=1,3,5,7
/proc/21435/task/21442/stat : 21442 (python) S 21433 21435 20058 0 -1 4202560 27 934 0 0 0 0 0 0 19 0 5 0 538623375 327852032 7238 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 2 18446604467105998912 0 0 -1 1 0 0 0
[pid=21435/tid=21445] ppid=21433 vsize=320168 CPUtime=0 cores=1,3,5,7
/proc/21435/task/21445/stat : 21445 (python) S 21433 21435 20058 0 -1 4202560 25 934 0 0 0 0 0 0 21 0 5 0 538623378 327852032 7238 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 2 18446604467105998912 0 0 -1 7 0 0 0
[pid=21435/tid=21448] ppid=21433 vsize=320168 CPUtime=0 cores=1,3,5,7
/proc/21435/task/21448/stat : 21448 (python) S 21433 21435 20058 0 -1 4202560 23 934 0 0 0 0 0 0 23 0 5 0 538623381 327852032 7238 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 2 18446604467105998912 0 0 -1 7 0 0 0
[pid=21435/tid=21451] ppid=21433 vsize=320168 CPUtime=0 cores=1,3,5,7
/proc/21435/task/21451/stat : 21451 (python) S 21433 21435 20058 0 -1 4202560 24 934 0 0 0 0 0 0 25 0 5 0 538623384 327852032 7238 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 2 18446604467105998912 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 6.26
Current children cumulated vsize (KiB) 3536084

[startup+6.30059 s]
/proc/loadavg: 9.06 9.32 9.03 12/269 21452
/proc/meminfo: memFree=23424732/32951124 swapFree=67111524/67111528
[pid=21435] ppid=21433 vsize=320168 CPUtime=0.59 cores=1,3,5,7
/proc/21435/stat : 21435 (python) S 21433 21435 20058 0 -1 4202496 12264 934 1 0 50 9 0 0 25 0 5 0 538623298 327852032 7238 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699502027 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/21435/statm: 80042 7238 1881 364 0 15951 0
[pid=21441] ppid=21435 vsize=278456 CPUtime=0 cores=1,3,5,7
/proc/21441/stat : 21441 (python) S 21435 21435 20058 0 -1 4202560 898 0 0 0 0 0 0 0 18 0 1 0 538623375 285138944 5573 33554432000 4194304 5683644 140734838671248 18446744073709551615 212687695375 0 0 16781312 514 4499150732741542151 0 0 17 1 0 0 0
/proc/21441/statm: 69614 5573 377 364 0 5532 0
[pid=21443] ppid=21441 vsize=1017800 CPUtime=2.81 cores=1,3,5,7
/proc/21443/stat : 21443 (gnovelty+2) R 21441 21443 21443 0 -1 4202496 436 0 0 0 281 0 0 0 25 0 1 0 538623375 1042227200 171 33554432000 134512640 135663704 4289825280 18446744073709551615 134525107 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/21443/statm: 254450 171 113 282 0 254166 0
[pid=21444] ppid=21435 vsize=289280 CPUtime=0.01 cores=1,3,5,7
/proc/21444/stat : 21444 (python) S 21435 21435 20058 0 -1 4202560 910 0 0 0 1 0 0 0 15 0 1 0 538623378 296222720 5699 33554432000 4194304 5683644 140734838671248 18446744073709551615 212687695430 0 0 16781312 514 6094830451568902408 0 0 17 7 0 0 0
/proc/21444/statm: 72320 5699 371 364 0 8235 0
[pid=21446] ppid=21444 vsize=1848 CPUtime=5.37 cores=1,3,5,7
/proc/21446/stat : 21446 (march_hi) R 21444 21446 21446 0 -1 4202496 545 0 0 0 537 0 0 0 25 0 1 0 538623380 1892352 283 33554432000 4194304 4955518 140737151146336 18446744073709551615 4208653 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/21446/statm: 462 283 106 186 0 275 0
[pid=21447] ppid=21435 vsize=299536 CPUtime=0 cores=1,3,5,7
/proc/21447/stat : 21447 (python) S 21435 21435 20058 0 -1 4202560 899 0 0 0 0 0 0 0 18 0 1 0 538623381 306724864 5703 33554432000 4194304 5683644 140734838671248 18446744073709551615 212687695430 0 0 16781312 514 4465196439286284551 0 0 17 7 0 0 0
/proc/21447/statm: 74884 5703 371 364 0 10796 0
[pid=21449] ppid=21447 vsize=1017800 CPUtime=2.72 cores=1,3,5,7
/proc/21449/stat : 21449 (gnovelty+2) R 21447 21449 21449 0 -1 4202496 440 0 0 0 272 0 0 0 25 0 1 0 538623383 1042227200 172 33554432000 134512640 135663704 4293665680 18446744073709551615 134525255 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/21449/statm: 254450 172 113 282 0 254166 0
[pid=21450] ppid=21435 vsize=309924 CPUtime=0 cores=1,3,5,7
/proc/21450/stat : 21450 (python) S 21435 21435 20058 0 -1 4202560 902 0 0 0 0 0 0 0 15 0 1 0 538623384 317362176 5723 33554432000 4194304 5683644 140734838671248 18446744073709551615 212687695430 0 0 16781312 514 1106601283046834438 0 0 17 7 0 0 0
/proc/21450/statm: 77481 5723 370 364 0 13390 0
[pid=21452] ppid=21450 vsize=1712 CPUtime=2.71 cores=1,3,5,7
/proc/21452/stat : 21452 (march_hi) R 21450 21452 21452 0 -1 4202496 514 0 0 0 271 0 0 0 25 0 1 0 538623384 1753088 248 33554432000 4194304 4955518 140734487367072 18446744073709551615 4224707 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/21452/statm: 428 248 106 186 0 241 0
[pid=21435/tid=21442] ppid=21433 vsize=320168 CPUtime=0 cores=1,3,5,7
/proc/21435/task/21442/stat : 21442 (python) S 21433 21435 20058 0 -1 4202560 27 934 0 0 0 0 0 0 19 0 5 0 538623375 327852032 7238 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 2 18446604467105998912 0 0 -1 1 0 0 0
[pid=21435/tid=21445] ppid=21433 vsize=320168 CPUtime=0 cores=1,3,5,7
/proc/21435/task/21445/stat : 21445 (python) S 21433 21435 20058 0 -1 4202560 25 934 0 0 0 0 0 0 21 0 5 0 538623378 327852032 7238 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 2 18446604467105998912 0 0 -1 7 0 0 0
[pid=21435/tid=21448] ppid=21433 vsize=320168 CPUtime=0 cores=1,3,5,7
/proc/21435/task/21448/stat : 21448 (python) S 21433 21435 20058 0 -1 4202560 23 934 0 0 0 0 0 0 23 0 5 0 538623381 327852032 7238 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 2 18446604467105998912 0 0 -1 7 0 0 0
[pid=21435/tid=21451] ppid=21433 vsize=320168 CPUtime=0 cores=1,3,5,7
/proc/21435/task/21451/stat : 21451 (python) S 21433 21435 20058 0 -1 4202560 24 934 0 0 0 0 0 0 25 0 5 0 538623384 327852032 7238 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 2 18446604467105998912 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 14.21
Current children cumulated vsize (KiB) 3536524

[startup+12.7014 s]
/proc/loadavg: 9.22 9.34 9.04 11/269 21452
/proc/meminfo: memFree=23422476/32951124 swapFree=67111524/67111528
[pid=21435] ppid=21433 vsize=320168 CPUtime=0.59 cores=1,3,5,7
/proc/21435/stat : 21435 (python) S 21433 21435 20058 0 -1 4202496 12264 934 1 0 50 9 0 0 25 0 5 0 538623298 327852032 7238 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699502027 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/21435/statm: 80042 7238 1881 364 0 15951 0
[pid=21441] ppid=21435 vsize=278456 CPUtime=0.01 cores=1,3,5,7
/proc/21441/stat : 21441 (python) S 21435 21435 20058 0 -1 4202560 901 0 0 0 0 1 0 0 18 0 1 0 538623375 285138944 5573 33554432000 4194304 5683644 140734838671248 18446744073709551615 212687695375 0 0 16781312 514 17293998490963116295 0 0 17 1 0 0 0
/proc/21441/statm: 69614 5573 377 364 0 5532 0
[pid=21443] ppid=21441 vsize=1017800 CPUtime=6.01 cores=1,3,5,7
/proc/21443/stat : 21443 (gnovelty+2) R 21441 21443 21443 0 -1 4202496 436 0 0 0 601 0 0 0 25 0 1 0 538623375 1042227200 171 33554432000 134512640 135663704 4289825280 18446744073709551615 134545605 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/21443/statm: 254450 171 113 282 0 254166 0
[pid=21444] ppid=21435 vsize=289280 CPUtime=0.02 cores=1,3,5,7

################
# More data... #
################

/proc/21446/stat : 21446 (march_hi) R 21444 21446 21446 0 -1 4202500 4820 0 0 0 35394 3 0 0 25 0 1 0 538623380 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 7 0 0 0
/proc/21446/statm: 0 0 0 0 0 0 0
[pid=21444/tid=21998] ppid=21435 vsize=289280 CPUtime=0 cores=1,3,5,7
/proc/21444/task/21998/stat : 21998 (python) S 21435 21435 20058 0 -1 4202560 47 0 0 0 0 0 0 0 16 0 2 0 538643458 296222720 5738 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 514 18446604463485186240 0 0 -1 7 0 0 0
[pid=21447] ppid=21435 vsize=299536 CPUtime=0.34 cores=1,3,5,7
/proc/21447/stat : 21447 (python) S 21435 21435 20058 0 -1 4202560 1116 0 0 0 20 14 0 0 19 0 2 0 538623381 306724864 5733 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699502027 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/21447/statm: 74884 5733 400 364 0 10796 0
[pid=21449] ppid=21447 vsize=1017800 CPUtime=98.19 cores=1,3,5,7
/proc/21449/stat : 21449 (gnovelty+2) T 21447 21449 21449 0 -1 4202496 440 0 0 0 9819 0 0 0 25 0 1 0 538623383 1042227200 172 33554432000 134512640 135663704 4293665680 18446744073709551615 134524882 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/21449/statm: 254450 172 113 282 0 254166 0
[pid=21447/tid=21994] ppid=21435 vsize=299536 CPUtime=0 cores=1,3,5,7
/proc/21447/task/21994/stat : 21994 (python) S 21435 21435 20058 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 538643026 306724864 5733 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 514 18446604471132196992 0 0 -1 7 0 0 0
[pid=21450] ppid=21435 vsize=309924 CPUtime=0.71 cores=1,3,5,7
/proc/21450/stat : 21450 (python) S 21435 21435 20058 0 -1 4202560 985 0 0 0 37 34 0 0 15 0 1 0 538623384 317362176 5724 33554432000 4194304 5683644 140734838671248 18446744073709551615 212687695430 0 0 16781312 514 1106601283046834438 0 0 17 7 0 0 0
/proc/21450/statm: 77481 5724 371 364 0 13390 0
[pid=21452] ppid=21450 vsize=9476 CPUtime=180.68 cores=1,3,5,7
/proc/21452/stat : 21452 (march_hi) R 21450 21452 21452 0 -1 4202496 2036 0 0 0 18068 0 0 0 25 0 1 0 538623384 9703424 1770 33554432000 4194304 4955518 140734487367072 18446744073709551615 4208656 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/21452/statm: 2369 1770 106 186 0 2182 0
[pid=21984] ppid=21435 vsize=320180 CPUtime=0.47 cores=1,3,5,7
/proc/21984/stat : 21984 (python) S 21435 21435 20058 0 -1 4202560 1010 0 0 0 25 22 0 0 15 0 1 0 538635797 327864320 5728 33554432000 4194304 5683644 140734838671248 18446744073709551615 212687695430 0 0 16781312 514 1422970360776589575 0 0 17 7 0 0 0
/proc/21984/statm: 80045 5728 371 364 0 15951 0
[pid=21986] ppid=21984 vsize=5164 CPUtime=118.63 cores=1,3,5,7
/proc/21986/stat : 21986 (march_hi) R 21984 21986 21986 0 -1 4202496 1368 0 0 0 11863 0 0 0 25 0 1 0 538635798 5287936 1103 33554432000 4194304 4955518 140737073954576 18446744073709551615 4208667 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/21986/statm: 1291 1103 106 186 0 1104 0
[pid=21995] ppid=21435 vsize=330044 CPUtime=0.34 cores=1,3,5,7
/proc/21995/stat : 21995 (python) S 21435 21435 20058 0 -1 4202560 998 0 0 0 19 15 0 0 15 0 1 0 538643029 337965056 5653 33554432000 4194304 5683644 140734838671248 18446744073709551615 212687695430 0 0 16781312 514 9689323078768623879 0 0 17 7 0 0 0
/proc/21995/statm: 82511 5653 371 364 0 18414 0
[pid=21997] ppid=21995 vsize=4012 CPUtime=82.5 cores=1,3,5,7
/proc/21997/stat : 21997 (march_hi) R 21995 21997 21997 0 -1 4202496 1060 0 0 0 8250 0 0 0 25 0 1 0 538643030 4108288 794 33554432000 4194304 4955518 140735303999312 18446744073709551615 4224471 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/21997/statm: 1003 794 106 186 0 816 0
[pid=21435/tid=21442] ppid=21433 vsize=340860 CPUtime=0 cores=1,3,5,7
/proc/21435/task/21442/stat : 21442 (python) S 21433 21435 20058 0 -1 4202560 27 934 0 0 0 0 0 0 19 0 7 0 538623375 349040640 7296 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 2 18446604467105998912 0 0 -1 1 0 0 0
[pid=21435/tid=21445] ppid=21433 vsize=340860 CPUtime=0 cores=1,3,5,7
/proc/21435/task/21445/stat : 21445 (python) S 21433 21435 20058 0 -1 4202560 28 934 0 0 0 0 0 0 18 0 7 0 538623378 349040640 7296 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 2 18446604467105998912 0 0 -1 7 0 0 0
[pid=21435/tid=21448] ppid=21433 vsize=340860 CPUtime=0 cores=1,3,5,7
/proc/21435/task/21448/stat : 21448 (python) S 21433 21435 20058 0 -1 4202560 23 934 0 0 0 0 0 0 23 0 7 0 538623381 349040640 7296 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 2 18446604467105998912 0 0 -1 7 0 0 0
[pid=21435/tid=21451] ppid=21433 vsize=340860 CPUtime=0 cores=1,3,5,7
/proc/21435/task/21451/stat : 21451 (python) S 21433 21435 20058 0 -1 4202560 24 934 0 0 0 0 0 0 25 0 7 0 538623384 349040640 7296 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 2 18446604467105998912 0 0 -1 7 0 0 0
[pid=21435/tid=21985] ppid=21433 vsize=340860 CPUtime=0 cores=1,3,5,7
/proc/21435/task/21985/stat : 21985 (python) S 21433 21435 20058 0 -1 4202560 30 934 0 0 0 0 0 0 20 0 7 0 538635798 349040640 7296 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 2 18446604467105998912 0 0 -1 7 0 0 0
[pid=21435/tid=21996] ppid=21433 vsize=340860 CPUtime=0 cores=1,3,5,7
/proc/21435/task/21996/stat : 21996 (python) S 21433 21435 20058 0 -1 4202560 25 934 0 0 0 0 0 0 20 0 7 0 538643030 349040640 7296 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 2 18446604467105998912 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 940.55
Current children cumulated vsize (KiB) 4232776

[startup+362.702 s]
/proc/loadavg: 10.11 9.94 9.41 13/282 22007
/proc/meminfo: memFree=23264228/32951124 swapFree=67111524/67111528
[pid=21435] ppid=21433 vsize=340860 CPUtime=0.69 cores=1,3,5,7
/proc/21435/stat : 21435 (python) S 21433 21435 20058 0 -1 4202496 14599 934 1 0 59 10 0 0 18 0 7 0 538623298 349040640 7296 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699502027 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/21435/statm: 85215 7296 1886 364 0 21118 0
[pid=21441] ppid=21435 vsize=288700 CPUtime=0.21 cores=1,3,5,7
/proc/21441/stat : 21441 (python) S 21435 21435 20058 0 -1 4202560 1096 0 0 0 11 10 0 0 19 0 2 0 538623375 295628800 5611 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699502027 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/21441/statm: 72175 5611 410 364 0 8093 0
[pid=21443] ppid=21441 vsize=1017800 CPUtime=101.17 cores=1,3,5,7
/proc/21443/stat : 21443 (gnovelty+2) T 21441 21443 21443 0 -1 4202496 436 0 0 0 10117 0 0 0 25 0 1 0 538623375 1042227200 171 33554432000 134512640 135663704 4289825280 18446744073709551615 134525271 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/21443/statm: 254450 171 113 282 0 254166 0
[pid=21441/tid=21983] ppid=21435 vsize=288700 CPUtime=0 cores=1,3,5,7
/proc/21441/task/21983/stat : 21983 (python) S 21435 21435 20058 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 538635795 295628800 5611 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 514 18446604471140102592 0 0 -1 1 0 0 0
[pid=21444] ppid=21435 vsize=289280 CPUtime=3.05 cores=1,3,5,7
/proc/21444/stat : 21444 (python) R 21435 21435 20058 0 -1 4202560 1155 0 0 0 241 64 0 0 18 0 2 0 538623378 296222720 5738 33554432000 4194304 5683644 140734838671248 18446744073709551615 4505999 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/21444/statm: 72320 5738 409 364 0 8235 0
[pid=21446] ppid=21444 vsize=0 CPUtime=353.97 cores=1,3,5,7
/proc/21446/stat : 21446 (march_hi) R 21444 21446 21446 0 -1 4202500 4820 0 0 0 35394 3 0 0 25 0 1 0 538623380 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 7 0 0 0
/proc/21446/statm: 0 0 0 0 0 0 0
[pid=21444/tid=21998] ppid=21435 vsize=289280 CPUtime=0 cores=1,3,5,7
/proc/21444/task/21998/stat : 21998 (python) S 21435 21435 20058 0 -1 4202560 47 0 0 0 0 0 0 0 16 0 2 0 538643458 296222720 5738 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 514 18446604463485186240 0 0 -1 7 0 0 0
[pid=21447] ppid=21435 vsize=299536 CPUtime=0.34 cores=1,3,5,7
/proc/21447/stat : 21447 (python) S 21435 21435 20058 0 -1 4202560 1116 0 0 0 20 14 0 0 19 0 2 0 538623381 306724864 5733 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699502027 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/21447/statm: 74884 5733 400 364 0 10796 0
[pid=21449] ppid=21447 vsize=1017800 CPUtime=98.19 cores=1,3,5,7
/proc/21449/stat : 21449 (gnovelty+2) T 21447 21449 21449 0 -1 4202496 440 0 0 0 9819 0 0 0 25 0 1 0 538623383 1042227200 172 33554432000 134512640 135663704 4293665680 18446744073709551615 134524882 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/21449/statm: 254450 172 113 282 0 254166 0
[pid=21447/tid=21994] ppid=21435 vsize=299536 CPUtime=0 cores=1,3,5,7
/proc/21447/task/21994/stat : 21994 (python) S 21435 21435 20058 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 538643026 306724864 5733 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 514 18446604471132196992 0 0 -1 7 0 0 0
[pid=21450] ppid=21435 vsize=309924 CPUtime=0.71 cores=1,3,5,7
/proc/21450/stat : 21450 (python) S 21435 21435 20058 0 -1 4202560 985 0 0 0 37 34 0 0 15 0 1 0 538623384 317362176 5724 33554432000 4194304 5683644 140734838671248 18446744073709551615 212687695430 0 0 16781312 514 1106601283046834438 0 0 17 7 0 0 0
/proc/21450/statm: 77481 5724 371 364 0 13390 0
[pid=21452] ppid=21450 vsize=9476 CPUtime=180.88 cores=1,3,5,7
/proc/21452/stat : 21452 (march_hi) R 21450 21452 21452 0 -1 4202496 2038 0 0 0 18088 0 0 0 25 0 1 0 538623384 9703424 1772 33554432000 4194304 4955518 140734487367072 18446744073709551615 4237810 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/21452/statm: 2369 1772 106 186 0 2182 0
[pid=21984] ppid=21435 vsize=320180 CPUtime=0.47 cores=1,3,5,7
/proc/21984/stat : 21984 (python) S 21435 21435 20058 0 -1 4202560 1010 0 0 0 25 22 0 0 15 0 1 0 538635797 327864320 5728 33554432000 4194304 5683644 140734838671248 18446744073709551615 212687695430 0 0 16781312 514 1422970360776589575 0 0 17 7 0 0 0
/proc/21984/statm: 80045 5728 371 364 0 15951 0
[pid=21986] ppid=21984 vsize=5164 CPUtime=118.83 cores=1,3,5,7
/proc/21986/stat : 21986 (march_hi) R 21984 21986 21986 0 -1 4202496 1370 0 0 0 11883 0 0 0 25 0 1 0 538635798 5287936 1105 33554432000 4194304 4955518 140737073954576 18446744073709551615 4297472 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/21986/statm: 1291 1105 106 186 0 1104 0
[pid=21995] ppid=21435 vsize=330044 CPUtime=0.34 cores=1,3,5,7
/proc/21995/stat : 21995 (python) S 21435 21435 20058 0 -1 4202560 998 0 0 0 19 15 0 0 15 0 1 0 538643029 337965056 5653 33554432000 4194304 5683644 140734838671248 18446744073709551615 212687695430 0 0 16781312 514 9689323078768623879 0 0 17 7 0 0 0
/proc/21995/statm: 82511 5653 371 364 0 18414 0
[pid=21997] ppid=21995 vsize=4012 CPUtime=82.7 cores=1,3,5,7
/proc/21997/stat : 21997 (march_hi) R 21995 21997 21997 0 -1 4202496 1061 0 0 0 8270 0 0 0 25 0 1 0 538643030 4108288 795 33554432000 4194304 4955518 140735303999312 18446744073709551615 4224443 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/21997/statm: 1003 795 106 186 0 816 0
[pid=21435/tid=21442] ppid=21433 vsize=340860 CPUtime=0 cores=1,3,5,7
/proc/21435/task/21442/stat : 21442 (python) S 21433 21435 20058 0 -1 4202560 27 934 0 0 0 0 0 0 19 0 7 0 538623375 349040640 7296 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 2 18446604467105998912 0 0 -1 1 0 0 0
[pid=21435/tid=21445] ppid=21433 vsize=340860 CPUtime=0 cores=1,3,5,7
/proc/21435/task/21445/stat : 21445 (python) S 21433 21435 20058 0 -1 4202560 28 934 0 0 0 0 0 0 18 0 7 0 538623378 349040640 7296 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 2 18446604467105998912 0 0 -1 7 0 0 0
[pid=21435/tid=21448] ppid=21433 vsize=340860 CPUtime=0 cores=1,3,5,7
/proc/21435/task/21448/stat : 21448 (python) S 21433 21435 20058 0 -1 4202560 23 934 0 0 0 0 0 0 23 0 7 0 538623381 349040640 7296 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 2 18446604467105998912 0 0 -1 7 0 0 0
[pid=21435/tid=21451] ppid=21433 vsize=340860 CPUtime=0 cores=1,3,5,7
/proc/21435/task/21451/stat : 21451 (python) S 21433 21435 20058 0 -1 4202560 24 934 0 0 0 0 0 0 25 0 7 0 538623384 349040640 7296 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 2 18446604467105998912 0 0 -1 7 0 0 0
[pid=21435/tid=21985] ppid=21433 vsize=340860 CPUtime=0 cores=1,3,5,7
/proc/21435/task/21985/stat : 21985 (python) S 21433 21435 20058 0 -1 4202560 30 934 0 0 0 0 0 0 20 0 7 0 538635798 349040640 7296 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 2 18446604467105998912 0 0 -1 7 0 0 0
[pid=21435/tid=21996] ppid=21433 vsize=340860 CPUtime=0 cores=1,3,5,7
/proc/21435/task/21996/stat : 21996 (python) S 21433 21435 20058 0 -1 4202560 25 934 0 0 0 0 0 0 20 0 7 0 538643030 349040640 7296 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 2 18446604467105998912 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 941.55
Current children cumulated vsize (KiB) 4232776

[startup+362.802 s]
/proc/loadavg: 10.11 9.94 9.41 13/282 22007
/proc/meminfo: memFree=23264228/32951124 swapFree=67111524/67111528
[pid=21435] ppid=21433 vsize=340860 CPUtime=0.69 cores=1,3,5,7
/proc/21435/stat : 21435 (python) S 21433 21435 20058 0 -1 4202496 14599 934 1 0 59 10 0 0 18 0 7 0 538623298 349040640 7296 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699502027 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/21435/statm: 85215 7296 1886 364 0 21118 0
[pid=21441] ppid=21435 vsize=288700 CPUtime=0.21 cores=1,3,5,7
/proc/21441/stat : 21441 (python) S 21435 21435 20058 0 -1 4202560 1096 0 0 0 11 10 0 0 19 0 2 0 538623375 295628800 5611 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699502027 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/21441/statm: 72175 5611 410 364 0 8093 0
[pid=21443] ppid=21441 vsize=1017800 CPUtime=101.17 cores=1,3,5,7
/proc/21443/stat : 21443 (gnovelty+2) T 21441 21443 21443 0 -1 4202496 436 0 0 0 10117 0 0 0 25 0 1 0 538623375 1042227200 171 33554432000 134512640 135663704 4289825280 18446744073709551615 134525271 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/21443/statm: 254450 171 113 282 0 254166 0
[pid=21441/tid=21983] ppid=21435 vsize=288700 CPUtime=0 cores=1,3,5,7
/proc/21441/task/21983/stat : 21983 (python) S 21435 21435 20058 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 538635795 295628800 5611 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 514 18446604471140102592 0 0 -1 1 0 0 0
[pid=21444] ppid=21435 vsize=289280 CPUtime=3.14 cores=1,3,5,7
/proc/21444/stat : 21444 (python) R 21435 21435 20058 0 -1 4202560 1155 0 0 0 249 65 0 0 18 0 2 0 538623378 296222720 5738 33554432000 4194304 5683644 140734838671248 18446744073709551615 212687333506 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/21444/statm: 72320 5738 409 364 0 8235 0
[pid=21446] ppid=21444 vsize=0 CPUtime=353.97 cores=1,3,5,7
/proc/21446/stat : 21446 (march_hi) R 21444 21446 21446 0 -1 4202500 4820 0 0 0 35394 3 0 0 25 0 1 0 538623380 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 7 0 0 0
/proc/21446/statm: 0 0 0 0 0 0 0
[pid=21444/tid=21998] ppid=21435 vsize=289280 CPUtime=0 cores=1,3,5,7
/proc/21444/task/21998/stat : 21998 (python) S 21435 21435 20058 0 -1 4202560 47 0 0 0 0 0 0 0 16 0 2 0 538643458 296222720 5738 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 514 18446604463485186240 0 0 -1 7 0 0 0
[pid=21447] ppid=21435 vsize=299536 CPUtime=0.34 cores=1,3,5,7
/proc/21447/stat : 21447 (python) S 21435 21435 20058 0 -1 4202560 1116 0 0 0 20 14 0 0 19 0 2 0 538623381 306724864 5733 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699502027 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/21447/statm: 74884 5733 400 364 0 10796 0
[pid=21449] ppid=21447 vsize=1017800 CPUtime=98.19 cores=1,3,5,7
/proc/21449/stat : 21449 (gnovelty+2) T 21447 21449 21449 0 -1 4202496 440 0 0 0 9819 0 0 0 25 0 1 0 538623383 1042227200 172 33554432000 134512640 135663704 4293665680 18446744073709551615 134524882 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/21449/statm: 254450 172 113 282 0 254166 0
[pid=21447/tid=21994] ppid=21435 vsize=299536 CPUtime=0 cores=1,3,5,7
/proc/21447/task/21994/stat : 21994 (python) S 21435 21435 20058 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 538643026 306724864 5733 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 514 18446604471132196992 0 0 -1 7 0 0 0
[pid=21450] ppid=21435 vsize=309924 CPUtime=0.71 cores=1,3,5,7
/proc/21450/stat : 21450 (python) S 21435 21435 20058 0 -1 4202560 985 0 0 0 37 34 0 0 15 0 1 0 538623384 317362176 5724 33554432000 4194304 5683644 140734838671248 18446744073709551615 212687695430 0 0 16781312 514 1106601283046834438 0 0 17 7 0 0 0
/proc/21450/statm: 77481 5724 371 364 0 13390 0
[pid=21452] ppid=21450 vsize=9476 CPUtime=180.95 cores=1,3,5,7
/proc/21452/stat : 21452 (march_hi) R 21450 21452 21452 0 -1 4202496 2039 0 0 0 18095 0 0 0 25 0 1 0 538623384 9703424 1773 33554432000 4194304 4955518 140734487367072 18446744073709551615 4237696 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/21452/statm: 2369 1773 106 186 0 2182 0
[pid=21984] ppid=21435 vsize=320180 CPUtime=0.47 cores=1,3,5,7
/proc/21984/stat : 21984 (python) S 21435 21435 20058 0 -1 4202560 1010 0 0 0 25 22 0 0 15 0 1 0 538635797 327864320 5728 33554432000 4194304 5683644 140734838671248 18446744073709551615 212687695430 0 0 16781312 514 1422970360776589575 0 0 17 7 0 0 0
/proc/21984/statm: 80045 5728 371 364 0 15951 0
[pid=21986] ppid=21984 vsize=5164 CPUtime=118.89 cores=1,3,5,7
/proc/21986/stat : 21986 (march_hi) R 21984 21986 21986 0 -1 4202496 1370 0 0 0 11889 0 0 0 25 0 1 0 538635798 5287936 1105 33554432000 4194304 4955518 140737073954576 18446744073709551615 4208638 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/21986/statm: 1291 1105 106 186 0 1104 0
[pid=21995] ppid=21435 vsize=330044 CPUtime=0.34 cores=1,3,5,7
/proc/21995/stat : 21995 (python) S 21435 21435 20058 0 -1 4202560 998 0 0 0 19 15 0 0 15 0 1 0 538643029 337965056 5653 33554432000 4194304 5683644 140734838671248 18446744073709551615 212687695430 0 0 16781312 514 9689323078768623879 0 0 17 7 0 0 0
/proc/21995/statm: 82511 5653 371 364 0 18414 0
[pid=21997] ppid=21995 vsize=4012 CPUtime=82.73 cores=1,3,5,7
/proc/21997/stat : 21997 (march_hi) R 21995 21997 21997 0 -1 4202496 1061 0 0 0 8273 0 0 0 25 0 1 0 538643030 4108288 795 33554432000 4194304 4955518 140735303999312 18446744073709551615 4237810 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/21997/statm: 1003 795 106 186 0 816 0
[pid=21435/tid=21442] ppid=21433 vsize=340860 CPUtime=0 cores=1,3,5,7
/proc/21435/task/21442/stat : 21442 (python) S 21433 21435 20058 0 -1 4202560 27 934 0 0 0 0 0 0 19 0 7 0 538623375 349040640 7296 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 2 18446604467105998912 0 0 -1 1 0 0 0
[pid=21435/tid=21445] ppid=21433 vsize=340860 CPUtime=0 cores=1,3,5,7
/proc/21435/task/21445/stat : 21445 (python) S 21433 21435 20058 0 -1 4202560 28 934 0 0 0 0 0 0 18 0 7 0 538623378 349040640 7296 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 2 18446604467105998912 0 0 -1 7 0 0 0
[pid=21435/tid=21448] ppid=21433 vsize=340860 CPUtime=0 cores=1,3,5,7
/proc/21435/task/21448/stat : 21448 (python) S 21433 21435 20058 0 -1 4202560 23 934 0 0 0 0 0 0 23 0 7 0 538623381 349040640 7296 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 2 18446604467105998912 0 0 -1 7 0 0 0
[pid=21435/tid=21451] ppid=21433 vsize=340860 CPUtime=0 cores=1,3,5,7
/proc/21435/task/21451/stat : 21451 (python) S 21433 21435 20058 0 -1 4202560 24 934 0 0 0 0 0 0 25 0 7 0 538623384 349040640 7296 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 2 18446604467105998912 0 0 -1 7 0 0 0
[pid=21435/tid=21985] ppid=21433 vsize=340860 CPUtime=0 cores=1,3,5,7
/proc/21435/task/21985/stat : 21985 (python) S 21433 21435 20058 0 -1 4202560 30 934 0 0 0 0 0 0 20 0 7 0 538635798 349040640 7296 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 2 18446604467105998912 0 0 -1 7 0 0 0
[pid=21435/tid=21996] ppid=21433 vsize=340860 CPUtime=0 cores=1,3,5,7
/proc/21435/task/21996/stat : 21996 (python) S 21433 21435 20058 0 -1 4202560 25 934 0 0 0 0 0 0 20 0 7 0 538643030 349040640 7296 33554432000 4194304 5683644 140734838671248 18446744073709551615 212699498929 0 0 16781312 2 18446604467105998912 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 941.8
Current children cumulated vsize (KiB) 4232776

Child status: 0
Real time (s): 362.887
CPU time (s): 942.118
CPU user time (s): 940.298
CPU system time (s): 1.81972
CPU usage (%): 259.617
Max. virtual memory (cumulated for all children) (KiB): 4252660

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 940.298
system time used= 1.81972
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 36344
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 5841
involuntary context switches= 14879

runsolver used 0.875866 second user time and 1.9757 second system time

The end

Launcher Data 

Begin job on node139 at 2011-05-01 19:17:35
IDJOB=3317591
IDBENCH=83056
IDSOLVER=1705
FILE ID=node139/3317591-1304270255
RUNJOBID= node139-1304267280-20077
PBS_JOBID= 13173940
Free space on /tmp= 72984 MiB

SOLVER NAME= borg-sat 11.03.02-random
BENCH NAME= SAT11/random/medium/unif-k5-r21.3-v100-c2130-S1741857700-014.UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3317591-1304270255/watcher-3317591-1304270255 -o /tmp/evaluation-result-3317591-1304270255/solver-3317591-1304270255 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3317591-1304270255.cnf -seed 1493208630 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 68bf08be908d1e2b9826bc30706a7365
RANDOM SEED=1493208630

node139.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5321.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      23467284 kB
Buffers:       2079140 kB
Cached:        6146016 kB
SwapCached:          4 kB
Active:        2115396 kB
Inactive:      6827324 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      23467284 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:           31872 kB
Writeback:           0 kB
AnonPages:      717312 kB
Mapped:          27072 kB
Slab:           463932 kB
PageTables:      16488 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 35141796 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72952 MiB
End job on node139 at 2011-05-01 19:23:39