Trace number 3378465

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)? 3620.92 1103.95

General information on the benchmark

Namerandom/medium/
unif-k3-r4.26-v700-c2982-S1800377425-032.UNKNOWN.cnf
MD5SUMd85bdfa4ee68fd1c7d18743407334b5d
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables700
Number of clauses2982
Sum of the clauses size8946
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 32982
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.30/0.45	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.30/0.45	c INFO: solving HOME/instance-3378465-1305465799.cnf
0.30/0.45	c INFO: computing task features
0.30/0.46	c INFO: computed variable-clause graph statistics
0.30/0.46	c INFO: computed clause balance statistics
0.39/0.51	c INFO: computed variable balance statistics
0.39/0.55	c INFO: computed small-clause counts
0.39/0.55	c INFO: computed Horn-clause counts
0.39/0.56	c INFO: computed variable graph statistics
0.39/0.56	c INFO: collected features for HOME/instance-3378465-1305465799.cnf in 0.11 s
0.39/0.56	c INFO: running gnovelty+2@0 for 97 with 1199 remaining (b = 0.58)
0.39/0.57	c INFO: running gnovelty+2@0 for 97 with 1199 remaining (b = 0.07)
0.39/0.58	c INFO: running march_hi@0 for 388 with 1199 remaining (b = 0.29)
0.39/0.59	c INFO: running march_hi@0 for 679 with 1199 remaining (b = 0.35)
349.98/100.90	c INFO: running march_hi@0 for 679 with 1099 remaining (b = 0.32)
362.20/105.06	c INFO: running march_hi@0 for 679 with 1095 remaining (b = 0.28)
1557.30/448.08	c INFO: running kcnfs-2006@0 for 582 with 752 remaining (b = 0.24)
2362.09/682.96	c INFO: running march_hi@0 for 517 with 517 remaining (b = 0.19)
3196.42/961.92	c INFO: running march_hi@393 for 97 with 238 remaining (b = 0.04)
3458.53/1049.63	c INFO: running march_hi@0 for 150 with 150 remaining (b = 0.04)
3597.53/1096.23	c INFO: running kcnfs-2006@679 for 104 with 104 remaining (b = 0.04)
3620.72/1103.93	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3378465-1305465799/watcher-3378465-1305465799 -o /tmp/evaluation-result-3378465-1305465799/solver-3378465-1305465799 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378465-1305465799.cnf -seed 266095689 -budget 4800 -cores 4 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 10.23 10.17 9.53 7/471 4015
/proc/meminfo: memFree=26253280/32950928 swapFree=60199044/67111528
[pid=4015] ppid=4013 vsize=15912 CPUtime=0 cores=0,2,4,6
/proc/4015/stat : 4015 (python) S 4013 4015 2082 0 -1 4202496 958 0 0 0 0 0 0 0 25 0 1 0 658178735 16293888 834 33554432000 4194304 5683644 140736223553824 18446744073709551615 268272687104 0 2147483391 16781312 2 18446744071699199166 0 0 17 4 0 0 0
/proc/4015/statm: 3978 834 357 364 0 465 0

[startup+0.053968 s]
/proc/loadavg: 10.23 10.17 9.53 7/471 4015
/proc/meminfo: memFree=26253280/32950928 swapFree=60199044/67111528
[pid=4015] ppid=4013 vsize=36512 CPUtime=0.03 cores=0,2,4,6
/proc/4015/stat : 4015 (python) R 4013 4015 2082 0 -1 4202496 1563 0 0 0 3 0 0 0 25 0 1 0 658178735 37388288 1398 33554432000 4194304 5683644 140736223553824 18446744073709551615 4509190 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/4015/statm: 9128 1398 452 364 0 945 0
Current children cumulated CPU time (s) 0.03
Current children cumulated vsize (KiB) 36512

[startup+0.100959 s]
/proc/loadavg: 10.23 10.17 9.53 7/471 4015
/proc/meminfo: memFree=26253280/32950928 swapFree=60199044/67111528
[pid=4015] ppid=4013 vsize=104616 CPUtime=0.06 cores=0,2,4,6
/proc/4015/stat : 4015 (python) S 4013 4015 2082 0 -1 4202496 3151 0 0 0 5 1 0 0 25 0 1 0 658178735 107126784 2827 33554432000 4194304 5683644 140736223553824 18446744073709551615 268272687104 0 2147483391 16781312 2 18446744071699199166 0 0 17 4 0 0 0
/proc/4015/statm: 26154 2827 847 364 0 1902 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 104616

[startup+0.300924 s]
/proc/loadavg: 10.23 10.17 9.53 7/471 4015
/proc/meminfo: memFree=26253280/32950928 swapFree=60199044/67111528
[pid=4015] ppid=4013 vsize=198920 CPUtime=0.22 cores=0,2,4,6
/proc/4015/stat : 4015 (python) R 4013 4015 2082 0 -1 4202496 7019 934 1 0 18 4 0 0 25 0 1 0 658178735 203694080 5053 33554432000 4194304 5683644 140736223553824 18446744073709551615 268272346865 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/4015/statm: 49730 5053 1466 364 0 3757 0
Current children cumulated CPU time (s) 0.22
Current children cumulated vsize (KiB) 198920

[startup+0.700854 s]
/proc/loadavg: 10.23 10.17 9.53 7/471 4015
/proc/meminfo: memFree=26253280/32950928 swapFree=60199044/67111528
[pid=4015] ppid=4013 vsize=320280 CPUtime=0.49 cores=0,2,4,6
/proc/4015/stat : 4015 (python) S 4013 4015 2082 0 -1 4202496 12212 934 1 0 42 7 0 0 25 0 5 0 658178735 327966720 7266 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284515787 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/4015/statm: 80070 7266 1881 364 0 15979 0
Current children cumulated CPU time (s) 0.49
Current children cumulated vsize (KiB) 320280

[startup+1.50071 s]
/proc/loadavg: 10.23 10.17 9.53 11/484 4032
/proc/meminfo: memFree=26213488/32950928 swapFree=60199044/67111528
[pid=4015] ppid=4013 vsize=320280 CPUtime=0.49 cores=0,2,4,6
/proc/4015/stat : 4015 (python) S 4013 4015 2082 0 -1 4202496 12212 934 1 0 42 7 0 0 25 0 5 0 658178735 327966720 7266 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284515787 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/4015/statm: 80070 7266 1881 364 0 15979 0
[pid=4021] ppid=4015 vsize=278616 CPUtime=0 cores=0,2,4,6
/proc/4021/stat : 4021 (python) S 4015 4015 2082 0 -1 4202560 733 0 0 0 0 0 0 0 25 0 1 0 658178791 285302784 5571 33554432000 4194304 5683644 140736223553824 18446744073709551615 268272709135 0 0 16781312 514 6843144596415840255 0 0 17 4 0 0 0
/proc/4021/statm: 69654 5571 354 364 0 5572 0
[pid=4023] ppid=4021 vsize=1017936 CPUtime=0.92 cores=0,2,4,6
/proc/4023/stat : 4023 (gnovelty+2) R 4021 4023 4023 0 -1 4202496 465 0 0 0 92 0 0 0 25 0 1 0 658178792 1042366464 197 33554432000 134512640 135663704 4287399264 18446744073709551615 134524546 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/4023/statm: 254484 197 113 282 0 254200 0
[pid=4024] ppid=4015 vsize=289424 CPUtime=0 cores=0,2,4,6
/proc/4024/stat : 4024 (python) S 4015 4015 2082 0 -1 4202560 735 0 0 0 0 0 0 0 25 0 1 0 658178792 296370176 5713 33554432000 4194304 5683644 140736223553824 18446744073709551615 268272709190 0 0 16781312 514 16149051198731714559 0 0 17 6 0 0 0
/proc/4024/statm: 72356 5713 348 364 0 8271 0
[pid=4026] ppid=4024 vsize=1017936 CPUtime=0.9 cores=0,2,4,6
/proc/4026/stat : 4026 (gnovelty+2) R 4024 4026 4026 0 -1 4202496 462 0 0 0 90 0 0 0 25 0 1 0 658178793 1042366464 196 33554432000 134512640 135663704 4290215168 18446744073709551615 134545622 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/4026/statm: 254484 196 113 282 0 254200 0
[pid=4027] ppid=4015 vsize=299172 CPUtime=0 cores=0,2,4,6
/proc/4027/stat : 4027 (python) S 4015 4015 2082 0 -1 4202560 750 0 0 0 0 0 0 0 17 0 1 0 658178793 306352128 5605 33554432000 4194304 5683644 140736223553824 18446744073709551615 268272709190 0 0 16781312 514 12359434290264506627 0 0 17 4 0 0 0
/proc/4027/statm: 74793 5605 348 364 0 10705 0
[pid=4031] ppid=4027 vsize=2828 CPUtime=0.39 cores=0,2,4,6
/proc/4031/stat : 4031 (march_hi) R 4027 4031 4031 0 -1 4202496 752 0 0 0 39 0 0 0 25 0 1 0 658178795 2895872 488 33554432000 4194304 4955518 140735508107280 18446744073709551615 4227827 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/4031/statm: 707 488 104 186 0 520 0
[pid=4029] ppid=4015 vsize=310036 CPUtime=0 cores=0,2,4,6
/proc/4029/stat : 4029 (python) S 4015 4015 2082 0 -1 4202560 760 0 0 0 0 0 0 0 19 0 1 0 658178794 317476864 5729 33554432000 4194304 5683644 140736223553824 18446744073709551615 268272709190 0 0 16781312 514 5556865800377172227 0 0 17 4 0 0 0
/proc/4029/statm: 77509 5729 348 364 0 13418 0
[pid=4032] ppid=4029 vsize=2980 CPUtime=0.89 cores=0,2,4,6
/proc/4032/stat : 4032 (march_hi) R 4029 4032 4032 0 -1 4202496 778 0 0 0 89 0 0 0 25 0 1 0 658178795 3051520 518 33554432000 4194304 4955518 140735153777200 18446744073709551615 4286629 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/4032/statm: 745 518 104 186 0 558 0
[pid=4015/tid=4022] ppid=4013 vsize=320280 CPUtime=0 cores=0,2,4,6
/proc/4015/task/4022/stat : 4022 (python) S 4013 4015 2082 0 -1 4202560 25 934 0 0 0 0 0 0 25 0 5 0 658178792 327966720 7266 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 4 0 0 0
[pid=4015/tid=4025] ppid=4013 vsize=320280 CPUtime=0 cores=0,2,4,6
/proc/4015/task/4025/stat : 4025 (python) S 4013 4015 2082 0 -1 4202560 24 934 0 0 0 0 0 0 25 0 5 0 658178793 327966720 7266 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 6 0 0 0
[pid=4015/tid=4028] ppid=4013 vsize=320280 CPUtime=0 cores=0,2,4,6
/proc/4015/task/4028/stat : 4028 (python) S 4013 4015 2082 0 -1 4202560 33 934 0 0 0 0 0 0 25 0 5 0 658178794 327966720 7266 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 4 0 0 0
[pid=4015/tid=4030] ppid=4013 vsize=320280 CPUtime=0 cores=0,2,4,6
/proc/4015/task/4030/stat : 4030 (python) S 4013 4015 2082 0 -1 4202560 21 934 0 0 0 0 0 0 25 0 5 0 658178795 327966720 7266 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3.59
Current children cumulated vsize (KiB) 3539208

[startup+3.10142 s]
/proc/loadavg: 10.23 10.17 9.53 12/484 4032
/proc/meminfo: memFree=26211008/32950928 swapFree=60199044/67111528
[pid=4015] ppid=4013 vsize=320280 CPUtime=0.49 cores=0,2,4,6
/proc/4015/stat : 4015 (python) S 4013 4015 2082 0 -1 4202496 12212 934 1 0 42 7 0 0 25 0 5 0 658178735 327966720 7266 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284515787 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/4015/statm: 80070 7266 1881 364 0 15979 0
[pid=4021] ppid=4015 vsize=278616 CPUtime=0 cores=0,2,4,6
/proc/4021/stat : 4021 (python) S 4015 4015 2082 0 -1 4202560 733 0 0 0 0 0 0 0 18 0 1 0 658178791 285302784 5571 33554432000 4194304 5683644 140736223553824 18446744073709551615 268272709135 0 0 16781312 514 6843144596415807748 0 0 17 4 0 0 0
/proc/4021/statm: 69654 5571 354 364 0 5572 0
[pid=4023] ppid=4021 vsize=1017936 CPUtime=2.52 cores=0,2,4,6
/proc/4023/stat : 4023 (gnovelty+2) R 4021 4023 4023 0 -1 4202496 465 0 0 0 252 0 0 0 25 0 1 0 658178792 1042366464 197 33554432000 134512640 135663704 4287399264 18446744073709551615 134524425 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/4023/statm: 254484 197 113 282 0 254200 0
[pid=4024] ppid=4015 vsize=289424 CPUtime=0 cores=0,2,4,6
/proc/4024/stat : 4024 (python) S 4015 4015 2082 0 -1 4202560 735 0 0 0 0 0 0 0 18 0 1 0 658178792 296370176 5713 33554432000 4194304 5683644 140736223553824 18446744073709551615 268272709190 0 0 16781312 514 16149051198731682049 0 0 17 6 0 0 0
/proc/4024/statm: 72356 5713 348 364 0 8271 0
[pid=4026] ppid=4024 vsize=1017936 CPUtime=2.48 cores=0,2,4,6
/proc/4026/stat : 4026 (gnovelty+2) R 4024 4026 4026 0 -1 4202496 462 0 0 0 248 0 0 0 25 0 1 0 658178793 1042366464 196 33554432000 134512640 135663704 4290215168 18446744073709551615 134545793 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/4026/statm: 254484 196 113 282 0 254200 0
[pid=4027] ppid=4015 vsize=299172 CPUtime=0 cores=0,2,4,6
/proc/4027/stat : 4027 (python) S 4015 4015 2082 0 -1 4202560 750 0 0 0 0 0 0 0 15 0 1 0 658178793 306352128 5605 33554432000 4194304 5683644 140736223553824 18446744073709551615 268272709190 0 0 16781312 514 12359434290264506627 0 0 17 4 0 0 0
/proc/4027/statm: 74793 5605 348 364 0 10705 0
[pid=4031] ppid=4027 vsize=2980 CPUtime=1.19 cores=0,2,4,6
/proc/4031/stat : 4031 (march_hi) R 4027 4031 4031 0 -1 4202496 795 0 0 0 119 0 0 0 25 0 1 0 658178795 3051520 531 33554432000 4194304 4955518 140735508107280 18446744073709551615 4238727 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/4031/statm: 745 531 104 186 0 558 0
[pid=4029] ppid=4015 vsize=310036 CPUtime=0 cores=0,2,4,6
/proc/4029/stat : 4029 (python) S 4015 4015 2082 0 -1 4202560 761 0 0 0 0 0 0 0 15 0 1 0 658178794 317476864 5729 33554432000 4194304 5683644 140736223553824 18446744073709551615 268272709190 0 0 16781312 514 15110501168524034047 0 0 17 4 0 0 0
/proc/4029/statm: 77509 5729 348 364 0 13418 0
[pid=4032] ppid=4029 vsize=3436 CPUtime=2.49 cores=0,2,4,6
/proc/4032/stat : 4032 (march_hi) R 4029 4032 4032 0 -1 4202496 893 0 0 0 249 0 0 0 25 0 1 0 658178795 3518464 633 33554432000 4194304 4955518 140735153777200 18446744073709551615 4227671 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/4032/statm: 859 633 104 186 0 672 0
[pid=4015/tid=4022] ppid=4013 vsize=320280 CPUtime=0 cores=0,2,4,6
/proc/4015/task/4022/stat : 4022 (python) S 4013 4015 2082 0 -1 4202560 25 934 0 0 0 0 0 0 25 0 5 0 658178792 327966720 7266 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 4 0 0 0
[pid=4015/tid=4025] ppid=4013 vsize=320280 CPUtime=0 cores=0,2,4,6
/proc/4015/task/4025/stat : 4025 (python) S 4013 4015 2082 0 -1 4202560 24 934 0 0 0 0 0 0 25 0 5 0 658178793 327966720 7266 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 6 0 0 0
[pid=4015/tid=4028] ppid=4013 vsize=320280 CPUtime=0 cores=0,2,4,6
/proc/4015/task/4028/stat : 4028 (python) S 4013 4015 2082 0 -1 4202560 33 934 0 0 0 0 0 0 25 0 5 0 658178794 327966720 7266 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 4 0 0 0
[pid=4015/tid=4030] ppid=4013 vsize=320280 CPUtime=0 cores=0,2,4,6
/proc/4015/task/4030/stat : 4030 (python) S 4013 4015 2082 0 -1 4202560 21 934 0 0 0 0 0 0 25 0 5 0 658178795 327966720 7266 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 9.17
Current children cumulated vsize (KiB) 3539816

[startup+6.30085 s]
/proc/loadavg: 10.21 10.16 9.53 11/484 4032
/proc/meminfo: memFree=26208560/32950928 swapFree=60199044/67111528
[pid=4015] ppid=4013 vsize=320280 CPUtime=0.49 cores=0,2,4,6
/proc/4015/stat : 4015 (python) S 4013 4015 2082 0 -1 4202496 12212 934 1 0 42 7 0 0 25 0 5 0 658178735 327966720 7266 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284515787 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/4015/statm: 80070 7266 1881 364 0 15979 0
[pid=4021] ppid=4015 vsize=278616 CPUtime=0.02 cores=0,2,4,6
/proc/4021/stat : 4021 (python) S 4015 4015 2082 0 -1 4202560 891 0 0 0 1 1 0 0 18 0 1 0 658178791 285302784 5594 33554432000 4194304 5683644 140736223553824 18446744073709551615 268272709135 0 0 16781312 514 6843144596415807748 0 0 17 4 0 0 0
/proc/4021/statm: 69654 5594 377 364 0 5572 0
[pid=4023] ppid=4021 vsize=1017936 CPUtime=5.72 cores=0,2,4,6
/proc/4023/stat : 4023 (gnovelty+2) R 4021 4023 4023 0 -1 4202496 465 0 0 0 572 0 0 0 25 0 1 0 658178792 1042366464 197 33554432000 134512640 135663704 4287399264 18446744073709551615 134525402 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/4023/statm: 254484 197 113 282 0 254200 0
[pid=4024] ppid=4015 vsize=289424 CPUtime=0.01 cores=0,2,4,6
/proc/4024/stat : 4024 (python) S 4015 4015 2082 0 -1 4202560 911 0 0 0 0 1 0 0 18 0 1 0 658178792 296370176 5736 33554432000 4194304 5683644 140736223553824 18446744073709551615 268272709190 0 0 16781312 514 16149051198731682049 0 0 17 6 0 0 0
/proc/4024/statm: 72356 5736 371 364 0 8271 0
[pid=4026] ppid=4024 vsize=1017936 CPUtime=5.61 cores=0,2,4,6
/proc/4026/stat : 4026 (gnovelty+2) R 4024 4026 4026 0 -1 4202496 462 0 0 0 561 0 0 0 25 0 1 0 658178793 1042366464 196 33554432000 134512640 135663704 4290215168 18446744073709551615 134525260 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/4026/statm: 254484 196 113 282 0 254200 0
[pid=4027] ppid=4015 vsize=299172 CPUtime=0.02 cores=0,2,4,6
/proc/4027/stat : 4027 (python) S 4015 4015 2082 0 -1 4202560 912 0 0 0 1 1 0 0 15 0 1 0 658178793 306352128 5628 33554432000 4194304 5683644 140736223553824 18446744073709551615 268272709190 0 0 16781312 514 12359434290264506627 0 0 17 4 0 0 0
/proc/4027/statm: 74793 5628 371 364 0 10705 0
[pid=4031] ppid=4027 vsize=3436 CPUtime=2.79 cores=0,2,4,6
/proc/4031/stat : 4031 (march_hi) R 4027 4031 4031 0 -1 4202496 908 0 0 0 279 0 0 0 25 0 1 0 658178795 3518464 644 33554432000 4194304 4955518 140735508107280 18446744073709551615 4227824 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/4031/statm: 859 644 104 186 0 672 0
[pid=4029] ppid=4015 vsize=310036 CPUtime=0.02 cores=0,2,4,6
/proc/4029/stat : 4029 (python) S 4015 4015 2082 0 -1 4202560 924 0 0 0 1 1 0 0 15 0 1 0 658178794 317476864 5751 33554432000 4194304 5683644 140736223553824 18446744073709551615 268272709190 0 0 16781312 514 5556865800377172227 0 0 17 4 0 0 0
/proc/4029/statm: 77509 5751 370 364 0 13418 0
[pid=4032] ppid=4029 vsize=4004 CPUtime=5.62 cores=0,2,4,6
/proc/4032/stat : 4032 (march_hi) R 4029 4032 4032 0 -1 4202496 1028 0 0 0 562 0 0 0 25 0 1 0 658178795 4100096 768 33554432000 4194304 4955518 140735153777200 18446744073709551615 4206496 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/4032/statm: 1001 768 104 186 0 814 0
[pid=4015/tid=4022] ppid=4013 vsize=320280 CPUtime=0 cores=0,2,4,6
/proc/4015/task/4022/stat : 4022 (python) S 4013 4015 2082 0 -1 4202560 25 934 0 0 0 0 0 0 25 0 5 0 658178792 327966720 7266 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 4 0 0 0
[pid=4015/tid=4025] ppid=4013 vsize=320280 CPUtime=0 cores=0,2,4,6
/proc/4015/task/4025/stat : 4025 (python) S 4013 4015 2082 0 -1 4202560 24 934 0 0 0 0 0 0 25 0 5 0 658178793 327966720 7266 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 6 0 0 0
[pid=4015/tid=4028] ppid=4013 vsize=320280 CPUtime=0 cores=0,2,4,6
/proc/4015/task/4028/stat : 4028 (python) S 4013 4015 2082 0 -1 4202560 33 934 0 0 0 0 0 0 25 0 5 0 658178794 327966720 7266 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 4 0 0 0
[pid=4015/tid=4030] ppid=4013 vsize=320280 CPUtime=0 cores=0,2,4,6
/proc/4015/task/4030/stat : 4030 (python) S 4013 4015 2082 0 -1 4202560 21 934 0 0 0 0 0 0 25 0 5 0 658178795 327966720 7266 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 20.3
Current children cumulated vsize (KiB) 3540840

[startup+12.7007 s]
/proc/loadavg: 10.19 10.16 9.53 11/484 4032
/proc/meminfo: memFree=26206240/32950928 swapFree=60199044/67111528
[pid=4015] ppid=4013 vsize=320280 CPUtime=0.49 cores=0,2,4,6
/proc/4015/stat : 4015 (python) S 4013 4015 2082 0 -1 4202496 12212 934 1 0 42 7 0 0 25 0 5 0 658178735 327966720 7266 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284515787 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/4015/statm: 80070 7266 1881 364 0 15979 0
[pid=4021] ppid=4015 vsize=278616 CPUtime=0.03 cores=0,2,4,6
/proc/4021/stat : 4021 (python) S 4015 4015 2082 0 -1 4202560 894 0 0 0 2 1 0 0 18 0 1 0 658178791 285302784 5594 33554432000 4194304 5683644 140736223553824 18446744073709551615 268272709135 0 0 16781312 514 14624788879002009860 0 0 17 4 0 0 0
/proc/4021/statm: 69654 5594 377 364 0 5572 0
[pid=4023] ppid=4021 vsize=1017936 CPUtime=12.12 cores=0,2,4,6
/proc/4023/stat : 4023 (gnovelty+2) R 4021 4023 4023 0 -1 4202496 465 0 0 0 1212 0 0 0 25 0 1 0 658178792 1042366464 197 33554432000 134512640 135663704 4287399264 18446744073709551615 134544196 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/4023/statm: 254484 197 113 282 0 254200 0
[pid=4024] ppid=4015 vsize=289424 CPUtime=0.03 cores=0,2,4,6

################
# More data... #
################

/proc/4080/stat : 4080 (kcnfs-2006) R 4078 4080 4080 0 -1 4202496 664 0 0 0 58928 2 0 0 25 0 1 0 658223543 186966016 405 33554432000 134512640 135163436 4290401024 18446744073709551615 134569074 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/4080/statm: 45646 405 77 159 0 45486 0
[pid=4078/tid=4167] ppid=4015 vsize=340964 CPUtime=0 cores=0,2,4,6
/proc/4078/task/4167/stat : 4167 (python) S 4015 4015 2082 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 658288358 349147136 5843 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 514 18446604445056680960 0 0 -1 2 0 0 0
[pid=4093] ppid=4015 vsize=351220 CPUtime=1.48 cores=0,2,4,6
/proc/4093/stat : 4093 (python) S 4015 4015 2082 0 -1 4202560 1063 0 0 0 77 71 0 0 15 0 1 0 658247031 359649280 5814 33554432000 4194304 5683644 140736223553824 18446744073709551615 268272709190 0 0 16781312 514 6546092013840728322 0 0 17 2 0 0 0
/proc/4093/statm: 87805 5814 377 364 0 23702 0
[pid=4095] ppid=4093 vsize=52304 CPUtime=251.06 cores=0,2,4,6
/proc/4095/stat : 4095 (march_hi) R 4093 4095 4095 0 -1 4202496 10186 0 0 0 25104 2 0 0 25 0 1 0 658247032 53559296 9875 33554432000 4194304 4955518 140733832900608 18446744073709551615 4307199 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/4095/statm: 13076 9875 104 186 0 12889 0
[pid=4015/tid=4022] ppid=4013 vsize=361024 CPUtime=0 cores=0,2,4,6
/proc/4015/task/4022/stat : 4022 (python) S 4013 4015 2082 0 -1 4202560 25 934 0 0 0 0 0 0 25 0 9 0 658178792 369688576 7253 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 4 0 0 0
[pid=4015/tid=4025] ppid=4013 vsize=361024 CPUtime=0 cores=0,2,4,6
/proc/4015/task/4025/stat : 4025 (python) S 4013 4015 2082 0 -1 4202560 24 934 0 0 0 0 0 0 25 0 9 0 658178793 369688576 7253 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 6 0 0 0
[pid=4015/tid=4028] ppid=4013 vsize=361024 CPUtime=0 cores=0,2,4,6
/proc/4015/task/4028/stat : 4028 (python) S 4013 4015 2082 0 -1 4202560 36 934 0 0 0 0 0 0 18 0 9 0 658178794 369688576 7253 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 4 0 0 0
[pid=4015/tid=4030] ppid=4013 vsize=361024 CPUtime=0 cores=0,2,4,6
/proc/4015/task/4030/stat : 4030 (python) S 4013 4015 2082 0 -1 4202560 21 934 0 0 0 0 0 0 25 0 9 0 658178795 369688576 7253 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 4 0 0 0
[pid=4015/tid=4042] ppid=4013 vsize=361024 CPUtime=0 cores=0,2,4,6
/proc/4015/task/4042/stat : 4042 (python) S 4013 4015 2082 0 -1 4202560 24 934 0 0 0 0 0 0 20 0 9 0 658188826 369688576 7253 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 4 0 0 0
[pid=4015/tid=4050] ppid=4013 vsize=361024 CPUtime=0 cores=0,2,4,6
/proc/4015/task/4050/stat : 4050 (python) S 4013 4015 2082 0 -1 4202560 33 934 0 0 0 0 0 0 18 0 9 0 658189241 369688576 7253 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 4 0 0 0
[pid=4015/tid=4079] ppid=4013 vsize=361024 CPUtime=0 cores=0,2,4,6
/proc/4015/task/4079/stat : 4079 (python) S 4013 4015 2082 0 -1 4202560 30 934 0 0 0 0 0 0 18 0 9 0 658223543 369688576 7253 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 4 0 0 0
[pid=4015/tid=4094] ppid=4013 vsize=361024 CPUtime=0 cores=0,2,4,6
/proc/4015/task/4094/stat : 4094 (python) S 4013 4015 2082 0 -1 4202560 31 934 0 0 0 0 0 0 20 0 9 0 658247031 369688576 7253 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3618.25
Current children cumulated vsize (KiB) 5642812

[startup+1103.5 s]
/proc/loadavg: 10.12 10.19 10.00 11/491 4167
/proc/meminfo: memFree=25807512/32950928 swapFree=60199044/67111528
[pid=4015] ppid=4013 vsize=361024 CPUtime=0.54 cores=0,2,4,6
/proc/4015/stat : 4015 (python) S 4013 4015 2082 0 -1 4202496 16171 934 1 0 46 8 0 0 18 0 9 0 658178735 369688576 7253 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284515787 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/4015/statm: 90256 7253 1890 364 0 26153 0
[pid=4021] ppid=4015 vsize=288860 CPUtime=0.33 cores=0,2,4,6
/proc/4021/stat : 4021 (python) S 4015 4015 2082 0 -1 4202560 1160 0 0 0 17 16 0 0 19 0 2 0 658178791 295792640 5638 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/4021/statm: 72215 5638 416 364 0 8133 0
[pid=4023] ppid=4021 vsize=1017936 CPUtime=100.31 cores=0,2,4,6
/proc/4023/stat : 4023 (gnovelty+2) T 4021 4023 4023 0 -1 4202496 465 0 0 0 10031 0 0 0 25 0 1 0 658178792 1042366464 197 33554432000 134512640 135663704 4287399264 18446744073709551615 134525544 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/4023/statm: 254484 197 113 282 0 254200 0
[pid=4021/tid=4040] ppid=4015 vsize=288860 CPUtime=0 cores=0,2,4,6
/proc/4021/task/4040/stat : 4040 (python) S 4015 4015 2082 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 658188825 295792640 5638 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 514 18446604445916052416 0 0 -1 4 0 0 0
[pid=4024] ppid=4015 vsize=289424 CPUtime=0.36 cores=0,2,4,6
/proc/4024/stat : 4024 (python) S 4015 4015 2082 0 -1 4202560 1101 0 0 0 19 17 0 0 19 0 2 0 658178792 296370176 5766 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/4024/statm: 72356 5766 400 364 0 8271 0
[pid=4026] ppid=4024 vsize=1017936 CPUtime=100.54 cores=0,2,4,6
/proc/4026/stat : 4026 (gnovelty+2) T 4024 4026 4026 0 -1 4202496 462 0 0 0 10054 0 0 0 25 0 1 0 658178793 1042366464 196 33554432000 134512640 135663704 4290215168 18446744073709551615 134525002 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/4026/statm: 254484 196 113 282 0 254200 0
[pid=4024/tid=4048] ppid=4015 vsize=289424 CPUtime=0 cores=0,2,4,6
/proc/4024/task/4048/stat : 4048 (python) S 4015 4015 2082 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 658189240 296370176 5766 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 514 18446604452377719936 0 0 -1 6 0 0 0
[pid=4027] ppid=4015 vsize=299172 CPUtime=1.81 cores=0,2,4,6
/proc/4027/stat : 4027 (python) S 4015 4015 2082 0 -1 4202560 1155 0 0 0 93 88 0 0 15 0 2 0 658178793 306352128 5658 33554432000 4194304 5683644 140736223553824 18446744073709551615 268272709190 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/4027/statm: 74793 5658 400 364 0 10705 0
[pid=4031] ppid=4027 vsize=82012 CPUtime=420.83 cores=0,2,4,6
/proc/4031/stat : 4031 (march_hi) R 4027 4031 4031 0 -1 4202496 16299 0 0 0 42080 3 0 0 25 0 1 0 658178795 83980288 15998 33554432000 4194304 4955518 140735508107280 18446744073709551615 4209688 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/4031/statm: 20503 15998 104 186 0 20316 0
[pid=4027/tid=4077] ppid=4015 vsize=299172 CPUtime=0 cores=0,2,4,6
/proc/4027/task/4077/stat : 4077 (python) S 4015 4015 2082 0 -1 4202560 44 0 0 0 0 0 0 0 17 0 2 0 658223542 306352128 5658 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 514 18446604447840947264 0 0 -1 4 0 0 0
[pid=4029] ppid=4015 vsize=310036 CPUtime=2.45 cores=0,2,4,6
/proc/4029/stat : 4029 (python) S 4015 4015 2082 0 -1 4202560 1165 0 0 0 127 118 0 0 16 0 2 0 658178794 317476864 5782 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/4029/statm: 77509 5782 400 364 0 13418 0
[pid=4032] ppid=4029 vsize=111480 CPUtime=679.84 cores=0,2,4,6
/proc/4032/stat : 4032 (march_hi) T 4029 4032 4032 0 -1 4202496 25495 0 0 0 67977 7 0 0 25 0 1 0 658178795 114155520 25198 33554432000 4194304 4955518 140735153777200 18446744073709551615 4290296 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/4032/statm: 27870 25198 104 186 0 27683 0
[pid=4029/tid=4092] ppid=4015 vsize=310036 CPUtime=0 cores=0,2,4,6
/proc/4029/task/4092/stat : 4092 (python) S 4015 4015 2082 0 -1 4202560 45 0 0 0 0 0 0 0 16 0 2 0 658247030 317476864 5782 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 514 18446604447840946368 0 0 -1 2 0 0 0
[pid=4041] ppid=4015 vsize=320292 CPUtime=3.42 cores=0,2,4,6
/proc/4041/stat : 4041 (python) S 4015 4015 2082 0 -1 4202560 1161 0 0 0 181 161 0 0 17 0 2 0 658188826 327979008 5803 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/4041/statm: 80073 5803 400 364 0 15979 0
[pid=4043] ppid=4041 vsize=111480 CPUtime=680.3 cores=0,2,4,6
/proc/4043/stat : 4043 (march_hi) T 4041 4043 4043 0 -1 4202496 25426 0 0 0 68024 6 0 0 25 0 1 0 658188826 114155520 25128 33554432000 4194304 4955518 140734342045488 18446744073709551615 4307000 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/4043/statm: 27870 25128 104 186 0 27683 0
[pid=4041/tid=4153] ppid=4015 vsize=320292 CPUtime=0 cores=0,2,4,6
/proc/4041/task/4153/stat : 4153 (python) S 4015 4015 2082 0 -1 4202560 44 0 0 0 0 0 0 0 17 0 2 0 658283697 327979008 5803 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 514 18446604445364177920 0 0 -1 4 0 0 0
[pid=4049] ppid=4015 vsize=330548 CPUtime=3.54 cores=0,2,4,6
/proc/4049/stat : 4049 (python) S 4015 4015 2082 0 -1 4202560 1170 0 0 0 179 175 0 0 15 0 2 0 658189241 338481152 5809 33554432000 4194304 5683644 140736223553824 18446744073709551615 268272709190 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/4049/statm: 82637 5809 400 364 0 18540 0
[pid=4051] ppid=4049 vsize=175540 CPUtime=780.47 cores=0,2,4,6
/proc/4051/stat : 4051 (march_hi) R 4049 4051 4051 0 -1 4202496 28979 0 0 0 78037 10 0 0 25 0 1 0 658189241 179752960 28684 33554432000 4194304 4955518 140734877163072 18446744073709551615 4346397 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/4051/statm: 43885 28684 104 186 0 43698 0
[pid=4049/tid=4107] ppid=4015 vsize=330548 CPUtime=0 cores=0,2,4,6
/proc/4049/task/4107/stat : 4107 (python) S 4015 4015 2082 0 -1 4202560 48 0 0 0 0 0 0 0 17 0 2 0 658274926 338481152 5809 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 514 18446604441961332800 0 0 -1 0 0 0 0
[pid=4078] ppid=4015 vsize=340964 CPUtime=2.27 cores=0,2,4,6
/proc/4078/stat : 4078 (python) S 4015 4015 2082 0 -1 4202560 1166 0 0 0 117 110 0 0 18 0 2 0 658223543 349147136 5843 33554432000 4194304 5683644 140736223553824 18446744073709551615 268272709190 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/4078/statm: 85241 5843 409 364 0 21141 0
[pid=4080] ppid=4078 vsize=182584 CPUtime=589.7 cores=0,2,4,6
/proc/4080/stat : 4080 (kcnfs-2006) R 4078 4080 4080 0 -1 4202496 664 0 0 0 58968 2 0 0 25 0 1 0 658223543 186966016 405 33554432000 134512640 135163436 4290401024 18446744073709551615 134561489 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/4080/statm: 45646 405 77 159 0 45486 0
[pid=4078/tid=4167] ppid=4015 vsize=340964 CPUtime=0 cores=0,2,4,6
/proc/4078/task/4167/stat : 4167 (python) S 4015 4015 2082 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 658288358 349147136 5843 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 514 18446604445056680960 0 0 -1 2 0 0 0
[pid=4093] ppid=4015 vsize=351220 CPUtime=1.48 cores=0,2,4,6
/proc/4093/stat : 4093 (python) S 4015 4015 2082 0 -1 4202560 1063 0 0 0 77 71 0 0 15 0 1 0 658247031 359649280 5814 33554432000 4194304 5683644 140736223553824 18446744073709551615 268272709190 0 0 16781312 514 6546092013840728322 0 0 17 2 0 0 0
/proc/4093/statm: 87805 5814 377 364 0 23702 0
[pid=4095] ppid=4093 vsize=52304 CPUtime=251.26 cores=0,2,4,6
/proc/4095/stat : 4095 (march_hi) R 4093 4095 4095 0 -1 4202496 10194 0 0 0 25124 2 0 0 25 0 1 0 658247032 53559296 9883 33554432000 4194304 4955518 140733832900608 18446744073709551615 4227800 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/4095/statm: 13076 9883 104 186 0 12889 0
[pid=4015/tid=4022] ppid=4013 vsize=361024 CPUtime=0 cores=0,2,4,6
/proc/4015/task/4022/stat : 4022 (python) S 4013 4015 2082 0 -1 4202560 25 934 0 0 0 0 0 0 25 0 9 0 658178792 369688576 7253 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 4 0 0 0
[pid=4015/tid=4025] ppid=4013 vsize=361024 CPUtime=0 cores=0,2,4,6
/proc/4015/task/4025/stat : 4025 (python) S 4013 4015 2082 0 -1 4202560 24 934 0 0 0 0 0 0 25 0 9 0 658178793 369688576 7253 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 6 0 0 0
[pid=4015/tid=4028] ppid=4013 vsize=361024 CPUtime=0 cores=0,2,4,6
/proc/4015/task/4028/stat : 4028 (python) S 4013 4015 2082 0 -1 4202560 36 934 0 0 0 0 0 0 18 0 9 0 658178794 369688576 7253 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 4 0 0 0
[pid=4015/tid=4030] ppid=4013 vsize=361024 CPUtime=0 cores=0,2,4,6
/proc/4015/task/4030/stat : 4030 (python) S 4013 4015 2082 0 -1 4202560 21 934 0 0 0 0 0 0 25 0 9 0 658178795 369688576 7253 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 4 0 0 0
[pid=4015/tid=4042] ppid=4013 vsize=361024 CPUtime=0 cores=0,2,4,6
/proc/4015/task/4042/stat : 4042 (python) S 4013 4015 2082 0 -1 4202560 24 934 0 0 0 0 0 0 20 0 9 0 658188826 369688576 7253 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 4 0 0 0
[pid=4015/tid=4050] ppid=4013 vsize=361024 CPUtime=0 cores=0,2,4,6
/proc/4015/task/4050/stat : 4050 (python) S 4013 4015 2082 0 -1 4202560 33 934 0 0 0 0 0 0 18 0 9 0 658189241 369688576 7253 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 4 0 0 0
[pid=4015/tid=4079] ppid=4013 vsize=361024 CPUtime=0 cores=0,2,4,6
/proc/4015/task/4079/stat : 4079 (python) S 4013 4015 2082 0 -1 4202560 30 934 0 0 0 0 0 0 18 0 9 0 658223543 369688576 7253 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 4 0 0 0
[pid=4015/tid=4094] ppid=4013 vsize=361024 CPUtime=0 cores=0,2,4,6
/proc/4015/task/4094/stat : 4094 (python) S 4013 4015 2082 0 -1 4202560 31 934 0 0 0 0 0 0 20 0 9 0 658247031 369688576 7253 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3619.45
Current children cumulated vsize (KiB) 5642812

[startup+1103.9 s]
/proc/loadavg: 10.12 10.19 10.00 11/491 4167
/proc/meminfo: memFree=25807512/32950928 swapFree=60199044/67111528
[pid=4015] ppid=4013 vsize=361564 CPUtime=1568.18 cores=0,2,4,6
/proc/4015/stat : 4015 (python) S 4013 4015 2082 0 -1 4202496 16285 57472 1 0 47 8 156434 329 17 0 9 0 658178735 370241536 7356 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284519231 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/4015/statm: 90391 7356 1890 364 0 26288 0
[pid=4027] ppid=4015 vsize=299172 CPUtime=1.81 cores=0,2,4,6
/proc/4027/stat : 4027 (python) S 4015 4015 2082 0 -1 4202560 1155 0 0 0 93 88 0 0 15 0 2 0 658178793 306352128 5658 33554432000 4194304 5683644 140736223553824 18446744073709551615 268272709190 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/4027/statm: 74793 5658 400 364 0 10705 0
[pid=4031] ppid=4027 vsize=82012 CPUtime=421.03 cores=0,2,4,6
/proc/4031/stat : 4031 (march_hi) R 4027 4031 4031 0 -1 4202496 16306 0 0 0 42100 3 0 0 25 0 1 0 658178795 83980288 16005 33554432000 4194304 4955518 140735508107280 18446744073709551615 4227571 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/4031/statm: 20503 16005 104 186 0 20316 0
[pid=4027/tid=4077] ppid=4015 vsize=299172 CPUtime=0 cores=0,2,4,6
/proc/4027/task/4077/stat : 4077 (python) S 4015 4015 2082 0 -1 4202560 44 0 0 0 0 0 0 0 17 0 2 0 658223542 306352128 5658 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 514 18446604447840947264 0 0 -1 4 0 0 0
[pid=4049] ppid=4015 vsize=330548 CPUtime=3.56 cores=0,2,4,6
/proc/4049/stat : 4049 (python) S 4015 4015 2082 0 -1 4202560 1173 0 0 0 180 176 0 0 16 0 2 0 658189241 338481152 5809 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284519231 0 0 16781824 2 18446744073709551615 0 0 17 4 0 0 0
/proc/4049/statm: 82637 5809 400 364 0 18540 0
[pid=4051] ppid=4049 vsize=175540 CPUtime=780.78 cores=0,2,4,6
/proc/4051/stat : 4051 (march_hi) R 4049 4051 4051 0 -1 4202496 28990 0 0 0 78068 10 0 0 25 0 1 0 658189241 179752960 28695 33554432000 4194304 4955518 140734877163072 18446744073709551615 4209737 256 0 16781312 0 0 0 0 17 2 0 0 0
/proc/4051/statm: 43885 28695 104 186 0 43698 0
[pid=4049/tid=4107] ppid=4015 vsize=330548 CPUtime=0 cores=0,2,4,6
/proc/4049/task/4107/stat : 4107 (python) S 4015 4015 2082 0 -1 4202560 48 0 0 0 0 0 0 0 15 0 2 0 658274926 338481152 5809 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781824 2 18446604441961332800 0 0 -1 0 0 0 0
[pid=4078] ppid=4015 vsize=340964 CPUtime=2.27 cores=0,2,4,6
/proc/4078/stat : 4078 (python) S 4015 4015 2082 0 -1 4202560 1166 0 0 0 117 110 0 0 18 0 2 0 658223543 349147136 5843 33554432000 4194304 5683644 140736223553824 18446744073709551615 268272709190 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/4078/statm: 85241 5843 409 364 0 21141 0
[pid=4080] ppid=4078 vsize=182584 CPUtime=590.07 cores=0,2,4,6
/proc/4080/stat : 4080 (kcnfs-2006) R 4078 4080 4080 0 -1 4202496 664 0 0 0 59005 2 0 0 25 0 1 0 658223543 186966016 405 33554432000 134512640 135163436 4290401024 18446744073709551615 134675111 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/4080/statm: 45646 405 77 159 0 45486 0
[pid=4078/tid=4167] ppid=4015 vsize=340964 CPUtime=0 cores=0,2,4,6
/proc/4078/task/4167/stat : 4167 (python) S 4015 4015 2082 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 658288358 349147136 5843 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 514 18446604445056680960 0 0 -1 2 0 0 0
[pid=4093] ppid=4015 vsize=351220 CPUtime=1.48 cores=0,2,4,6
/proc/4093/stat : 4093 (python) S 4015 4015 2082 0 -1 4202560 1063 0 0 0 77 71 0 0 15 0 1 0 658247031 359649280 5814 33554432000 4194304 5683644 140736223553824 18446744073709551615 268272709190 0 0 16781312 514 6546092013840728322 0 0 17 2 0 0 0
/proc/4093/statm: 87805 5814 377 364 0 23702 0
[pid=4095] ppid=4093 vsize=52304 CPUtime=251.54 cores=0,2,4,6
/proc/4095/stat : 4095 (march_hi) R 4093 4095 4095 0 -1 4202496 10205 0 0 0 25152 2 0 0 25 0 1 0 658247032 53559296 9894 33554432000 4194304 4955518 140733832900608 18446744073709551615 4227679 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/4095/statm: 13076 9894 104 186 0 12889 0
[pid=4015/tid=4022] ppid=4013 vsize=361564 CPUtime=1567.63 cores=0,2,4,6
/proc/4015/task/4022/stat : 4022 (python) S 4013 4015 2082 0 -1 4202560 25 57472 0 0 0 0 156434 329 25 0 9 0 658178792 370241536 7356 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 4 0 0 0
[pid=4015/tid=4025] ppid=4013 vsize=361564 CPUtime=1567.63 cores=0,2,4,6
/proc/4015/task/4025/stat : 4025 (python) S 4013 4015 2082 0 -1 4202560 24 57472 0 0 0 0 156434 329 25 0 9 0 658178793 370241536 7356 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 6 0 0 0
[pid=4015/tid=4028] ppid=4013 vsize=361564 CPUtime=1567.63 cores=0,2,4,6
/proc/4015/task/4028/stat : 4028 (python) S 4013 4015 2082 0 -1 4202560 36 57472 0 0 0 0 156434 329 18 0 9 0 658178794 370241536 7356 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 4 0 0 0
[pid=4015/tid=4030] ppid=4013 vsize=361564 CPUtime=1567.63 cores=0,2,4,6
/proc/4015/task/4030/stat : 4030 (python) S 4013 4015 2082 0 -1 4202560 21 57472 0 0 0 0 156434 329 25 0 9 0 658178795 370241536 7356 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 4 0 0 0
[pid=4015/tid=4042] ppid=4013 vsize=361564 CPUtime=1567.63 cores=0,2,4,6
/proc/4015/task/4042/stat : 4042 (python) S 4013 4015 2082 0 -1 4202560 24 57472 0 0 0 0 156434 329 20 0 9 0 658188826 370241536 7356 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 4 0 0 0
[pid=4015/tid=4050] ppid=4013 vsize=361564 CPUtime=1567.63 cores=0,2,4,6
/proc/4015/task/4050/stat : 4050 (python) S 4013 4015 2082 0 -1 4202560 33 57472 0 0 0 0 156434 329 18 0 9 0 658189241 370241536 7356 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 4 0 0 0
[pid=4015/tid=4079] ppid=4013 vsize=361564 CPUtime=1567.63 cores=0,2,4,6
/proc/4015/task/4079/stat : 4079 (python) S 4013 4015 2082 0 -1 4202560 30 57472 0 0 0 0 156434 329 18 0 9 0 658223543 370241536 7356 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 4 0 0 0
[pid=4015/tid=4094] ppid=4013 vsize=361564 CPUtime=1567.63 cores=0,2,4,6
/proc/4015/task/4094/stat : 4094 (python) S 4013 4015 2082 0 -1 4202560 31 57472 0 0 0 0 156434 329 20 0 9 0 658247031 370241536 7356 33554432000 4194304 5683644 140736223553824 18446744073709551615 268284512689 0 0 16781312 2 18446604446564379520 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3620.72
Current children cumulated vsize (KiB) 2175908

Child status: 0
Real time (s): 1103.95
CPU time (s): 3620.92
CPU user time (s): 3612.87
CPU system time (s): 8.05177
CPU usage (%): 327.996
Max. virtual memory (cumulated for all children) (KiB): 5643428

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 3612.87
system time used= 8.05177
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 138683
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 7177
involuntary context switches= 30600

runsolver used 5.9461 second user time and 12.0942 second system time

The end

Launcher Data 

Begin job on node122 at 2011-05-15 15:23:19
IDJOB=3378465
IDBENCH=82874
IDSOLVER=1859
FILE ID=node122/3378465-1305465799
RUNJOBID= node122-1305462946-2100
PBS_JOBID= 13324977
Free space on /tmp= 73380 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/medium/unif-k3-r4.26-v700-c2982-S1800377425-032.UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3378465-1305465799/watcher-3378465-1305465799 -o /tmp/evaluation-result-3378465-1305465799/solver-3378465-1305465799 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378465-1305465799.cnf -seed 266095689 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= d85bdfa4ee68fd1c7d18743407334b5d
RANDOM SEED=266095689

node122.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.64
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.97
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      26253196 kB
Buffers:        317472 kB
Cached:        3968504 kB
SwapCached:      50104 kB
Active:        2322084 kB
Inactive:      4119016 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      26253196 kB
SwapTotal:    67111528 kB
SwapFree:     60199044 kB
Dirty:           31856 kB
Writeback:           4 kB
AnonPages:     2116648 kB
Mapped:          31688 kB
Slab:           133628 kB
PageTables:      55172 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 124596092 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73348 MiB
End job on node122 at 2011-05-15 15:41:43