Trace number 3317395

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-randomUNSAT 1147.48 371.402

General information on the benchmark

Namerandom/medium/
unif-k5-r21.3-v100-c2130-S206772908-015.UNKNOWN.cnf
MD5SUM74f6f450e8e5ce3811b2d7ad1f57d348
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark326.484
Satisfiable
(Un)Satisfiability was proved
Number of variables100
Number of clauses2130
Sum of the clauses size10650
Maximum clause length5
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 52130
Number of clauses of size over 50

Solver Data 

0.32/0.45	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.32/0.45	c INFO: solving HOME/instance-3317395-1304269356.cnf
0.32/0.45	c INFO: computing task features
0.32/0.46	c INFO: computed variable-clause graph statistics
0.32/0.46	c INFO: computed clause balance statistics
0.41/0.52	c INFO: computed variable balance statistics
0.41/0.55	c INFO: computed small-clause counts
0.41/0.55	c INFO: computed Horn-clause counts
0.41/0.55	c INFO: computed variable graph statistics
0.41/0.55	c INFO: collected features for HOME/instance-3317395-1304269356.cnf in 0.10 s
0.41/0.58	c INFO: running gnovelty+2@0 for 97 with 4799 remaining (b = 0.62)
0.51/0.61	c INFO: running march_hi@0 for 194 with 4799 remaining (b = 0.19)
0.51/0.64	c INFO: running gnovelty+2@0 for 97 with 4799 remaining (b = 0.07)
0.51/0.67	c INFO: running march_hi@0 for 388 with 4799 remaining (b = 0.29)
317.41/99.79	c INFO: running march_hi@0 for 485 with 4700 remaining (b = 0.30)
422.82/129.99	c INFO: running march_hi@0 for 485 with 4670 remaining (b = 0.27)
992.60/319.81	c INFO: running march_hi@0 for 291 with 4480 remaining (b = 0.17)
1147.08/371.38	s UNSATISFIABLE

Verifier Data 

No possible verification on an UNSAT instance

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3317395-1304269356/watcher-3317395-1304269356 -o /tmp/evaluation-result-3317395-1304269356/solver-3317395-1304269356 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3317395-1304269356.cnf -seed 672243287 -budget 4800 -cores 4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.13 8.25 7.34 7/297 12147
/proc/meminfo: memFree=22576960/32951124 swapFree=67102140/67111528
[pid=12147] ppid=12145 vsize=15772 CPUtime=0 cores=1,3,5,7
/proc/12147/stat : 12147 (python) R 12145 12147 10986 0 -1 4202496 898 0 0 0 0 0 0 0 25 0 1 0 617166289 16150528 778 33554432000 4194304 5683644 140733496556240 18446744073709551615 216735221895 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/12147/statm: 3943 778 343 364 0 430 0

[startup+0.0516409 s]
/proc/loadavg: 8.13 8.25 7.34 7/297 12147
/proc/meminfo: memFree=22576960/32951124 swapFree=67102140/67111528
[pid=12147] ppid=12145 vsize=30112 CPUtime=0.04 cores=1,3,5,7
/proc/12147/stat : 12147 (python) S 12145 12147 10986 0 -1 4202496 1511 0 0 0 3 1 0 0 25 0 1 0 617166289 30834688 1348 33554432000 4194304 5683644 140733496556240 18446744073709551615 216735176704 0 2147483391 16781312 2 18446744071699199166 0 0 17 5 0 0 0
/proc/12147/statm: 7528 1348 443 364 0 898 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 30112

[startup+0.100631 s]
/proc/loadavg: 8.13 8.25 7.34 7/297 12147
/proc/meminfo: memFree=22576960/32951124 swapFree=67102140/67111528
[pid=12147] ppid=12145 vsize=104612 CPUtime=0.07 cores=1,3,5,7
/proc/12147/stat : 12147 (python) R 12145 12147 10986 0 -1 4202496 3144 0 0 0 6 1 0 0 17 0 1 0 617166289 107122688 2821 33554432000 4194304 5683644 140733496556240 18446744073709551615 216735221895 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/12147/statm: 26153 2821 847 364 0 1901 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 104612

[startup+0.300594 s]
/proc/loadavg: 8.13 8.25 7.34 7/297 12147
/proc/meminfo: memFree=22576960/32951124 swapFree=67102140/67111528
[pid=12147] ppid=12145 vsize=198916 CPUtime=0.23 cores=1,3,5,7
/proc/12147/stat : 12147 (python) R 12145 12147 10986 0 -1 4202496 7021 935 1 0 17 6 0 0 17 0 1 0 617166289 203689984 5053 33554432000 4194304 5683644 140733496556240 18446744073709551615 46952075031252 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/12147/statm: 49729 5053 1466 364 0 3756 0
Current children cumulated CPU time (s) 0.23
Current children cumulated vsize (KiB) 198916

[startup+0.700492 s]
/proc/loadavg: 8.13 8.25 7.34 7/297 12147
/proc/meminfo: memFree=22576960/32951124 swapFree=67102140/67111528
[pid=12147] ppid=12145 vsize=320164 CPUtime=0.57 cores=1,3,5,7
/proc/12147/stat : 12147 (python) S 12145 12147 10986 0 -1 4202496 12267 935 1 0 48 9 0 0 25 0 5 0 617166289 327847936 7238 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747005387 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/12147/statm: 80041 7238 1881 364 0 15950 0
Current children cumulated CPU time (s) 0.57
Current children cumulated vsize (KiB) 320164

[startup+1.50135 s]
/proc/loadavg: 8.13 8.25 7.34 10/310 12164
/proc/meminfo: memFree=22537308/32951124 swapFree=67102140/67111528
[pid=12147] ppid=12145 vsize=320164 CPUtime=0.57 cores=1,3,5,7
/proc/12147/stat : 12147 (python) S 12145 12147 10986 0 -1 4202496 12267 935 1 0 48 9 0 0 25 0 5 0 617166289 327847936 7238 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747005387 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/12147/statm: 80041 7238 1881 364 0 15950 0
[pid=12153] ppid=12147 vsize=278452 CPUtime=0 cores=1,3,5,7
/proc/12153/stat : 12153 (python) S 12147 12147 10986 0 -1 4202560 738 0 0 0 0 0 0 0 18 0 1 0 617166347 285134848 5550 33554432000 4194304 5683644 140733496556240 18446744073709551615 216735198735 0 0 16781312 514 2747706775019978751 0 0 17 5 0 0 0
/proc/12153/statm: 69613 5550 354 364 0 5531 0
[pid=12155] ppid=12153 vsize=1017800 CPUtime=0.91 cores=1,3,5,7
/proc/12155/stat : 12155 (gnovelty+2) R 12153 12155 12155 0 -1 4202496 437 0 0 0 91 0 0 0 21 0 1 0 617166348 1042227200 172 33554432000 134512640 135663704 4287644240 18446744073709551615 134524498 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/12155/statm: 254450 172 113 282 0 254166 0
[pid=12156] ppid=12147 vsize=289276 CPUtime=0 cores=1,3,5,7
/proc/12156/stat : 12156 (python) S 12147 12147 10986 0 -1 4202560 751 0 0 0 0 0 0 0 15 0 1 0 617166350 296218624 5676 33554432000 4194304 5683644 140733496556240 18446744073709551615 216735198790 0 0 16781312 514 7282610872822759684 0 0 17 7 0 0 0
/proc/12156/statm: 72319 5676 348 364 0 8234 0
[pid=12158] ppid=12156 vsize=1560 CPUtime=0.51 cores=1,3,5,7
/proc/12158/stat : 12158 (march_hi) R 12156 12158 12158 0 -1 4202496 503 0 0 0 51 0 0 0 23 0 1 0 617166351 1597440 240 33554432000 4194304 4955518 140735268431072 18446744073709551615 4237736 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/12158/statm: 390 240 106 186 0 203 0
[pid=12159] ppid=12147 vsize=299532 CPUtime=0 cores=1,3,5,7
/proc/12159/stat : 12159 (python) S 12147 12147 10986 0 -1 4202560 747 0 0 0 0 0 0 0 22 0 1 0 617166353 306720768 5680 33554432000 4194304 5683644 140733496556240 18446744073709551615 216735198790 0 0 16781312 514 4431818015046107135 0 0 17 5 0 0 0
/proc/12159/statm: 74883 5680 348 364 0 10795 0
[pid=12161] ppid=12159 vsize=1017800 CPUtime=0.35 cores=1,3,5,7
/proc/12161/stat : 12161 (gnovelty+2) R 12159 12161 12161 0 -1 4202496 436 0 0 0 35 0 0 0 25 0 1 0 617166356 1042227200 172 33554432000 134512640 135663704 4294170688 18446744073709551615 134524548 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/12161/statm: 254450 172 113 282 0 254166 0
[pid=12162] ppid=12147 vsize=309920 CPUtime=0 cores=1,3,5,7
/proc/12162/stat : 12162 (python) S 12147 12147 10986 0 -1 4202560 764 0 0 0 0 0 0 0 18 0 1 0 617166357 317358080 5701 33554432000 4194304 5683644 140733496556240 18446744073709551615 216735198790 0 0 16781312 514 15921873954491302149 0 0 17 5 0 0 0
/proc/12162/statm: 77480 5701 348 364 0 13389 0
[pid=12164] ppid=12162 vsize=1560 CPUtime=0.82 cores=1,3,5,7
/proc/12164/stat : 12164 (march_hi) R 12162 12164 12164 0 -1 4202496 503 0 0 0 82 0 0 0 25 0 1 0 617166357 1597440 239 33554432000 4194304 4955518 140736378687696 18446744073709551615 4297451 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/12164/statm: 390 239 106 186 0 203 0
[pid=12147/tid=12154] ppid=12145 vsize=320164 CPUtime=0 cores=1,3,5,7
/proc/12147/task/12154/stat : 12154 (python) S 12145 12147 10986 0 -1 4202560 27 935 0 0 0 0 0 0 19 0 5 0 617166348 327847936 7238 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 2 18446604457180261440 0 0 -1 5 0 0 0
[pid=12147/tid=12157] ppid=12145 vsize=320164 CPUtime=0 cores=1,3,5,7
/proc/12147/task/12157/stat : 12157 (python) S 12145 12147 10986 0 -1 4202560 25 935 0 0 0 0 0 0 21 0 5 0 617166351 327847936 7238 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 2 18446604457180261440 0 0 -1 7 0 0 0
[pid=12147/tid=12160] ppid=12145 vsize=320164 CPUtime=0 cores=1,3,5,7
/proc/12147/task/12160/stat : 12160 (python) S 12145 12147 10986 0 -1 4202560 23 935 0 0 0 0 0 0 23 0 5 0 617166354 327847936 7238 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 2 18446604457180261440 0 0 -1 5 0 0 0
[pid=12147/tid=12163] ppid=12145 vsize=320164 CPUtime=0 cores=1,3,5,7
/proc/12147/task/12163/stat : 12163 (python) S 12145 12147 10986 0 -1 4202560 28 935 0 0 0 0 0 0 25 0 5 0 617166357 327847936 7238 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 2 18446604457180261440 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3.16
Current children cumulated vsize (KiB) 3536064

[startup+3.10107 s]
/proc/loadavg: 8.13 8.25 7.34 10/310 12164
/proc/meminfo: memFree=22536836/32951124 swapFree=67102140/67111528
[pid=12147] ppid=12145 vsize=320164 CPUtime=0.57 cores=1,3,5,7
/proc/12147/stat : 12147 (python) S 12145 12147 10986 0 -1 4202496 12267 935 1 0 48 9 0 0 25 0 5 0 617166289 327847936 7238 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747005387 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/12147/statm: 80041 7238 1881 364 0 15950 0
[pid=12153] ppid=12147 vsize=278452 CPUtime=0 cores=1,3,5,7
/proc/12153/stat : 12153 (python) S 12147 12147 10986 0 -1 4202560 738 0 0 0 0 0 0 0 18 0 1 0 617166347 285134848 5550 33554432000 4194304 5683644 140733496556240 18446744073709551615 216735198735 0 0 16781312 514 2747706775019946248 0 0 17 5 0 0 0
/proc/12153/statm: 69613 5550 354 364 0 5531 0
[pid=12155] ppid=12153 vsize=1017800 CPUtime=2.51 cores=1,3,5,7
/proc/12155/stat : 12155 (gnovelty+2) R 12153 12155 12155 0 -1 4202496 437 0 0 0 251 0 0 0 25 0 1 0 617166348 1042227200 172 33554432000 134512640 135663704 4287644240 18446744073709551615 135008500 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/12155/statm: 254450 172 113 282 0 254166 0
[pid=12156] ppid=12147 vsize=289276 CPUtime=0 cores=1,3,5,7
/proc/12156/stat : 12156 (python) S 12147 12147 10986 0 -1 4202560 753 0 0 0 0 0 0 0 15 0 1 0 617166350 296218624 5676 33554432000 4194304 5683644 140733496556240 18446744073709551615 216735198790 0 0 16781312 514 7282610872822759684 0 0 17 7 0 0 0
/proc/12156/statm: 72319 5676 348 364 0 8234 0
[pid=12158] ppid=12156 vsize=1560 CPUtime=1.31 cores=1,3,5,7
/proc/12158/stat : 12158 (march_hi) R 12156 12158 12158 0 -1 4202496 503 0 0 0 131 0 0 0 25 0 1 0 617166351 1597440 240 33554432000 4194304 4955518 140735268431072 18446744073709551615 4297483 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/12158/statm: 390 240 106 186 0 203 0
[pid=12159] ppid=12147 vsize=299532 CPUtime=0 cores=1,3,5,7
/proc/12159/stat : 12159 (python) S 12147 12147 10986 0 -1 4202560 747 0 0 0 0 0 0 0 18 0 1 0 617166353 306720768 5680 33554432000 4194304 5683644 140733496556240 18446744073709551615 216735198790 0 0 16781312 514 4431818015046074631 0 0 17 5 0 0 0
/proc/12159/statm: 74883 5680 348 364 0 10795 0
[pid=12161] ppid=12159 vsize=1017800 CPUtime=1.14 cores=1,3,5,7
/proc/12161/stat : 12161 (gnovelty+2) R 12159 12161 12161 0 -1 4202496 436 0 0 0 114 0 0 0 25 0 1 0 617166356 1042227200 172 33554432000 134512640 135663704 4294170688 18446744073709551615 134525107 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/12161/statm: 254450 172 113 282 0 254166 0
[pid=12162] ppid=12147 vsize=309920 CPUtime=0 cores=1,3,5,7
/proc/12162/stat : 12162 (python) S 12147 12147 10986 0 -1 4202560 768 0 0 0 0 0 0 0 15 0 1 0 617166357 317358080 5701 33554432000 4194304 5683644 140733496556240 18446744073709551615 216735198790 0 0 16781312 514 15921873954491302149 0 0 17 5 0 0 0
/proc/12162/statm: 77480 5701 348 364 0 13389 0
[pid=12164] ppid=12162 vsize=1560 CPUtime=2.42 cores=1,3,5,7
/proc/12164/stat : 12164 (march_hi) R 12162 12164 12164 0 -1 4202496 509 0 0 0 242 0 0 0 25 0 1 0 617166357 1597440 245 33554432000 4194304 4955518 140736378687696 18446744073709551615 4285313 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/12164/statm: 390 245 106 186 0 203 0
[pid=12147/tid=12154] ppid=12145 vsize=320164 CPUtime=0 cores=1,3,5,7
/proc/12147/task/12154/stat : 12154 (python) S 12145 12147 10986 0 -1 4202560 27 935 0 0 0 0 0 0 19 0 5 0 617166348 327847936 7238 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 2 18446604457180261440 0 0 -1 5 0 0 0
[pid=12147/tid=12157] ppid=12145 vsize=320164 CPUtime=0 cores=1,3,5,7
/proc/12147/task/12157/stat : 12157 (python) S 12145 12147 10986 0 -1 4202560 25 935 0 0 0 0 0 0 21 0 5 0 617166351 327847936 7238 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 2 18446604457180261440 0 0 -1 7 0 0 0
[pid=12147/tid=12160] ppid=12145 vsize=320164 CPUtime=0 cores=1,3,5,7
/proc/12147/task/12160/stat : 12160 (python) S 12145 12147 10986 0 -1 4202560 23 935 0 0 0 0 0 0 23 0 5 0 617166354 327847936 7238 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 2 18446604457180261440 0 0 -1 5 0 0 0
[pid=12147/tid=12163] ppid=12145 vsize=320164 CPUtime=0 cores=1,3,5,7
/proc/12147/task/12163/stat : 12163 (python) S 12145 12147 10986 0 -1 4202560 28 935 0 0 0 0 0 0 25 0 5 0 617166357 327847936 7238 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 2 18446604457180261440 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 7.95
Current children cumulated vsize (KiB) 3536064

[startup+6.30051 s]
/proc/loadavg: 8.28 8.28 7.36 10/310 12164
/proc/meminfo: memFree=22533064/32951124 swapFree=67102140/67111528
[pid=12147] ppid=12145 vsize=320164 CPUtime=0.57 cores=1,3,5,7
/proc/12147/stat : 12147 (python) S 12145 12147 10986 0 -1 4202496 12267 935 1 0 48 9 0 0 25 0 5 0 617166289 327847936 7238 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747005387 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/12147/statm: 80041 7238 1881 364 0 15950 0
[pid=12153] ppid=12147 vsize=278452 CPUtime=0 cores=1,3,5,7
/proc/12153/stat : 12153 (python) S 12147 12147 10986 0 -1 4202560 899 0 0 0 0 0 0 0 18 0 1 0 617166347 285134848 5573 33554432000 4194304 5683644 140733496556240 18446744073709551615 216735198735 0 0 16781312 514 2747706775019946248 0 0 17 5 0 0 0
/proc/12153/statm: 69613 5573 377 364 0 5531 0
[pid=12155] ppid=12153 vsize=1017800 CPUtime=5.71 cores=1,3,5,7
/proc/12155/stat : 12155 (gnovelty+2) R 12153 12155 12155 0 -1 4202496 437 0 0 0 571 0 0 0 25 0 1 0 617166348 1042227200 172 33554432000 134512640 135663704 4287644240 18446744073709551615 134525402 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/12155/statm: 254450 172 113 282 0 254166 0
[pid=12156] ppid=12147 vsize=289276 CPUtime=0 cores=1,3,5,7
/proc/12156/stat : 12156 (python) S 12147 12147 10986 0 -1 4202560 911 0 0 0 0 0 0 0 15 0 1 0 617166350 296218624 5699 33554432000 4194304 5683644 140733496556240 18446744073709551615 216735198790 0 0 16781312 514 7282610872822759684 0 0 17 7 0 0 0
/proc/12156/statm: 72319 5699 371 364 0 8234 0
[pid=12158] ppid=12156 vsize=1768 CPUtime=2.9 cores=1,3,5,7
/proc/12158/stat : 12158 (march_hi) R 12156 12158 12158 0 -1 4202496 526 0 0 0 290 0 0 0 25 0 1 0 617166351 1810432 263 33554432000 4194304 4955518 140735268431072 18446744073709551615 4285584 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/12158/statm: 442 263 106 186 0 255 0
[pid=12159] ppid=12147 vsize=299532 CPUtime=0.01 cores=1,3,5,7
/proc/12159/stat : 12159 (python) S 12147 12147 10986 0 -1 4202560 910 0 0 0 1 0 0 0 18 0 1 0 617166353 306720768 5703 33554432000 4194304 5683644 140733496556240 18446744073709551615 216735198790 0 0 16781312 514 4431818015046074631 0 0 17 5 0 0 0
/proc/12159/statm: 74883 5703 371 364 0 10795 0
[pid=12161] ppid=12159 vsize=1017800 CPUtime=2.72 cores=1,3,5,7
/proc/12161/stat : 12161 (gnovelty+2) R 12159 12161 12161 0 -1 4202496 436 0 0 0 272 0 0 0 25 0 1 0 617166356 1042227200 172 33554432000 134512640 135663704 4294170688 18446744073709551615 135008463 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/12161/statm: 254450 172 113 282 0 254166 0
[pid=12162] ppid=12147 vsize=309920 CPUtime=0.01 cores=1,3,5,7
/proc/12162/stat : 12162 (python) S 12147 12147 10986 0 -1 4202560 937 0 0 0 0 1 0 0 15 0 1 0 617166357 317358080 5724 33554432000 4194304 5683644 140733496556240 18446744073709551615 216735198790 0 0 16781312 514 15921873954491302149 0 0 17 5 0 0 0
/proc/12162/statm: 77480 5724 371 364 0 13389 0
[pid=12164] ppid=12162 vsize=1768 CPUtime=5.58 cores=1,3,5,7
/proc/12164/stat : 12164 (march_hi) R 12162 12164 12164 0 -1 4202496 541 0 0 0 558 0 0 0 25 0 1 0 617166357 1810432 277 33554432000 4194304 4955518 140736378687696 18446744073709551615 4208656 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/12164/statm: 442 277 106 186 0 255 0
[pid=12147/tid=12154] ppid=12145 vsize=320164 CPUtime=0 cores=1,3,5,7
/proc/12147/task/12154/stat : 12154 (python) S 12145 12147 10986 0 -1 4202560 27 935 0 0 0 0 0 0 19 0 5 0 617166348 327847936 7238 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 2 18446604457180261440 0 0 -1 5 0 0 0
[pid=12147/tid=12157] ppid=12145 vsize=320164 CPUtime=0 cores=1,3,5,7
/proc/12147/task/12157/stat : 12157 (python) S 12145 12147 10986 0 -1 4202560 25 935 0 0 0 0 0 0 21 0 5 0 617166351 327847936 7238 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 2 18446604457180261440 0 0 -1 7 0 0 0
[pid=12147/tid=12160] ppid=12145 vsize=320164 CPUtime=0 cores=1,3,5,7
/proc/12147/task/12160/stat : 12160 (python) S 12145 12147 10986 0 -1 4202560 23 935 0 0 0 0 0 0 23 0 5 0 617166354 327847936 7238 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 2 18446604457180261440 0 0 -1 5 0 0 0
[pid=12147/tid=12163] ppid=12145 vsize=320164 CPUtime=0 cores=1,3,5,7
/proc/12147/task/12163/stat : 12163 (python) S 12145 12147 10986 0 -1 4202560 28 935 0 0 0 0 0 0 25 0 5 0 617166357 327847936 7238 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 2 18446604457180261440 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 17.5
Current children cumulated vsize (KiB) 3536480

[startup+12.7014 s]
/proc/loadavg: 8.34 8.29 7.37 10/310 12164
/proc/meminfo: memFree=22529740/32951124 swapFree=67102140/67111528
[pid=12147] ppid=12145 vsize=320164 CPUtime=0.57 cores=1,3,5,7
/proc/12147/stat : 12147 (python) S 12145 12147 10986 0 -1 4202496 12267 935 1 0 48 9 0 0 25 0 5 0 617166289 327847936 7238 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747005387 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/12147/statm: 80041 7238 1881 364 0 15950 0
[pid=12153] ppid=12147 vsize=278452 CPUtime=0.02 cores=1,3,5,7
/proc/12153/stat : 12153 (python) S 12147 12147 10986 0 -1 4202560 902 0 0 0 1 1 0 0 18 0 1 0 617166347 285134848 5573 33554432000 4194304 5683644 140733496556240 18446744073709551615 216735198735 0 0 16781312 514 17009321736392376584 0 0 17 5 0 0 0
/proc/12153/statm: 69613 5573 377 364 0 5531 0
[pid=12155] ppid=12153 vsize=1017800 CPUtime=12.11 cores=1,3,5,7
/proc/12155/stat : 12155 (gnovelty+2) R 12153 12155 12155 0 -1 4202496 437 0 0 0 1211 0 0 0 25 0 1 0 617166348 1042227200 172 33554432000 134512640 135663704 4287644240 18446744073709551615 134544212 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/12155/statm: 254450 172 113 282 0 254166 0
[pid=12156] ppid=12147 vsize=289276 CPUtime=0.01 cores=1,3,5,7

################
# More data... #
################

/proc/12161/statm: 254450 172 113 282 0 254166 0
[pid=12159/tid=12176] ppid=12147 vsize=299532 CPUtime=0 cores=1,3,5,7
/proc/12159/task/12176/stat : 12176 (python) S 12147 12147 10986 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 617179286 306720768 5733 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 514 18446604471124425472 0 0 -1 5 0 0 0
[pid=12162] ppid=12147 vsize=309920 CPUtime=1.84 cores=1,3,5,7
/proc/12162/stat : 12162 (python) R 12147 12147 10986 0 -1 4202560 1022 0 0 0 135 49 0 0 17 0 1 0 617166357 317358080 5724 33554432000 4194304 5683644 140733496556240 18446744073709551615 216735198790 0 0 16781312 514 0 0 0 17 3 0 0 0
/proc/12162/statm: 77480 5724 371 364 0 13389 0
[pid=12164] ppid=12162 vsize=0 CPUtime=331.5 cores=1,3,5,7
/proc/12164/stat : 12164 (march_hi) R 12162 12164 12164 0 -1 4202500 4748 0 0 0 33149 1 0 0 25 0 1 0 617166357 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 3 0 0 0
/proc/12164/statm: 0 0 0 0 0 0 0
[pid=12172] ppid=12147 vsize=320176 CPUtime=0.57 cores=1,3,5,7
/proc/12172/stat : 12172 (python) S 12147 12147 10986 0 -1 4202560 1026 0 0 0 31 26 0 0 15 0 1 0 617176269 327860224 5728 33554432000 4194304 5683644 140733496556240 18446744073709551615 216735198790 0 0 16781312 514 11945995927988175111 0 0 17 5 0 0 0
/proc/12172/statm: 80044 5728 371 364 0 15950 0
[pid=12174] ppid=12172 vsize=13680 CPUtime=268.48 cores=1,3,5,7
/proc/12174/stat : 12174 (march_hi) R 12172 12174 12174 0 -1 4202496 3490 0 0 0 26847 1 0 0 25 0 1 0 617176269 14008320 3208 33554432000 4194304 4955518 140736505721648 18446744073709551615 4224471 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/12174/statm: 3420 3208 106 186 0 3233 0
[pid=12177] ppid=12147 vsize=330172 CPUtime=0.5 cores=1,3,5,7
/proc/12177/stat : 12177 (python) S 12147 12147 10986 0 -1 4202560 1019 0 0 0 28 22 0 0 15 0 1 0 617179289 338096128 5667 33554432000 4194304 5683644 140733496556240 18446744073709551615 216735198790 0 0 16781312 514 4509629353432154376 0 0 17 5 0 0 0
/proc/12177/statm: 82543 5667 371 364 0 18446 0
[pid=12179] ppid=12177 vsize=6856 CPUtime=120.19 cores=1,3,5,7
/proc/12179/stat : 12179 (march_hi) R 12177 12179 12179 0 -1 4202496 1503 0 0 0 12019 0 0 0 25 0 1 0 617179289 7020544 1218 33554432000 4194304 4955518 140736064435840 18446744073709551615 4285951 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/12179/statm: 1714 1218 106 186 0 1527 0
[pid=12147/tid=12154] ppid=12145 vsize=340768 CPUtime=0 cores=1,3,5,7
/proc/12147/task/12154/stat : 12154 (python) S 12145 12147 10986 0 -1 4202560 27 935 0 0 0 0 0 0 19 0 7 0 617166348 348946432 7274 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 2 18446604457180261440 0 0 -1 5 0 0 0
[pid=12147/tid=12157] ppid=12145 vsize=340768 CPUtime=0 cores=1,3,5,7
/proc/12147/task/12157/stat : 12157 (python) S 12145 12147 10986 0 -1 4202560 28 935 0 0 0 0 0 0 18 0 7 0 617166351 348946432 7274 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 2 18446604457180261440 0 0 -1 7 0 0 0
[pid=12147/tid=12160] ppid=12145 vsize=340768 CPUtime=0 cores=1,3,5,7
/proc/12147/task/12160/stat : 12160 (python) S 12145 12147 10986 0 -1 4202560 23 935 0 0 0 0 0 0 23 0 7 0 617166354 348946432 7274 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 2 18446604457180261440 0 0 -1 5 0 0 0
[pid=12147/tid=12163] ppid=12145 vsize=340768 CPUtime=0 cores=1,3,5,7
/proc/12147/task/12163/stat : 12163 (python) S 12145 12147 10986 0 -1 4202560 28 935 0 0 0 0 0 0 25 0 7 0 617166357 348946432 7274 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 2 18446604457180261440 0 0 -1 5 0 0 0
[pid=12147/tid=12173] ppid=12145 vsize=340768 CPUtime=0 cores=1,3,5,7
/proc/12147/task/12173/stat : 12173 (python) S 12145 12147 10986 0 -1 4202560 33 935 0 0 0 0 0 0 20 0 7 0 617176269 348946432 7274 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 2 18446604457180261440 0 0 -1 5 0 0 0
[pid=12147/tid=12178] ppid=12145 vsize=340768 CPUtime=0 cores=1,3,5,7
/proc/12147/task/12178/stat : 12178 (python) S 12145 12147 10986 0 -1 4202560 26 935 0 0 0 0 0 0 20 0 7 0 617179289 348946432 7274 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 2 18446604457180261440 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 1144.08
Current children cumulated vsize (KiB) 4248180

[startup+371.102 s]
/proc/loadavg: 9.14 8.85 7.91 11/323 12187
/proc/meminfo: memFree=22311384/32951124 swapFree=67102140/67111528
[pid=12147] ppid=12145 vsize=340768 CPUtime=0.67 cores=1,3,5,7
/proc/12147/stat : 12147 (python) S 12145 12147 10986 0 -1 4202496 14588 935 1 0 58 9 0 0 18 0 7 0 617166289 348946432 7274 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747005387 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/12147/statm: 85192 7274 1886 364 0 21095 0
[pid=12153] ppid=12147 vsize=288696 CPUtime=0.17 cores=1,3,5,7
/proc/12153/stat : 12153 (python) S 12147 12147 10986 0 -1 4202560 1089 0 0 0 10 7 0 0 19 0 2 0 617166347 295624704 5611 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747005387 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/12153/statm: 72174 5611 410 364 0 8092 0
[pid=12155] ppid=12153 vsize=1017800 CPUtime=99.17 cores=1,3,5,7
/proc/12155/stat : 12155 (gnovelty+2) T 12153 12155 12155 0 -1 4202496 437 0 0 0 9917 0 0 0 25 0 1 0 617166348 1042227200 172 33554432000 134512640 135663704 4287644240 18446744073709551615 134524470 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/12155/statm: 254450 172 113 282 0 254166 0
[pid=12153/tid=12171] ppid=12147 vsize=288696 CPUtime=0 cores=1,3,5,7
/proc/12153/task/12171/stat : 12171 (python) S 12147 12147 10986 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 617176266 295624704 5611 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 514 18446604470732312768 0 0 -1 1 0 0 0
[pid=12156] ppid=12147 vsize=289276 CPUtime=0.77 cores=1,3,5,7
/proc/12156/stat : 12156 (python) S 12147 12147 10986 0 -1 4202560 1141 0 0 0 42 35 0 0 15 0 2 0 617166350 296218624 5729 33554432000 4194304 5683644 140733496556240 18446744073709551615 216735198790 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/12156/statm: 72319 5729 400 364 0 8234 0
[pid=12158] ppid=12156 vsize=13504 CPUtime=220.33 cores=1,3,5,7
/proc/12158/stat : 12158 (march_hi) R 12156 12158 12158 0 -1 4202496 2723 0 0 0 22032 1 0 0 25 0 1 0 617166351 13828096 2442 33554432000 4194304 4955518 140735268431072 18446744073709551615 4297061 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/12158/statm: 3376 2442 106 186 0 3189 0
[pid=12156/tid=12186] ppid=12147 vsize=289276 CPUtime=0 cores=1,3,5,7
/proc/12156/task/12186/stat : 12186 (python) S 12147 12147 10986 0 -1 4202560 46 0 0 0 0 0 0 0 16 0 2 0 617198268 296218624 5729 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 514 18446604469527134528 0 0 -1 5 0 0 0
[pid=12159] ppid=12147 vsize=299532 CPUtime=0.23 cores=1,3,5,7
/proc/12159/stat : 12159 (python) S 12147 12147 10986 0 -1 4202560 1111 0 0 0 12 11 0 0 19 0 2 0 617166353 306720768 5733 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747005387 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/12159/statm: 74883 5733 400 364 0 10795 0
[pid=12161] ppid=12159 vsize=1017800 CPUtime=100.06 cores=1,3,5,7
/proc/12161/stat : 12161 (gnovelty+2) T 12159 12161 12161 0 -1 4202496 436 0 0 0 10006 0 0 0 25 0 1 0 617166356 1042227200 172 33554432000 134512640 135663704 4294170688 18446744073709551615 134524945 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/12161/statm: 254450 172 113 282 0 254166 0
[pid=12159/tid=12176] ppid=12147 vsize=299532 CPUtime=0 cores=1,3,5,7
/proc/12159/task/12176/stat : 12176 (python) S 12147 12147 10986 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 617179286 306720768 5733 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 514 18446604471124425472 0 0 -1 5 0 0 0
[pid=12162] ppid=12147 vsize=309920 CPUtime=2.64 cores=1,3,5,7
/proc/12162/stat : 12162 (python) R 12147 12147 10986 0 -1 4202560 1022 0 0 0 204 60 0 0 18 0 1 0 617166357 317358080 5724 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747008831 0 0 16781312 514 0 0 0 17 3 0 0 0
/proc/12162/statm: 77480 5724 371 364 0 13389 0
[pid=12164] ppid=12162 vsize=0 CPUtime=331.5 cores=1,3,5,7
/proc/12164/stat : 12164 (march_hi) R 12162 12164 12164 0 -1 4202500 4748 0 0 0 33149 1 0 0 25 0 1 0 617166357 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 3 0 0 0
/proc/12164/statm: 0 0 0 0 0 0 0
[pid=12172] ppid=12147 vsize=320176 CPUtime=0.58 cores=1,3,5,7
/proc/12172/stat : 12172 (python) S 12147 12147 10986 0 -1 4202560 1028 0 0 0 32 26 0 0 15 0 1 0 617176269 327860224 5728 33554432000 4194304 5683644 140733496556240 18446744073709551615 216735198790 0 0 16781312 514 17009321736392376583 0 0 17 5 0 0 0
/proc/12172/statm: 80044 5728 371 364 0 15950 0
[pid=12174] ppid=12172 vsize=13680 CPUtime=269.27 cores=1,3,5,7
/proc/12174/stat : 12174 (march_hi) R 12172 12174 12174 0 -1 4202496 3506 0 0 0 26926 1 0 0 25 0 1 0 617176269 14008320 3224 33554432000 4194304 4955518 140736505721648 18446744073709551615 4224615 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/12174/statm: 3420 3224 106 186 0 3233 0
[pid=12177] ppid=12147 vsize=330172 CPUtime=0.5 cores=1,3,5,7
/proc/12177/stat : 12177 (python) S 12147 12147 10986 0 -1 4202560 1019 0 0 0 28 22 0 0 15 0 1 0 617179289 338096128 5667 33554432000 4194304 5683644 140733496556240 18446744073709551615 216735198790 0 0 16781312 514 4509629353432154376 0 0 17 5 0 0 0
/proc/12177/statm: 82543 5667 371 364 0 18446 0
[pid=12179] ppid=12177 vsize=6856 CPUtime=120.59 cores=1,3,5,7
/proc/12179/stat : 12179 (march_hi) R 12177 12179 12179 0 -1 4202496 1507 0 0 0 12059 0 0 0 25 0 1 0 617179289 7020544 1222 33554432000 4194304 4955518 140736064435840 18446744073709551615 4297061 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/12179/statm: 1714 1222 106 186 0 1527 0
[pid=12147/tid=12154] ppid=12145 vsize=340768 CPUtime=0 cores=1,3,5,7
/proc/12147/task/12154/stat : 12154 (python) S 12145 12147 10986 0 -1 4202560 27 935 0 0 0 0 0 0 19 0 7 0 617166348 348946432 7274 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 2 18446604457180261440 0 0 -1 5 0 0 0
[pid=12147/tid=12157] ppid=12145 vsize=340768 CPUtime=0 cores=1,3,5,7
/proc/12147/task/12157/stat : 12157 (python) S 12145 12147 10986 0 -1 4202560 28 935 0 0 0 0 0 0 18 0 7 0 617166351 348946432 7274 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 2 18446604457180261440 0 0 -1 7 0 0 0
[pid=12147/tid=12160] ppid=12145 vsize=340768 CPUtime=0 cores=1,3,5,7
/proc/12147/task/12160/stat : 12160 (python) S 12145 12147 10986 0 -1 4202560 23 935 0 0 0 0 0 0 23 0 7 0 617166354 348946432 7274 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 2 18446604457180261440 0 0 -1 5 0 0 0
[pid=12147/tid=12163] ppid=12145 vsize=340768 CPUtime=0 cores=1,3,5,7
/proc/12147/task/12163/stat : 12163 (python) S 12145 12147 10986 0 -1 4202560 28 935 0 0 0 0 0 0 25 0 7 0 617166357 348946432 7274 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 2 18446604457180261440 0 0 -1 5 0 0 0
[pid=12147/tid=12173] ppid=12145 vsize=340768 CPUtime=0 cores=1,3,5,7
/proc/12147/task/12173/stat : 12173 (python) S 12145 12147 10986 0 -1 4202560 33 935 0 0 0 0 0 0 20 0 7 0 617176269 348946432 7274 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 2 18446604457180261440 0 0 -1 5 0 0 0
[pid=12147/tid=12178] ppid=12145 vsize=340768 CPUtime=0 cores=1,3,5,7
/proc/12147/task/12178/stat : 12178 (python) S 12145 12147 10986 0 -1 4202560 26 935 0 0 0 0 0 0 20 0 7 0 617179289 348946432 7274 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 2 18446604457180261440 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 1146.48
Current children cumulated vsize (KiB) 4248180

[startup+371.302 s]
/proc/loadavg: 9.14 8.85 7.91 11/323 12187
/proc/meminfo: memFree=22311384/32951124 swapFree=67102140/67111528
[pid=12147] ppid=12145 vsize=340768 CPUtime=0.67 cores=1,3,5,7
/proc/12147/stat : 12147 (python) S 12145 12147 10986 0 -1 4202496 14588 935 1 0 58 9 0 0 18 0 7 0 617166289 348946432 7274 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747005387 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/12147/statm: 85192 7274 1886 364 0 21095 0
[pid=12153] ppid=12147 vsize=288696 CPUtime=0.17 cores=1,3,5,7
/proc/12153/stat : 12153 (python) S 12147 12147 10986 0 -1 4202560 1089 0 0 0 10 7 0 0 19 0 2 0 617166347 295624704 5611 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747005387 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/12153/statm: 72174 5611 410 364 0 8092 0
[pid=12155] ppid=12153 vsize=1017800 CPUtime=99.17 cores=1,3,5,7
/proc/12155/stat : 12155 (gnovelty+2) T 12153 12155 12155 0 -1 4202496 437 0 0 0 9917 0 0 0 25 0 1 0 617166348 1042227200 172 33554432000 134512640 135663704 4287644240 18446744073709551615 134524470 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/12155/statm: 254450 172 113 282 0 254166 0
[pid=12153/tid=12171] ppid=12147 vsize=288696 CPUtime=0 cores=1,3,5,7
/proc/12153/task/12171/stat : 12171 (python) S 12147 12147 10986 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 617176266 295624704 5611 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 514 18446604470732312768 0 0 -1 1 0 0 0
[pid=12156] ppid=12147 vsize=289276 CPUtime=0.77 cores=1,3,5,7
/proc/12156/stat : 12156 (python) S 12147 12147 10986 0 -1 4202560 1141 0 0 0 42 35 0 0 15 0 2 0 617166350 296218624 5729 33554432000 4194304 5683644 140733496556240 18446744073709551615 216735198790 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/12156/statm: 72319 5729 400 364 0 8234 0
[pid=12158] ppid=12156 vsize=13504 CPUtime=220.43 cores=1,3,5,7
/proc/12158/stat : 12158 (march_hi) R 12156 12158 12158 0 -1 4202496 2723 0 0 0 22042 1 0 0 25 0 1 0 617166351 13828096 2442 33554432000 4194304 4955518 140735268431072 18446744073709551615 4208624 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/12158/statm: 3376 2442 106 186 0 3189 0
[pid=12156/tid=12186] ppid=12147 vsize=289276 CPUtime=0 cores=1,3,5,7
/proc/12156/task/12186/stat : 12186 (python) S 12147 12147 10986 0 -1 4202560 46 0 0 0 0 0 0 0 16 0 2 0 617198268 296218624 5729 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 514 18446604469527134528 0 0 -1 5 0 0 0
[pid=12159] ppid=12147 vsize=299532 CPUtime=0.23 cores=1,3,5,7
/proc/12159/stat : 12159 (python) S 12147 12147 10986 0 -1 4202560 1111 0 0 0 12 11 0 0 19 0 2 0 617166353 306720768 5733 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747005387 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/12159/statm: 74883 5733 400 364 0 10795 0
[pid=12161] ppid=12159 vsize=1017800 CPUtime=100.06 cores=1,3,5,7
/proc/12161/stat : 12161 (gnovelty+2) T 12159 12161 12161 0 -1 4202496 436 0 0 0 10006 0 0 0 25 0 1 0 617166356 1042227200 172 33554432000 134512640 135663704 4294170688 18446744073709551615 134524945 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/12161/statm: 254450 172 113 282 0 254166 0
[pid=12159/tid=12176] ppid=12147 vsize=299532 CPUtime=0 cores=1,3,5,7
/proc/12159/task/12176/stat : 12176 (python) S 12147 12147 10986 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 617179286 306720768 5733 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 514 18446604471124425472 0 0 -1 5 0 0 0
[pid=12162] ppid=12147 vsize=309920 CPUtime=2.84 cores=1,3,5,7
/proc/12162/stat : 12162 (python) R 12147 12147 10986 0 -1 4202560 1022 0 0 0 222 62 0 0 18 0 1 0 617166357 317358080 5724 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747008831 0 0 16781312 514 0 0 0 17 3 0 0 0
/proc/12162/statm: 77480 5724 371 364 0 13389 0
[pid=12164] ppid=12162 vsize=0 CPUtime=331.5 cores=1,3,5,7
/proc/12164/stat : 12164 (march_hi) R 12162 12164 12164 0 -1 4202500 4748 0 0 0 33149 1 0 0 25 0 1 0 617166357 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 3 0 0 0
/proc/12164/statm: 0 0 0 0 0 0 0
[pid=12172] ppid=12147 vsize=320176 CPUtime=0.58 cores=1,3,5,7
/proc/12172/stat : 12172 (python) S 12147 12147 10986 0 -1 4202560 1028 0 0 0 32 26 0 0 15 0 1 0 617176269 327860224 5728 33554432000 4194304 5683644 140733496556240 18446744073709551615 216735198790 0 0 16781312 514 17009321736392376583 0 0 17 5 0 0 0
/proc/12172/statm: 80044 5728 371 364 0 15950 0
[pid=12174] ppid=12172 vsize=13680 CPUtime=269.47 cores=1,3,5,7
/proc/12174/stat : 12174 (march_hi) R 12172 12174 12174 0 -1 4202496 3509 0 0 0 26946 1 0 0 25 0 1 0 617176269 14008320 3227 33554432000 4194304 4955518 140736505721648 18446744073709551615 4286525 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/12174/statm: 3420 3227 106 186 0 3233 0
[pid=12177] ppid=12147 vsize=330172 CPUtime=0.5 cores=1,3,5,7
/proc/12177/stat : 12177 (python) S 12147 12147 10986 0 -1 4202560 1019 0 0 0 28 22 0 0 15 0 1 0 617179289 338096128 5667 33554432000 4194304 5683644 140733496556240 18446744073709551615 216735198790 0 0 16781312 514 4509629353432154376 0 0 17 5 0 0 0
/proc/12177/statm: 82543 5667 371 364 0 18446 0
[pid=12179] ppid=12177 vsize=6856 CPUtime=120.69 cores=1,3,5,7
/proc/12179/stat : 12179 (march_hi) R 12177 12179 12179 0 -1 4202496 1508 0 0 0 12069 0 0 0 25 0 1 0 617179289 7020544 1223 33554432000 4194304 4955518 140736064435840 18446744073709551615 4208667 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/12179/statm: 1714 1223 106 186 0 1527 0
[pid=12147/tid=12154] ppid=12145 vsize=340768 CPUtime=0 cores=1,3,5,7
/proc/12147/task/12154/stat : 12154 (python) S 12145 12147 10986 0 -1 4202560 27 935 0 0 0 0 0 0 19 0 7 0 617166348 348946432 7274 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 2 18446604457180261440 0 0 -1 5 0 0 0
[pid=12147/tid=12157] ppid=12145 vsize=340768 CPUtime=0 cores=1,3,5,7
/proc/12147/task/12157/stat : 12157 (python) S 12145 12147 10986 0 -1 4202560 28 935 0 0 0 0 0 0 18 0 7 0 617166351 348946432 7274 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 2 18446604457180261440 0 0 -1 7 0 0 0
[pid=12147/tid=12160] ppid=12145 vsize=340768 CPUtime=0 cores=1,3,5,7
/proc/12147/task/12160/stat : 12160 (python) S 12145 12147 10986 0 -1 4202560 23 935 0 0 0 0 0 0 23 0 7 0 617166354 348946432 7274 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 2 18446604457180261440 0 0 -1 5 0 0 0
[pid=12147/tid=12163] ppid=12145 vsize=340768 CPUtime=0 cores=1,3,5,7
/proc/12147/task/12163/stat : 12163 (python) S 12145 12147 10986 0 -1 4202560 28 935 0 0 0 0 0 0 25 0 7 0 617166357 348946432 7274 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 2 18446604457180261440 0 0 -1 5 0 0 0
[pid=12147/tid=12173] ppid=12145 vsize=340768 CPUtime=0 cores=1,3,5,7
/proc/12147/task/12173/stat : 12173 (python) S 12145 12147 10986 0 -1 4202560 33 935 0 0 0 0 0 0 20 0 7 0 617176269 348946432 7274 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 2 18446604457180261440 0 0 -1 5 0 0 0
[pid=12147/tid=12178] ppid=12145 vsize=340768 CPUtime=0 cores=1,3,5,7
/proc/12147/task/12178/stat : 12178 (python) S 12145 12147 10986 0 -1 4202560 26 935 0 0 0 0 0 0 20 0 7 0 617179289 348946432 7274 33554432000 4194304 5683644 140733496556240 18446744073709551615 216747002289 0 0 16781312 2 18446604457180261440 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 1147.08
Current children cumulated vsize (KiB) 4248180

[startup+371.402 s]
/proc/loadavg: 9.14 8.85 7.91 11/323 12187
/proc/meminfo: memFree=22311384/32951124 swapFree=67102140/67111528
[pid=12147] ppid=12145 vsize=0 CPUtime=1147.47 cores=1,3,5,7
/proc/12147/stat : 12147 (python) R 12145 12147 10986 0 -1 4202500 18611 21105 1 0 59 10 114505 173 17 0 1 0 617166289 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 3 0 0 0
/proc/12147/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 1147.47
Current children cumulated vsize (KiB) 0

Child status: 0
Real time (s): 371.402
CPU time (s): 1147.48
CPU user time (s): 1145.65
CPU system time (s): 1.83272
CPU usage (%): 308.959
Max. virtual memory (cumulated for all children) (KiB): 4268032

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1145.65
system time used= 1.83272
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 39716
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 4186
involuntary context switches= 13168

runsolver used 0.986849 second user time and 2.05669 second system time

The end

Launcher Data 

Begin job on node119 at 2011-05-01 19:02:36
IDJOB=3317395
IDBENCH=82860
IDSOLVER=1705
FILE ID=node119/3317395-1304269356
RUNJOBID= node119-1304267283-11005
PBS_JOBID= 13173960
Free space on /tmp= 72988 MiB

SOLVER NAME= borg-sat 11.03.02-random
BENCH NAME= SAT11/random/medium/unif-k5-r21.3-v100-c2130-S206772908-015.UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3317395-1304269356/watcher-3317395-1304269356 -o /tmp/evaluation-result-3317395-1304269356/solver-3317395-1304269356 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3317395-1304269356.cnf -seed 672243287 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 74f6f450e8e5ce3811b2d7ad1f57d348
RANDOM SEED=672243287

node119.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.63
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      22576992 kB
Buffers:       1968328 kB
Cached:        7115652 kB
SwapCached:       3688 kB
Active:        1315356 kB
Inactive:      8453768 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      22576992 kB
SwapTotal:    67111528 kB
SwapFree:     67102140 kB
Dirty:           33112 kB
Writeback:           0 kB
AnonPages:      684728 kB
Mapped:          27220 kB
Slab:           519860 kB
PageTables:      23712 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 39688364 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72956 MiB
End job on node119 at 2011-05-01 19:08:48