Trace number 3321978

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-randomUNSAT 319.47 40.6111

General information on the benchmark

Namerandom/medium/
unif-k3-r4.26-v400-c1704-S1081907273-083.UNKNOWN.cnf
MD5SUM4b706363c6542ff406d07bd657988d05
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark38.9001
Satisfiable
(Un)Satisfiability was proved
Number of variables400
Number of clauses1704
Sum of the clauses size5112
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 31704
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.32/0.47	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.32/0.47	c INFO: solving HOME/instance-3321978-1304370299.cnf
0.32/0.47	c INFO: computing task features
0.32/0.47	c INFO: computed variable-clause graph statistics
0.32/0.47	c INFO: computed clause balance statistics
0.41/0.50	c INFO: computed variable balance statistics
0.41/0.53	c INFO: computed small-clause counts
0.41/0.53	c INFO: computed Horn-clause counts
0.41/0.53	c INFO: computed variable graph statistics
0.41/0.53	c INFO: collected features for HOME/instance-3321978-1304370299.cnf in 0.06 s
0.41/0.57	c INFO: running gnovelty+2@0 for 97 with 39999 remaining (b = 0.56)
0.50/0.60	c INFO: running march_hi@0 for 388 with 39999 remaining (b = 0.39)
0.50/0.64	c INFO: running march_hi@0 for 485 with 39999 remaining (b = 0.39)
0.50/0.68	c INFO: running march_hi@0 for 485 with 39999 remaining (b = 0.35)
0.60/0.72	c INFO: running march_hi@0 for 97 with 39999 remaining (b = 0.12)
0.60/0.76	c INFO: running TNM@0 for 97 with 39999 remaining (b = 0.07)
0.69/0.80	c INFO: running march_hi@0 for 485 with 39999 remaining (b = 0.24)
0.69/0.84	c INFO: running march_hi@0 for 485 with 39999 remaining (b = 0.21)
318.57/40.59	s UNSATISFIABLE

Verifier Data 

No possible verification on an UNSAT instance

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3321978-1304370299/watcher-3321978-1304370299 -o /tmp/evaluation-result-3321978-1304370299/solver-3321978-1304370299 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3321978-1304370299.cnf -seed 1055185407 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.09 6.26 7.37 1/244 2124
/proc/meminfo: memFree=17060656/32950904 swapFree=67082284/67111528
[pid=2124] ppid=2122 vsize=15384 CPUtime=0 cores=0-7
/proc/2124/stat : 2124 (python) R 2122 2124 1221 0 -1 4202496 751 0 0 0 0 0 0 0 25 0 1 0 548629653 15753216 640 33554432000 4194304 5683644 140733294077040 18446744073709551615 237278573549 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/2124/statm: 3846 655 322 364 0 333 0

[startup+0.044121 s]
/proc/loadavg: 3.09 6.26 7.37 1/244 2124
/proc/meminfo: memFree=17060656/32950904 swapFree=67082284/67111528
[pid=2124] ppid=2122 vsize=27348 CPUtime=0.02 cores=0-7
/proc/2124/stat : 2124 (python) S 2122 2124 1221 0 -1 4202496 1352 0 0 0 2 0 0 0 25 0 1 0 548629653 28004352 1197 33554432000 4194304 5683644 140733294077040 18446744073709551615 237278877696 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 0
/proc/2124/statm: 6837 1197 440 364 0 724 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 27348

[startup+0.100111 s]
/proc/loadavg: 3.09 6.26 7.37 1/244 2124
/proc/meminfo: memFree=17060656/32950904 swapFree=67082284/67111528
[pid=2124] ppid=2122 vsize=104220 CPUtime=0.07 cores=0-7
/proc/2124/stat : 2124 (python) S 2122 2124 1221 0 -1 4202496 3048 0 0 0 5 2 0 0 25 0 1 0 548629653 106721280 2732 33554432000 4194304 5683644 140733294077040 18446744073709551615 237278877696 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 0
/proc/2124/statm: 26055 2732 847 364 0 1803 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 104220

[startup+0.300076 s]
/proc/loadavg: 3.09 6.26 7.37 1/244 2124
/proc/meminfo: memFree=17060656/32950904 swapFree=67082284/67111528
[pid=2124] ppid=2122 vsize=170752 CPUtime=0.24 cores=0-7
/proc/2124/stat : 2124 (python) R 2122 2124 1221 0 -1 4202496 6736 932 1 0 17 7 0 0 25 0 1 0 548629653 174850048 4805 33554432000 4194304 5683644 140733294077040 18446744073709551615 4619772 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/2124/statm: 42688 4805 1336 364 0 3453 0
Current children cumulated CPU time (s) 0.24
Current children cumulated vsize (KiB) 170752

[startup+0.701004 s]
/proc/loadavg: 3.09 6.26 7.37 1/244 2124
/proc/meminfo: memFree=17060656/32950904 swapFree=67082284/67111528
[pid=2124] ppid=2122 vsize=320216 CPUtime=0.6 cores=0-7
/proc/2124/stat : 2124 (python) R 2122 2124 1221 0 -1 4202496 12782 932 1 0 50 10 0 0 25 0 5 0 548629653 327901184 7250 33554432000 4194304 5683644 140733294077040 18446744073709551615 237282318295 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/2124/statm: 80054 7250 1880 364 0 15963 0
Current children cumulated CPU time (s) 0.6
Current children cumulated vsize (KiB) 320216

[startup+1.50086 s]
/proc/loadavg: 3.00 6.19 7.34 9/269 2153
/proc/meminfo: memFree=17000972/32950904 swapFree=67082284/67111528
[pid=2124] ppid=2122 vsize=361240 CPUtime=0.74 cores=0-7
/proc/2124/stat : 2124 (python) S 2122 2124 1221 0 -1 4202496 15853 932 1 0 63 11 0 0 25 0 9 0 548629653 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290706379 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/2124/statm: 90310 7283 1889 364 0 26207 0
[pid=2130] ppid=2124 vsize=278592 CPUtime=0 cores=0-7
/proc/2130/stat : 2130 (python) S 2124 2124 1221 0 -1 4202560 734 0 0 0 0 0 0 0 25 0 1 0 548629710 285278208 5571 33554432000 4194304 5683644 140733294077040 18446744073709551615 237278899727 0 0 16781312 514 4392090735789211647 0 0 17 1 0 0 0
/proc/2130/statm: 69648 5571 354 364 0 5566 0
[pid=2132] ppid=2130 vsize=1017800 CPUtime=0.92 cores=0-7
/proc/2132/stat : 2132 (gnovelty+2) R 2130 2132 2132 0 -1 4202496 434 0 0 0 92 0 0 0 25 0 1 0 548629710 1042227200 168 33554432000 134512640 135663704 4286865712 18446744073709551615 134524747 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/2132/statm: 254450 168 113 282 0 254166 0
[pid=2133] ppid=2124 vsize=289288 CPUtime=0 cores=0-7
/proc/2133/stat : 2133 (python) S 2124 2124 1221 0 -1 4202560 749 0 0 0 0 0 0 0 20 0 1 0 548629713 296230912 5680 33554432000 4194304 5683644 140733294077040 18446744073709551615 237278899782 0 0 16781312 514 4477259731110428932 0 0 17 5 0 0 0
/proc/2133/statm: 72322 5680 348 364 0 8237 0
[pid=2135] ppid=2133 vsize=1972 CPUtime=0.88 cores=0-7
/proc/2135/stat : 2135 (march_hi) R 2133 2135 2135 0 -1 4202496 588 0 0 0 88 0 0 0 25 0 1 0 548629714 2019328 324 33554432000 4194304 4955518 140734931363584 18446744073709551615 4227831 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/2135/statm: 493 324 104 186 0 306 0
[pid=2136] ppid=2124 vsize=299176 CPUtime=0 cores=0-7
/proc/2136/stat : 2136 (python) S 2124 2124 1221 0 -1 4202560 756 0 0 0 0 0 0 0 20 0 1 0 548629717 306356224 5591 33554432000 4194304 5683644 140733294077040 18446744073709551615 237278899782 0 0 16781312 514 5927146132240040194 0 0 17 2 0 0 0
/proc/2136/statm: 74794 5591 348 364 0 10706 0
[pid=2138] ppid=2136 vsize=1972 CPUtime=0.85 cores=0-7
/proc/2138/stat : 2138 (march_hi) R 2136 2138 2138 0 -1 4202496 587 0 0 0 85 0 0 0 25 0 1 0 548629718 2019328 323 33554432000 4194304 4955518 140737019768640 18446744073709551615 4227044 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/2138/statm: 493 323 104 186 0 306 0
[pid=2139] ppid=2124 vsize=309972 CPUtime=0 cores=0-7
/proc/2139/stat : 2139 (python) S 2124 2124 1221 0 -1 4202560 755 0 0 0 0 0 0 0 20 0 1 0 548629721 317411328 5714 33554432000 4194304 5683644 140733294077040 18446744073709551615 237278899782 0 0 16781312 514 3127600411922366722 0 0 17 7 0 0 0
/proc/2139/statm: 77493 5714 348 364 0 13402 0
[pid=2141] ppid=2139 vsize=1972 CPUtime=0.8 cores=0-7
/proc/2141/stat : 2141 (march_hi) R 2139 2141 2141 0 -1 4202496 588 0 0 0 80 0 0 0 25 0 1 0 548629721 2019328 322 33554432000 4194304 4955518 140733741124720 18446744073709551615 4209742 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/2141/statm: 493 322 104 186 0 306 0
[pid=2142] ppid=2124 vsize=320228 CPUtime=0 cores=0-7
/proc/2142/stat : 2142 (python) S 2124 2124 1221 0 -1 4202560 768 0 0 0 0 0 0 0 20 0 1 0 548629725 327913472 5718 33554432000 4194304 5683644 140733294077040 18446744073709551615 237278899782 0 0 16781312 514 1101262054582354180 0 0 17 6 0 0 0
/proc/2142/statm: 80057 5718 348 364 0 15963 0
[pid=2144] ppid=2142 vsize=1972 CPUtime=0.77 cores=0-7
/proc/2144/stat : 2144 (march_hi) R 2142 2144 2144 0 -1 4202496 589 0 0 0 77 0 0 0 25 0 1 0 548629725 2019328 323 33554432000 4194304 4955518 140735904755376 18446744073709551615 4263685 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/2144/statm: 493 323 104 186 0 306 0
[pid=2145] ppid=2124 vsize=330484 CPUtime=0 cores=0-7
/proc/2145/stat : 2145 (python) S 2124 2124 1221 0 -1 4202560 746 0 0 0 0 0 0 0 25 0 1 0 548629729 338415616 5726 33554432000 4194304 5683644 140733294077040 18446744073709551615 237278899782 0 0 16781312 514 4392094584079908863 0 0 17 4 0 0 0
/proc/2145/statm: 82621 5726 344 364 0 18524 0
[pid=2147] ppid=2145 vsize=454908 CPUtime=0.73 cores=0-7
/proc/2147/stat : 2147 (TNM) R 2145 2147 2147 0 -1 4202496 423 0 0 0 73 0 0 0 25 0 1 0 548629729 465825792 160 33554432000 134512640 135008748 4289670896 18446744073709551615 134527896 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/2147/statm: 113727 160 46 122 0 112627 0
[pid=2148] ppid=2124 vsize=340740 CPUtime=0 cores=0-7
/proc/2148/stat : 2148 (python) S 2124 2124 1221 0 -1 4202560 780 0 0 0 0 0 0 0 20 0 1 0 548629733 348917760 5734 33554432000 4194304 5683644 140733294077040 18446744073709551615 237278899782 0 0 16781312 514 4255290323695010049 0 0 17 4 0 0 0
/proc/2148/statm: 85185 5734 348 364 0 21085 0
[pid=2150] ppid=2148 vsize=1972 CPUtime=0.68 cores=0-7
/proc/2150/stat : 2150 (march_hi) R 2148 2150 2150 0 -1 4202496 588 0 0 0 68 0 0 0 25 0 1 0 548629733 2019328 323 33554432000 4194304 4955518 140735112728288 18446744073709551615 4209643 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/2150/statm: 493 323 104 186 0 306 0
[pid=2151] ppid=2124 vsize=350996 CPUtime=0 cores=0-7
/proc/2151/stat : 2151 (python) S 2124 2124 1221 0 -1 4202560 819 0 0 0 0 0 0 0 20 0 1 0 548629737 359419904 5744 33554432000 4194304 5683644 140733294077040 18446744073709551615 237278899782 0 0 16781312 514 8233706023035044097 0 0 17 4 0 0 0
/proc/2151/statm: 87749 5744 354 364 0 23646 0
[pid=2153] ppid=2151 vsize=1972 CPUtime=0.61 cores=0-7
/proc/2153/stat : 2153 (march_hi) R 2151 2153 2153 0 -1 4202496 591 0 0 0 61 0 0 0 25 0 1 0 548629737 2019328 322 33554432000 4194304 4955518 140734855145376 18446744073709551615 4238783 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/2153/statm: 493 322 104 186 0 306 0
[pid=2124/tid=2131] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2131/stat : 2131 (python) S 2122 2124 1221 0 -1 4202560 27 932 0 0 0 0 0 0 25 0 9 0 548629710 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 5 0 0 0
[pid=2124/tid=2134] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2134/stat : 2134 (python) S 2122 2124 1221 0 -1 4202560 24 932 0 0 0 0 0 0 25 0 9 0 548629714 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 0 0 0 0
[pid=2124/tid=2137] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2137/stat : 2137 (python) S 2122 2124 1221 0 -1 4202560 20 932 0 0 0 0 0 0 25 0 9 0 548629717 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 2 0 0 0
[pid=2124/tid=2140] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2140/stat : 2140 (python) S 2122 2124 1221 0 -1 4202560 22 932 0 0 0 0 0 0 25 0 9 0 548629721 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 0 0 0 0
[pid=2124/tid=2143] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2143/stat : 2143 (python) S 2122 2124 1221 0 -1 4202560 23 932 0 0 0 0 0 0 25 0 9 0 548629725 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 4 0 0 0
[pid=2124/tid=2146] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2146/stat : 2146 (python) S 2122 2124 1221 0 -1 4202560 28 932 0 0 0 0 0 0 25 0 9 0 548629729 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 4 0 0 0
[pid=2124/tid=2149] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2149/stat : 2149 (python) S 2122 2124 1221 0 -1 4202560 26 932 0 0 0 0 0 0 25 0 9 0 548629733 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 0 0 0 0
[pid=2124/tid=2152] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2152/stat : 2152 (python) S 2122 2124 1221 0 -1 4202560 81 932 0 0 0 0 0 0 25 0 9 0 548629737 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 6.98
Current children cumulated vsize (KiB) 4365256

[startup+3.10158 s]
/proc/loadavg: 3.00 6.19 7.34 9/269 2153
/proc/meminfo: memFree=16999980/32950904 swapFree=67082284/67111528
[pid=2124] ppid=2122 vsize=361240 CPUtime=0.74 cores=0-7
/proc/2124/stat : 2124 (python) S 2122 2124 1221 0 -1 4202496 15853 932 1 0 63 11 0 0 25 0 9 0 548629653 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290706379 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/2124/statm: 90310 7283 1889 364 0 26207 0
[pid=2130] ppid=2124 vsize=278592 CPUtime=0 cores=0-7
/proc/2130/stat : 2130 (python) S 2124 2124 1221 0 -1 4202560 734 0 0 0 0 0 0 0 18 0 1 0 548629710 285278208 5571 33554432000 4194304 5683644 140733294077040 18446744073709551615 237278899727 0 0 16781312 514 4392090735789179138 0 0 17 1 0 0 0
/proc/2130/statm: 69648 5571 354 364 0 5566 0
[pid=2132] ppid=2130 vsize=1017800 CPUtime=2.52 cores=0-7
/proc/2132/stat : 2132 (gnovelty+2) R 2130 2132 2132 0 -1 4202496 434 0 0 0 252 0 0 0 25 0 1 0 548629710 1042227200 168 33554432000 134512640 135663704 4286865712 18446744073709551615 134525187 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/2132/statm: 254450 168 113 282 0 254166 0
[pid=2133] ppid=2124 vsize=289288 CPUtime=0 cores=0-7
/proc/2133/stat : 2133 (python) S 2124 2124 1221 0 -1 4202560 752 0 0 0 0 0 0 0 15 0 1 0 548629713 296230912 5680 33554432000 4194304 5683644 140733294077040 18446744073709551615 237278899782 0 0 16781312 514 4477259731110428932 0 0 17 5 0 0 0
/proc/2133/statm: 72322 5680 348 364 0 8237 0
[pid=2135] ppid=2133 vsize=2240 CPUtime=2.48 cores=0-7
/proc/2135/stat : 2135 (march_hi) R 2133 2135 2135 0 -1 4202496 647 0 0 0 248 0 0 0 25 0 1 0 548629714 2293760 383 33554432000 4194304 4955518 140734931363584 18446744073709551615 4209688 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/2135/statm: 560 383 104 186 0 373 0
[pid=2136] ppid=2124 vsize=299176 CPUtime=0 cores=0-7
/proc/2136/stat : 2136 (python) S 2124 2124 1221 0 -1 4202560 758 0 0 0 0 0 0 0 15 0 1 0 548629717 306356224 5591 33554432000 4194304 5683644 140733294077040 18446744073709551615 237278899782 0 0 16781312 514 5927146132240040194 0 0 17 2 0 0 0
/proc/2136/statm: 74794 5591 348 364 0 10706 0
[pid=2138] ppid=2136 vsize=2240 CPUtime=2.44 cores=0-7
/proc/2138/stat : 2138 (march_hi) R 2136 2138 2138 0 -1 4202496 645 0 0 0 244 0 0 0 25 0 1 0 548629718 2293760 381 33554432000 4194304 4955518 140737019768640 18446744073709551615 4224642 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/2138/statm: 560 381 104 186 0 373 0
[pid=2139] ppid=2124 vsize=309972 CPUtime=0 cores=0-7
/proc/2139/stat : 2139 (python) S 2124 2124 1221 0 -1 4202560 757 0 0 0 0 0 0 0 15 0 1 0 548629721 317411328 5714 33554432000 4194304 5683644 140733294077040 18446744073709551615 237278899782 0 0 16781312 514 3127600411922366722 0 0 17 7 0 0 0
/proc/2139/statm: 77493 5714 348 364 0 13402 0
[pid=2141] ppid=2139 vsize=2240 CPUtime=2.39 cores=0-7
/proc/2141/stat : 2141 (march_hi) R 2139 2141 2141 0 -1 4202496 647 0 0 0 239 0 0 0 25 0 1 0 548629721 2293760 381 33554432000 4194304 4955518 140733741124720 18446744073709551615 4224691 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/2141/statm: 560 381 104 186 0 373 0
[pid=2142] ppid=2124 vsize=320228 CPUtime=0 cores=0-7
/proc/2142/stat : 2142 (python) S 2124 2124 1221 0 -1 4202560 775 0 0 0 0 0 0 0 15 0 1 0 548629725 327913472 5718 33554432000 4194304 5683644 140733294077040 18446744073709551615 237278899782 0 0 16781312 514 1101262054582354180 0 0 17 6 0 0 0
/proc/2142/statm: 80057 5718 348 364 0 15963 0
[pid=2144] ppid=2142 vsize=2240 CPUtime=2.37 cores=0-7
/proc/2144/stat : 2144 (march_hi) R 2142 2144 2144 0 -1 4202496 648 0 0 0 237 0 0 0 25 0 1 0 548629725 2293760 382 33554432000 4194304 4955518 140735904755376 18446744073709551615 4227667 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/2144/statm: 560 382 104 186 0 373 0
[pid=2145] ppid=2124 vsize=330484 CPUtime=0 cores=0-7
/proc/2145/stat : 2145 (python) S 2124 2124 1221 0 -1 4202560 763 0 0 0 0 0 0 0 18 0 1 0 548629729 338415616 5730 33554432000 4194304 5683644 140733294077040 18446744073709551615 237278899782 0 0 16781312 514 4392094584079876354 0 0 17 4 0 0 0
/proc/2145/statm: 82621 5730 348 364 0 18524 0
[pid=2147] ppid=2145 vsize=454908 CPUtime=2.32 cores=0-7
/proc/2147/stat : 2147 (TNM) R 2145 2147 2147 0 -1 4202496 423 0 0 0 232 0 0 0 25 0 1 0 548629729 465825792 160 33554432000 134512640 135008748 4289670896 18446744073709551615 134527280 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/2147/statm: 113727 160 46 122 0 112627 0
[pid=2148] ppid=2124 vsize=340740 CPUtime=0 cores=0-7
/proc/2148/stat : 2148 (python) S 2124 2124 1221 0 -1 4202560 785 0 0 0 0 0 0 0 15 0 1 0 548629733 348917760 5734 33554432000 4194304 5683644 140733294077040 18446744073709551615 237278899782 0 0 16781312 514 4255290323695010049 0 0 17 4 0 0 0
/proc/2148/statm: 85185 5734 348 364 0 21085 0
[pid=2150] ppid=2148 vsize=2240 CPUtime=2.28 cores=0-7
/proc/2150/stat : 2150 (march_hi) R 2148 2150 2150 0 -1 4202496 646 0 0 0 228 0 0 0 25 0 1 0 548629733 2293760 381 33554432000 4194304 4955518 140735112728288 18446744073709551615 4224693 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/2150/statm: 560 381 104 186 0 373 0
[pid=2151] ppid=2124 vsize=350996 CPUtime=0 cores=0-7
/proc/2151/stat : 2151 (python) S 2124 2124 1221 0 -1 4202560 821 0 0 0 0 0 0 0 15 0 1 0 548629737 359419904 5744 33554432000 4194304 5683644 140733294077040 18446744073709551615 237278899782 0 0 16781312 514 8233706023035044097 0 0 17 4 0 0 0
/proc/2151/statm: 87749 5744 354 364 0 23646 0
[pid=2153] ppid=2151 vsize=2240 CPUtime=2.21 cores=0-7
/proc/2153/stat : 2153 (march_hi) R 2151 2153 2153 0 -1 4202496 650 0 0 0 221 0 0 0 25 0 1 0 548629737 2293760 381 33554432000 4194304 4955518 140734855145376 18446744073709551615 4209727 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/2153/statm: 560 381 104 186 0 373 0
[pid=2124/tid=2131] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2131/stat : 2131 (python) S 2122 2124 1221 0 -1 4202560 27 932 0 0 0 0 0 0 25 0 9 0 548629710 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 5 0 0 0
[pid=2124/tid=2134] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2134/stat : 2134 (python) S 2122 2124 1221 0 -1 4202560 24 932 0 0 0 0 0 0 25 0 9 0 548629714 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 0 0 0 0
[pid=2124/tid=2137] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2137/stat : 2137 (python) S 2122 2124 1221 0 -1 4202560 20 932 0 0 0 0 0 0 25 0 9 0 548629717 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 2 0 0 0
[pid=2124/tid=2140] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2140/stat : 2140 (python) S 2122 2124 1221 0 -1 4202560 22 932 0 0 0 0 0 0 25 0 9 0 548629721 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 0 0 0 0
[pid=2124/tid=2143] ppid=2122 vsize=361240 CPUtime=0 cores=0-7

################
# More data... #
################

/proc/2124/task/2134/stat : 2134 (python) S 2122 2124 1221 0 -1 4202560 24 932 0 0 0 0 0 0 25 0 9 0 548629714 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 0 0 0 0
[pid=2124/tid=2137] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2137/stat : 2137 (python) S 2122 2124 1221 0 -1 4202560 20 932 0 0 0 0 0 0 25 0 9 0 548629717 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 2 0 0 0
[pid=2124/tid=2140] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2140/stat : 2140 (python) S 2122 2124 1221 0 -1 4202560 22 932 0 0 0 0 0 0 25 0 9 0 548629721 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 0 0 0 0
[pid=2124/tid=2143] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2143/stat : 2143 (python) S 2122 2124 1221 0 -1 4202560 23 932 0 0 0 0 0 0 25 0 9 0 548629725 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 4 0 0 0
[pid=2124/tid=2146] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2146/stat : 2146 (python) S 2122 2124 1221 0 -1 4202560 28 932 0 0 0 0 0 0 25 0 9 0 548629729 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 4 0 0 0
[pid=2124/tid=2149] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2149/stat : 2149 (python) S 2122 2124 1221 0 -1 4202560 26 932 0 0 0 0 0 0 25 0 9 0 548629733 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 0 0 0 0
[pid=2124/tid=2152] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2152/stat : 2152 (python) S 2122 2124 1221 0 -1 4202560 81 932 0 0 0 0 0 0 25 0 9 0 548629737 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 313.79
Current children cumulated vsize (KiB) 4385004

[startup+40.301 s]
/proc/loadavg: 5.21 6.39 7.37 9/269 2154
/proc/meminfo: memFree=16980184/32950904 swapFree=67082284/67111528
[pid=2124] ppid=2122 vsize=361240 CPUtime=0.74 cores=0-7
/proc/2124/stat : 2124 (python) S 2122 2124 1221 0 -1 4202496 15853 932 1 0 63 11 0 0 25 0 9 0 548629653 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290706379 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/2124/statm: 90310 7283 1889 364 0 26207 0
[pid=2130] ppid=2124 vsize=278592 CPUtime=0.05 cores=0-7
/proc/2130/stat : 2130 (python) S 2124 2124 1221 0 -1 4202560 906 0 0 0 3 2 0 0 18 0 1 0 548629710 285278208 5594 33554432000 4194304 5683644 140733294077040 18446744073709551615 237278899727 0 0 16781312 514 13423647192447418626 0 0 17 1 0 0 0
/proc/2130/statm: 69648 5594 377 364 0 5566 0
[pid=2132] ppid=2130 vsize=1017800 CPUtime=39.72 cores=0-7
/proc/2132/stat : 2132 (gnovelty+2) R 2130 2132 2132 0 -1 4202496 434 0 0 0 3972 0 0 0 25 0 1 0 548629710 1042227200 168 33554432000 134512640 135663704 4286865712 18446744073709551615 134525560 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/2132/statm: 254450 168 113 282 0 254166 0
[pid=2133] ppid=2124 vsize=289288 CPUtime=0.55 cores=0-7
/proc/2133/stat : 2133 (python) R 2124 2124 1221 0 -1 4202560 941 0 0 0 45 10 0 0 16 0 1 0 548629713 296230912 5703 33554432000 4194304 5683644 140733294077040 18446744073709551615 237278899782 0 0 16781312 514 0 0 0 17 5 0 0 0
/proc/2133/statm: 72322 5703 371 364 0 8237 0
[pid=2135] ppid=2133 vsize=0 CPUtime=39.13 cores=0-7
/proc/2135/stat : 2135 (march_hi) R 2133 2135 2135 0 -1 4202500 1490 0 0 0 3913 0 0 0 25 0 1 0 548629714 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 5 0 0 0
/proc/2135/statm: 0 0 0 0 0 0 0
[pid=2136] ppid=2124 vsize=299176 CPUtime=0.44 cores=0-7
/proc/2136/stat : 2136 (python) R 2124 2124 1221 0 -1 4202560 936 0 0 0 35 9 0 0 16 0 1 0 548629717 306356224 5614 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290709823 0 0 16781312 514 0 0 0 17 2 0 0 0
/proc/2136/statm: 74794 5614 371 364 0 10706 0
[pid=2138] ppid=2136 vsize=0 CPUtime=39.19 cores=0-7
/proc/2138/stat : 2138 (march_hi) R 2136 2138 2138 0 -1 4202500 1489 0 0 0 3919 0 0 0 25 0 1 0 548629718 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 2 0 0 0
/proc/2138/statm: 0 0 0 0 0 0 0
[pid=2139] ppid=2124 vsize=309972 CPUtime=0.08 cores=0-7
/proc/2139/stat : 2139 (python) S 2124 2124 1221 0 -1 4202560 949 0 0 0 4 4 0 0 15 0 1 0 548629721 317411328 5737 33554432000 4194304 5683644 140733294077040 18446744073709551615 237278899782 0 0 16781312 514 3127600411922366722 0 0 17 7 0 0 0
/proc/2139/statm: 77493 5737 371 364 0 13402 0
[pid=2141] ppid=2139 vsize=6316 CPUtime=39.22 cores=0-7
/proc/2141/stat : 2141 (march_hi) R 2139 2141 2141 0 -1 4202496 1483 0 0 0 3922 0 0 0 25 0 1 0 548629721 6467584 1217 33554432000 4194304 4955518 140733741124720 18446744073709551615 4290819 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/2141/statm: 1579 1217 104 186 0 1392 0
[pid=2142] ppid=2124 vsize=320228 CPUtime=0.39 cores=0-7
/proc/2142/stat : 2142 (python) R 2124 2124 1221 0 -1 4202560 964 0 0 0 31 8 0 0 16 0 1 0 548629725 327913472 5741 33554432000 4194304 5683644 140733294077040 18446744073709551615 237278899782 0 0 16781312 514 0 0 0 17 6 0 0 0
/proc/2142/statm: 80057 5741 371 364 0 15963 0
[pid=2144] ppid=2142 vsize=0 CPUtime=39.16 cores=0-7
/proc/2144/stat : 2144 (march_hi) R 2142 2144 2144 0 -1 4202500 1492 0 0 0 3916 0 0 0 25 0 1 0 548629725 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 6 0 0 0
/proc/2144/statm: 0 0 0 0 0 0 0
[pid=2145] ppid=2124 vsize=330484 CPUtime=0.05 cores=0-7
/proc/2145/stat : 2145 (python) S 2124 2124 1221 0 -1 4202560 953 0 0 0 3 2 0 0 18 0 1 0 548629729 338415616 5753 33554432000 4194304 5683644 140733294077040 18446744073709551615 237278899782 0 0 16781312 514 13423647192447418626 0 0 17 4 0 0 0
/proc/2145/statm: 82621 5753 371 364 0 18524 0
[pid=2147] ppid=2145 vsize=454908 CPUtime=39.46 cores=0-7
/proc/2147/stat : 2147 (TNM) R 2145 2147 2147 0 -1 4202496 423 0 0 0 3946 0 0 0 25 0 1 0 548629729 465825792 160 33554432000 134512640 135008748 4289670896 18446744073709551615 134529448 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/2147/statm: 113727 160 46 122 0 112627 0
[pid=2148] ppid=2124 vsize=340740 CPUtime=0.08 cores=0-7
/proc/2148/stat : 2148 (python) S 2124 2124 1221 0 -1 4202560 973 0 0 0 5 3 0 0 15 0 1 0 548629733 348917760 5757 33554432000 4194304 5683644 140733294077040 18446744073709551615 237278899782 0 0 16781312 514 5602227952172368129 0 0 17 0 0 0 0
/proc/2148/statm: 85185 5757 371 364 0 21085 0
[pid=2150] ppid=2148 vsize=6316 CPUtime=39.38 cores=0-7
/proc/2150/stat : 2150 (march_hi) R 2148 2150 2150 0 -1 4202496 1489 0 0 0 3938 0 0 0 25 0 1 0 548629733 6467584 1224 33554432000 4194304 4955518 140735112728288 18446744073709551615 4284781 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/2150/statm: 1579 1224 104 186 0 1392 0
[pid=2151] ppid=2124 vsize=350996 CPUtime=0.21 cores=0-7
/proc/2151/stat : 2151 (python) R 2124 2124 1221 0 -1 4202560 1016 0 0 0 16 5 0 0 15 0 1 0 548629737 359419904 5767 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290709823 0 0 16781312 514 0 0 0 17 4 0 0 0
/proc/2151/statm: 87749 5767 377 364 0 23646 0
[pid=2153] ppid=2151 vsize=0 CPUtime=39.13 cores=0-7
/proc/2153/stat : 2153 (march_hi) R 2151 2153 2153 0 -1 4202500 1495 0 0 0 3913 0 0 0 25 0 1 0 548629737 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 4 0 0 0
/proc/2153/statm: 0 0 0 0 0 0 0
[pid=2124/tid=2131] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2131/stat : 2131 (python) S 2122 2124 1221 0 -1 4202560 27 932 0 0 0 0 0 0 25 0 9 0 548629710 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 5 0 0 0
[pid=2124/tid=2134] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2134/stat : 2134 (python) S 2122 2124 1221 0 -1 4202560 24 932 0 0 0 0 0 0 25 0 9 0 548629714 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 0 0 0 0
[pid=2124/tid=2137] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2137/stat : 2137 (python) S 2122 2124 1221 0 -1 4202560 20 932 0 0 0 0 0 0 25 0 9 0 548629717 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 2 0 0 0
[pid=2124/tid=2140] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2140/stat : 2140 (python) S 2122 2124 1221 0 -1 4202560 22 932 0 0 0 0 0 0 25 0 9 0 548629721 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 0 0 0 0
[pid=2124/tid=2143] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2143/stat : 2143 (python) S 2122 2124 1221 0 -1 4202560 23 932 0 0 0 0 0 0 25 0 9 0 548629725 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 4 0 0 0
[pid=2124/tid=2146] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2146/stat : 2146 (python) S 2122 2124 1221 0 -1 4202560 28 932 0 0 0 0 0 0 25 0 9 0 548629729 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 4 0 0 0
[pid=2124/tid=2149] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2149/stat : 2149 (python) S 2122 2124 1221 0 -1 4202560 26 932 0 0 0 0 0 0 25 0 9 0 548629733 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 0 0 0 0
[pid=2124/tid=2152] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2152/stat : 2152 (python) S 2122 2124 1221 0 -1 4202560 81 932 0 0 0 0 0 0 25 0 9 0 548629737 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 316.98
Current children cumulated vsize (KiB) 4366056

[startup+40.5009 s]
/proc/loadavg: 5.21 6.39 7.37 9/269 2154
/proc/meminfo: memFree=16980184/32950904 swapFree=67082284/67111528
[pid=2124] ppid=2122 vsize=361240 CPUtime=0.74 cores=0-7
/proc/2124/stat : 2124 (python) S 2122 2124 1221 0 -1 4202496 15853 932 1 0 63 11 0 0 25 0 9 0 548629653 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290706379 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/2124/statm: 90310 7283 1889 364 0 26207 0
[pid=2130] ppid=2124 vsize=278592 CPUtime=0.05 cores=0-7
/proc/2130/stat : 2130 (python) S 2124 2124 1221 0 -1 4202560 906 0 0 0 3 2 0 0 18 0 1 0 548629710 285278208 5594 33554432000 4194304 5683644 140733294077040 18446744073709551615 237278899727 0 0 16781312 514 13423647192447418626 0 0 17 1 0 0 0
/proc/2130/statm: 69648 5594 377 364 0 5566 0
[pid=2132] ppid=2130 vsize=1017800 CPUtime=39.92 cores=0-7
/proc/2132/stat : 2132 (gnovelty+2) R 2130 2132 2132 0 -1 4202496 434 0 0 0 3992 0 0 0 25 0 1 0 548629710 1042227200 168 33554432000 134512640 135663704 4286865712 18446744073709551615 134524732 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/2132/statm: 254450 168 113 282 0 254166 0
[pid=2133] ppid=2124 vsize=289288 CPUtime=0.74 cores=0-7
/proc/2133/stat : 2133 (python) R 2124 2124 1221 0 -1 4202560 944 0 0 0 62 12 0 0 16 0 1 0 548629713 296230912 5703 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290709823 0 0 16781312 514 0 0 0 17 5 0 0 0
/proc/2133/statm: 72322 5703 371 364 0 8237 0
[pid=2135] ppid=2133 vsize=0 CPUtime=39.13 cores=0-7
/proc/2135/stat : 2135 (march_hi) R 2133 2135 2135 0 -1 4202500 1490 0 0 0 3913 0 0 0 25 0 1 0 548629714 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 5 0 0 0
/proc/2135/statm: 0 0 0 0 0 0 0
[pid=2136] ppid=2124 vsize=299176 CPUtime=0.64 cores=0-7
/proc/2136/stat : 2136 (python) R 2124 2124 1221 0 -1 4202560 936 0 0 0 53 11 0 0 16 0 1 0 548629717 306356224 5614 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290709823 0 0 16781312 514 0 0 0 17 2 0 0 0
/proc/2136/statm: 74794 5614 371 364 0 10706 0
[pid=2138] ppid=2136 vsize=0 CPUtime=39.19 cores=0-7
/proc/2138/stat : 2138 (march_hi) R 2136 2138 2138 0 -1 4202500 1489 0 0 0 3919 0 0 0 25 0 1 0 548629718 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 2 0 0 0
/proc/2138/statm: 0 0 0 0 0 0 0
[pid=2139] ppid=2124 vsize=309972 CPUtime=0.09 cores=0-7
/proc/2139/stat : 2139 (python) S 2124 2124 1221 0 -1 4202560 951 0 0 0 5 4 0 0 15 0 1 0 548629721 317411328 5737 33554432000 4194304 5683644 140733294077040 18446744073709551615 237278899782 0 0 16781312 514 3127600411922366722 0 0 17 7 0 0 0
/proc/2139/statm: 77493 5737 371 364 0 13402 0
[pid=2141] ppid=2139 vsize=6316 CPUtime=39.41 cores=0-7
/proc/2141/stat : 2141 (march_hi) R 2139 2141 2141 0 -1 4202496 1488 0 0 0 3941 0 0 0 25 0 1 0 548629721 6467584 1222 33554432000 4194304 4955518 140733741124720 18446744073709551615 4227502 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/2141/statm: 1579 1222 104 186 0 1392 0
[pid=2142] ppid=2124 vsize=320228 CPUtime=0.59 cores=0-7
/proc/2142/stat : 2142 (python) R 2124 2124 1221 0 -1 4202560 964 0 0 0 48 11 0 0 16 0 1 0 548629725 327913472 5741 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290709823 0 0 16781312 514 0 0 0 17 6 0 0 0
/proc/2142/statm: 80057 5741 371 364 0 15963 0
[pid=2144] ppid=2142 vsize=0 CPUtime=39.16 cores=0-7
/proc/2144/stat : 2144 (march_hi) R 2142 2144 2144 0 -1 4202500 1492 0 0 0 3916 0 0 0 25 0 1 0 548629725 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 6 0 0 0
/proc/2144/statm: 0 0 0 0 0 0 0
[pid=2145] ppid=2124 vsize=330484 CPUtime=0.05 cores=0-7
/proc/2145/stat : 2145 (python) S 2124 2124 1221 0 -1 4202560 953 0 0 0 3 2 0 0 18 0 1 0 548629729 338415616 5753 33554432000 4194304 5683644 140733294077040 18446744073709551615 237278899782 0 0 16781312 514 13423647192447418626 0 0 17 4 0 0 0
/proc/2145/statm: 82621 5753 371 364 0 18524 0
[pid=2147] ppid=2145 vsize=454908 CPUtime=39.66 cores=0-7
/proc/2147/stat : 2147 (TNM) R 2145 2147 2147 0 -1 4202496 423 0 0 0 3966 0 0 0 25 0 1 0 548629729 465825792 160 33554432000 134512640 135008748 4289670896 18446744073709551615 134528574 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/2147/statm: 113727 160 46 122 0 112627 0
[pid=2148] ppid=2124 vsize=340740 CPUtime=0.24 cores=0-7
/proc/2148/stat : 2148 (python) R 2124 2124 1221 0 -1 4202560 974 0 0 0 19 5 0 0 16 0 1 0 548629733 348917760 5757 33554432000 4194304 5683644 140733294077040 18446744073709551615 237278899782 0 0 16781312 514 0 0 0 17 0 0 0 0
/proc/2148/statm: 85185 5757 371 364 0 21085 0
[pid=2150] ppid=2148 vsize=0 CPUtime=39.43 cores=0-7
/proc/2150/stat : 2150 (march_hi) R 2148 2150 2150 0 -1 4202500 1491 0 0 0 3943 0 0 0 25 0 1 0 548629733 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 3 0 0 0
/proc/2150/statm: 0 0 0 0 0 0 0
[pid=2151] ppid=2124 vsize=350996 CPUtime=0.4 cores=0-7
/proc/2151/stat : 2151 (python) R 2124 2124 1221 0 -1 4202560 1016 0 0 0 32 8 0 0 16 0 1 0 548629737 359419904 5767 33554432000 4194304 5683644 140733294077040 18446744073709551615 237278899782 0 0 16781312 514 0 0 0 17 4 0 0 0
/proc/2151/statm: 87749 5767 377 364 0 23646 0
[pid=2153] ppid=2151 vsize=0 CPUtime=39.13 cores=0-7
/proc/2153/stat : 2153 (march_hi) R 2151 2153 2153 0 -1 4202500 1495 0 0 0 3913 0 0 0 25 0 1 0 548629737 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 4 0 0 0
/proc/2153/statm: 0 0 0 0 0 0 0
[pid=2124/tid=2131] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2131/stat : 2131 (python) S 2122 2124 1221 0 -1 4202560 27 932 0 0 0 0 0 0 25 0 9 0 548629710 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 5 0 0 0
[pid=2124/tid=2134] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2134/stat : 2134 (python) S 2122 2124 1221 0 -1 4202560 24 932 0 0 0 0 0 0 25 0 9 0 548629714 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 0 0 0 0
[pid=2124/tid=2137] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2137/stat : 2137 (python) S 2122 2124 1221 0 -1 4202560 20 932 0 0 0 0 0 0 25 0 9 0 548629717 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 2 0 0 0
[pid=2124/tid=2140] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2140/stat : 2140 (python) S 2122 2124 1221 0 -1 4202560 22 932 0 0 0 0 0 0 25 0 9 0 548629721 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 0 0 0 0
[pid=2124/tid=2143] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2143/stat : 2143 (python) S 2122 2124 1221 0 -1 4202560 23 932 0 0 0 0 0 0 25 0 9 0 548629725 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 4 0 0 0
[pid=2124/tid=2146] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2146/stat : 2146 (python) S 2122 2124 1221 0 -1 4202560 28 932 0 0 0 0 0 0 25 0 9 0 548629729 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 4 0 0 0
[pid=2124/tid=2149] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2149/stat : 2149 (python) S 2122 2124 1221 0 -1 4202560 26 932 0 0 0 0 0 0 25 0 9 0 548629733 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 0 0 0 0
[pid=2124/tid=2152] ppid=2122 vsize=361240 CPUtime=0 cores=0-7
/proc/2124/task/2152/stat : 2152 (python) S 2122 2124 1221 0 -1 4202560 81 932 0 0 0 0 0 0 25 0 9 0 548629737 369909760 7283 33554432000 4194304 5683644 140733294077040 18446744073709551615 237290703281 0 0 16781312 2 18446604453127945024 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 318.57
Current children cumulated vsize (KiB) 4359740

[startup+40.6009 s]
/proc/loadavg: 5.83 6.50 7.40 2/245 2159
/proc/meminfo: memFree=17036424/32950904 swapFree=67082284/67111528
[pid=2124] ppid=2122 vsize=309912 CPUtime=319.43 cores=0-7
/proc/2124/stat : 2124 (python) R 2122 2124 1221 0 -1 4202496 19854 19722 1 0 64 11 31798 70 17 0 1 0 548629653 317349888 7248 33554432000 4194304 5683644 140733294077040 18446744073709551615 4501827 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/2124/statm: 77478 7248 1874 364 0 13402 0
Current children cumulated CPU time (s) 319.43
Current children cumulated vsize (KiB) 309912

Child status: 0
Real time (s): 40.6111
CPU time (s): 319.47
CPU user time (s): 318.642
CPU system time (s): 0.828873
CPU usage (%): 786.657
Max. virtual memory (cumulated for all children) (KiB): 4391320

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 318.642
system time used= 0.828873
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 39928
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 5211
involuntary context switches= 5147

runsolver used 0.103984 second user time and 0.207968 second system time

The end

Launcher Data 

Begin job on node121 at 2011-05-02 23:04:59
IDJOB=3321978
IDBENCH=82756
IDSOLVER=1712
FILE ID=node121/3321978-1304370299
RUNJOBID= node121-1304366467-1235
PBS_JOBID= 13181750
Free space on /tmp= 73496 MiB

SOLVER NAME= borg-sat 11.03.02-random
BENCH NAME= SAT11/random/medium/unif-k3-r4.26-v400-c1704-S1081907273-083.UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3321978-1304370299/watcher-3321978-1304370299 -o /tmp/evaluation-result-3321978-1304370299/solver-3321978-1304370299 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3321978-1304370299.cnf -seed 1055185407 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 4b706363c6542ff406d07bd657988d05
RANDOM SEED=1055185407

node121.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.63
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950904 kB
MemFree:      17061060 kB
Buffers:       2198988 kB
Cached:        9687572 kB
SwapCached:       2776 kB
Active:       12335680 kB
Inactive:      3016676 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950904 kB
LowFree:      17061060 kB
SwapTotal:    67111528 kB
SwapFree:     67082284 kB
Dirty:           32696 kB
Writeback:           0 kB
AnonPages:     3464168 kB
Mapped:          20688 kB
Slab:           454036 kB
PageTables:      18848 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586980 kB
Committed_AS: 24270084 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73464 MiB
End job on node121 at 2011-05-02 23:05:40