Trace number 3385504

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)UNSAT 2474.04 355.174

General information on the benchmark

Namerandom/medium/
unif-k5-r21.3-v100-c2130-S2098066936-001.UNKNOWN.cnf
MD5SUM7a1181bb5cb0b1221253c27ad7ead33e
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark347.316
Satisfiable
(Un)Satisfiability was proved
Number of variables100
Number of clauses2130
Sum of the clauses size10650
Maximum clause length5
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 52130
Number of clauses of size over 50

Solver Data 

0.35/0.64	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.35/0.64	c INFO: solving HOME/instance-3385504-1305567535.cnf
0.35/0.64	c INFO: computing task features
0.35/0.65	c INFO: computed variable-clause graph statistics
0.35/0.65	c INFO: computed clause balance statistics
0.43/0.70	c INFO: computed variable balance statistics
0.43/0.73	c INFO: computed small-clause counts
0.43/0.73	c INFO: computed Horn-clause counts
0.43/0.74	c INFO: computed variable graph statistics
0.43/0.74	c INFO: collected features for HOME/instance-3385504-1305567535.cnf in 0.10 s
0.43/0.77	c INFO: running gnovelty+2@0 for 97 with 4999 remaining (b = 0.62)
0.53/0.80	c INFO: running march_hi@0 for 194 with 4999 remaining (b = 0.19)
0.53/0.83	c INFO: running gnovelty+2@0 for 97 with 4999 remaining (b = 0.07)
0.53/0.86	c INFO: running march_hi@0 for 388 with 4999 remaining (b = 0.29)
0.53/0.89	c INFO: running march_hi@0 for 485 with 4999 remaining (b = 0.30)
0.64/0.92	c INFO: running march_hi@0 for 485 with 4999 remaining (b = 0.27)
0.64/0.95	c INFO: running march_hi@0 for 485 with 4999 remaining (b = 0.24)
1.58/1.02	c INFO: running march_hi@0 for 388 with 4999 remaining (b = 0.18)
700.18/101.09	c INFO: running march_hi@0 for 388 with 4899 remaining (b = 0.15)
862.28/124.25	c INFO: running kcnfs-2006@0 for 582 with 4876 remaining (b = 0.19)
1376.57/197.95	c INFO: running march_hi@0 for 97 with 4802 remaining (b = 0.04)
2059.74/295.90	c INFO: running march_hi@97 for 97 with 4704 remaining (b = 0.04)
2473.50/355.14	s UNSATISFIABLE

Verifier Data 

No possible verification on an UNSAT instance

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3385504-1305567535/watcher-3385504-1305567535 -o /tmp/evaluation-result-3385504-1305567535/solver-3385504-1305567535 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385504-1305567535.cnf -seed 1468926289 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.76 8.55 7.92 2/405 10322
/proc/meminfo: memFree=22156496/32951124 swapFree=67023148/67111528
[pid=10322] ppid=10320 vsize=15384 CPUtime=0 cores=0-7
/proc/10322/stat : 10322 (python) R 10320 10322 9715 0 -1 4202496 796 0 0 0 0 0 0 0 21 0 1 0 649169385 15753216 684 33554432000 4194304 5683644 140735125281104 18446744073709551615 4540219 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/10322/statm: 3845 683 322 364 0 332 0

[startup+0.025492 s]
/proc/loadavg: 8.76 8.55 7.92 2/405 10322
/proc/meminfo: memFree=22156496/32951124 swapFree=67023148/67111528
[pid=10322] ppid=10320 vsize=16396 CPUtime=0 cores=0-7
/proc/10322/stat : 10322 (python) S 10320 10322 9715 0 -1 4202496 1115 0 0 0 0 0 0 0 21 0 1 0 649169385 16789504 984 33554432000 4194304 5683644 140735125281104 18446744073709551615 233510295552 0 2147483391 16781312 2 18446744071699207358 0 0 17 2 0 0 0
/proc/10322/statm: 4099 984 408 364 0 586 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 16396

[startup+0.100461 s]
/proc/loadavg: 8.76 8.55 7.92 2/405 10322
/proc/meminfo: memFree=22156496/32951124 swapFree=67023148/67111528
[pid=10322] ppid=10320 vsize=93960 CPUtime=0.05 cores=0-7
/proc/10322/stat : 10322 (python) R 10320 10322 9715 0 -1 4202496 2692 0 0 0 4 1 0 0 17 0 1 0 649169385 96215040 2407 33554432000 4194304 5683644 140735125281104 18446744073709551615 233510295552 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/10322/statm: 23490 2407 764 364 0 1622 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 93960

[startup+0.300443 s]
/proc/loadavg: 8.76 8.55 7.92 2/405 10322
/proc/meminfo: memFree=22156496/32951124 swapFree=67023148/67111528
[pid=10322] ppid=10320 vsize=135748 CPUtime=0.17 cores=0-7
/proc/10322/stat : 10322 (python) R 10320 10322 9715 0 -1 4202496 5965 934 1 0 13 4 0 0 18 0 1 0 649169385 139005952 4151 33554432000 4194304 5683644 140735125281104 18446744073709551615 4620076 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/10322/statm: 33937 4151 1032 364 0 3123 0
Current children cumulated CPU time (s) 0.17
Current children cumulated vsize (KiB) 135748

[startup+0.700316 s]
/proc/loadavg: 8.76 8.55 7.92 2/405 10322
/proc/meminfo: memFree=22156496/32951124 swapFree=67023148/67111528
[pid=10322] ppid=10320 vsize=278428 CPUtime=0.43 cores=0-7
/proc/10322/stat : 10322 (python) R 10320 10322 9715 0 -1 4202496 9334 934 1 0 34 9 0 0 17 0 1 0 649169385 285110272 7000 33554432000 4194304 5683644 140735125281104 18446744073709551615 233509958171 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/10322/statm: 69607 7000 1826 364 0 5531 0
Current children cumulated CPU time (s) 0.43
Current children cumulated vsize (KiB) 278428

[startup+1.50114 s]
/proc/loadavg: 8.76 8.55 7.92 11/427 10348
/proc/meminfo: memFree=22106332/32951124 swapFree=67023148/67111528
[pid=10322] ppid=10320 vsize=426024 CPUtime=0.72 cores=0-7
/proc/10322/stat : 10322 (python) S 10320 10322 9715 0 -1 4202496 15128 934 1 0 61 11 0 0 21 0 9 0 649169385 436248576 7288 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522124235 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/10322/statm: 106506 7288 1889 364 0 42403 0
[pid=10328] ppid=10322 vsize=278452 CPUtime=0 cores=0-7
/proc/10328/stat : 10328 (python) S 10322 10322 9715 0 -1 4202560 734 0 0 0 0 0 0 0 18 0 1 0 649169463 285134848 5552 33554432000 4194304 5683644 140735125281104 18446744073709551615 233510317583 0 0 16781312 514 3164658626702868479 0 0 17 1 0 0 0
/proc/10328/statm: 69613 5552 354 364 0 5531 0
[pid=10330] ppid=10328 vsize=1017800 CPUtime=0.72 cores=0-7
/proc/10330/stat : 10330 (gnovelty+2) R 10328 10330 10330 0 -1 4202496 436 0 0 0 72 0 0 0 21 0 1 0 649169463 1042227200 172 33554432000 134512640 135663704 4288905216 18446744073709551615 134525544 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/10330/statm: 254450 172 113 282 0 254166 0
[pid=10331] ppid=10322 vsize=354244 CPUtime=0 cores=0-7
/proc/10331/stat : 10331 (python) S 10322 10322 9715 0 -1 4202560 708 0 0 0 0 0 0 0 15 0 1 0 649169466 362745856 5723 33554432000 4194304 5683644 140735125281104 18446744073709551615 233510317638 0 0 16781312 514 5607369878429204739 0 0 17 6 0 0 0
/proc/10331/statm: 88561 5723 348 364 0 24476 0
[pid=10333] ppid=10331 vsize=1560 CPUtime=0.68 cores=0-7
/proc/10333/stat : 10333 (march_hi) R 10331 10333 10333 0 -1 4202496 495 0 0 0 68 0 0 0 22 0 1 0 649169466 1597440 240 33554432000 4194304 4955518 140734214739760 18446744073709551615 4224471 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/10333/statm: 390 240 106 186 0 203 0
[pid=10334] ppid=10322 vsize=364500 CPUtime=0 cores=0-7
/proc/10334/stat : 10334 (python) S 10322 10322 9715 0 -1 4202560 713 0 0 0 0 0 0 0 20 0 1 0 649169469 373248000 5686 33554432000 4194304 5683644 140735125281104 18446744073709551615 233510317638 0 0 16781312 514 8302791637143126015 0 0 17 1 0 0 0
/proc/10334/statm: 91125 5686 348 364 0 27037 0
[pid=10336] ppid=10334 vsize=1017800 CPUtime=0.66 cores=0-7
/proc/10336/stat : 10336 (gnovelty+2) R 10334 10336 10336 0 -1 4202496 427 0 0 0 66 0 0 0 24 0 1 0 649169469 1042227200 172 33554432000 134512640 135663704 4291462864 18446744073709551615 134524923 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/10336/statm: 254450 172 113 282 0 254166 0
[pid=10337] ppid=10322 vsize=374756 CPUtime=0 cores=0-7
/proc/10337/stat : 10337 (python) S 10322 10322 9715 0 -1 4202560 721 0 0 0 0 0 0 0 16 0 1 0 649169472 383750144 5708 33554432000 4194304 5683644 140735125281104 18446744073709551615 233510317638 0 0 16781312 514 16890842765724713218 0 0 17 1 0 0 0
/proc/10337/statm: 93689 5708 348 364 0 29598 0
[pid=10339] ppid=10337 vsize=1560 CPUtime=0.58 cores=0-7
/proc/10339/stat : 10339 (march_hi) R 10337 10339 10339 0 -1 4202496 492 0 0 0 58 0 0 0 23 0 1 0 649169472 1597440 239 33554432000 4194304 4955518 140737238699104 18446744073709551615 4224682 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/10339/statm: 390 239 106 186 0 203 0
[pid=10340] ppid=10322 vsize=385012 CPUtime=0 cores=0-7
/proc/10340/stat : 10340 (python) S 10322 10322 9715 0 -1 4202560 733 0 0 0 0 0 0 0 18 0 1 0 649169475 394252288 5729 33554432000 4194304 5683644 140735125281104 18446744073709551615 233510317638 0 0 16781312 514 16890851561817735427 0 0 17 1 0 0 0
/proc/10340/statm: 96253 5729 348 364 0 32159 0
[pid=10342] ppid=10340 vsize=1560 CPUtime=0.58 cores=0-7
/proc/10342/stat : 10342 (march_hi) R 10340 10342 10342 0 -1 4202496 499 0 0 0 58 0 0 0 25 0 1 0 649169475 1597440 240 33554432000 4194304 4955518 140735256827680 18446744073709551615 4314106 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/10342/statm: 390 240 106 186 0 203 0
[pid=10343] ppid=10322 vsize=395268 CPUtime=0 cores=0-7
/proc/10343/stat : 10343 (python) S 10322 10322 9715 0 -1 4202560 742 0 0 0 0 0 0 0 16 0 1 0 649169478 404754432 5716 33554432000 4194304 5683644 140735125281104 18446744073709551615 233510317638 0 0 16781312 514 6366660512280707330 0 0 17 1 0 0 0
/proc/10343/statm: 98817 5716 348 364 0 34720 0
[pid=10345] ppid=10343 vsize=1560 CPUtime=0.54 cores=0-7
/proc/10345/stat : 10345 (march_hi) R 10343 10345 10345 0 -1 4202496 501 0 0 0 54 0 0 0 22 0 1 0 649169480 1597440 240 33554432000 4194304 4955518 140735630500528 18446744073709551615 4224468 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/10345/statm: 390 240 106 186 0 203 0
[pid=10346] ppid=10322 vsize=405524 CPUtime=0 cores=0-7
/proc/10346/stat : 10346 (python) S 10322 10322 9715 0 -1 4202560 751 0 0 0 0 0 0 0 15 0 1 0 649169481 415256576 5840 33554432000 4194304 5683644 140735125281104 18446744073709551615 233510317638 0 0 16781312 514 14596764530928222466 0 0 17 0 0 0 0
/proc/10346/statm: 101381 5840 348 364 0 37281 0
[pid=10348] ppid=10346 vsize=1560 CPUtime=0.4 cores=0-7
/proc/10348/stat : 10348 (march_hi) R 10346 10348 10348 0 -1 4202496 502 0 0 0 40 0 0 0 22 0 1 0 649169482 1597440 240 33554432000 4194304 4955518 140737076708416 18446744073709551615 4297490 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/10348/statm: 390 240 106 186 0 203 0
[pid=10322/tid=10329] ppid=10320 vsize=426024 CPUtime=0 cores=0-7
/proc/10322/task/10329/stat : 10329 (python) S 10320 10322 9715 0 -1 4202560 30 934 0 0 0 0 0 0 18 0 9 0 649169463 436248576 7288 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 3 0 0 0
[pid=10322/tid=10332] ppid=10320 vsize=426024 CPUtime=0 cores=0-7
/proc/10322/task/10332/stat : 10332 (python) S 10320 10322 9715 0 -1 4202560 68 934 0 0 0 0 0 0 19 0 9 0 649169466 436248576 7288 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 0 0 0 0
[pid=10322/tid=10335] ppid=10320 vsize=426024 CPUtime=0 cores=0-7
/proc/10322/task/10335/stat : 10335 (python) S 10320 10322 9715 0 -1 4202560 32 934 0 0 0 0 0 0 18 0 9 0 649169469 436248576 7288 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 5 0 0 0
[pid=10322/tid=10338] ppid=10320 vsize=426024 CPUtime=0 cores=0-7
/proc/10322/task/10338/stat : 10338 (python) S 10320 10322 9715 0 -1 4202560 30 934 0 0 0 0 0 0 19 0 9 0 649169472 436248576 7288 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 0 0 0 0
[pid=10322/tid=10341] ppid=10320 vsize=426024 CPUtime=0 cores=0-7
/proc/10322/task/10341/stat : 10341 (python) S 10320 10322 9715 0 -1 4202560 31 934 0 0 0 0 0 0 20 0 9 0 649169475 436248576 7288 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 7 0 0 0
[pid=10322/tid=10344] ppid=10320 vsize=426024 CPUtime=0 cores=0-7
/proc/10322/task/10344/stat : 10344 (python) S 10320 10322 9715 0 -1 4202560 31 934 0 0 0 0 0 0 19 0 9 0 649169478 436248576 7288 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 1 0 0 0
[pid=10322/tid=10347] ppid=10320 vsize=426024 CPUtime=0 cores=0-7
/proc/10322/task/10347/stat : 10347 (python) S 10320 10322 9715 0 -1 4202560 33 934 0 0 0 0 0 0 20 0 9 0 649169481 436248576 7288 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4.88
Current children cumulated vsize (KiB) 5027180

[startup+3.10085 s]
/proc/loadavg: 8.78 8.56 7.93 10/430 10351
/proc/meminfo: memFree=22101968/32951124 swapFree=67023148/67111528
[pid=10322] ppid=10320 vsize=426024 CPUtime=0.72 cores=0-7
/proc/10322/stat : 10322 (python) S 10320 10322 9715 0 -1 4202496 15128 934 1 0 61 11 0 0 21 0 9 0 649169385 436248576 7288 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522124235 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/10322/statm: 106506 7288 1889 364 0 42403 0
[pid=10328] ppid=10322 vsize=278452 CPUtime=0 cores=0-7
/proc/10328/stat : 10328 (python) S 10322 10322 9715 0 -1 4202560 734 0 0 0 0 0 0 0 18 0 1 0 649169463 285134848 5552 33554432000 4194304 5683644 140735125281104 18446744073709551615 233510317583 0 0 16781312 514 3164658626702835972 0 0 17 1 0 0 0
/proc/10328/statm: 69613 5552 354 364 0 5531 0
[pid=10330] ppid=10328 vsize=1017800 CPUtime=2.32 cores=0-7
/proc/10330/stat : 10330 (gnovelty+2) R 10328 10330 10330 0 -1 4202496 436 0 0 0 232 0 0 0 25 0 1 0 649169463 1042227200 172 33554432000 134512640 135663704 4288905216 18446744073709551615 134545727 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/10330/statm: 254450 172 113 282 0 254166 0
[pid=10331] ppid=10322 vsize=354244 CPUtime=0 cores=0-7
/proc/10331/stat : 10331 (python) S 10322 10322 9715 0 -1 4202560 710 0 0 0 0 0 0 0 15 0 1 0 649169466 362745856 5723 33554432000 4194304 5683644 140735125281104 18446744073709551615 233510317638 0 0 16781312 514 4731216130801107203 0 0 17 6 0 0 0
/proc/10331/statm: 88561 5723 348 364 0 24476 0
[pid=10333] ppid=10331 vsize=1560 CPUtime=2.28 cores=0-7
/proc/10333/stat : 10333 (march_hi) R 10331 10333 10333 0 -1 4202496 498 0 0 0 228 0 0 0 25 0 1 0 649169466 1597440 243 33554432000 4194304 4955518 140734214739760 18446744073709551615 4237810 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/10333/statm: 390 243 106 186 0 203 0
[pid=10334] ppid=10322 vsize=364500 CPUtime=0 cores=0-7
/proc/10334/stat : 10334 (python) S 10322 10322 9715 0 -1 4202560 713 0 0 0 0 0 0 0 18 0 1 0 649169469 373248000 5686 33554432000 4194304 5683644 140735125281104 18446744073709551615 233510317638 0 0 16781312 514 8302791637143093506 0 0 17 1 0 0 0
/proc/10334/statm: 91125 5686 348 364 0 27037 0
[pid=10336] ppid=10334 vsize=1017800 CPUtime=2.26 cores=0-7
/proc/10336/stat : 10336 (gnovelty+2) R 10334 10336 10336 0 -1 4202496 427 0 0 0 226 0 0 0 25 0 1 0 649169469 1042227200 172 33554432000 134512640 135663704 4291462864 18446744073709551615 134524888 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/10336/statm: 254450 172 113 282 0 254166 0
[pid=10337] ppid=10322 vsize=374756 CPUtime=0 cores=0-7
/proc/10337/stat : 10337 (python) S 10322 10322 9715 0 -1 4202560 723 0 0 0 0 0 0 0 15 0 1 0 649169472 383750144 5708 33554432000 4194304 5683644 140735125281104 18446744073709551615 233510317638 0 0 16781312 514 16890842765724713218 0 0 17 1 0 0 0
/proc/10337/statm: 93689 5708 348 364 0 29598 0
[pid=10339] ppid=10337 vsize=1560 CPUtime=2.17 cores=0-7
/proc/10339/stat : 10339 (march_hi) R 10337 10339 10339 0 -1 4202496 494 0 0 0 217 0 0 0 25 0 1 0 649169472 1597440 241 33554432000 4194304 4955518 140737238699104 18446744073709551615 4283048 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/10339/statm: 390 241 106 186 0 203 0
[pid=10340] ppid=10322 vsize=385012 CPUtime=0 cores=0-7
/proc/10340/stat : 10340 (python) S 10322 10322 9715 0 -1 4202560 734 0 0 0 0 0 0 0 15 0 1 0 649169475 394252288 5729 33554432000 4194304 5683644 140735125281104 18446744073709551615 233510317638 0 0 16781312 514 16890851561817735427 0 0 17 1 0 0 0
/proc/10340/statm: 96253 5729 348 364 0 32159 0
[pid=10342] ppid=10340 vsize=1560 CPUtime=2.17 cores=0-7
/proc/10342/stat : 10342 (march_hi) R 10340 10342 10342 0 -1 4202496 501 0 0 0 217 0 0 0 25 0 1 0 649169475 1597440 242 33554432000 4194304 4955518 140735256827680 18446744073709551615 4314282 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/10342/statm: 390 242 106 186 0 203 0
[pid=10343] ppid=10322 vsize=395268 CPUtime=0 cores=0-7
/proc/10343/stat : 10343 (python) S 10322 10322 9715 0 -1 4202560 742 0 0 0 0 0 0 0 15 0 1 0 649169478 404754432 5716 33554432000 4194304 5683644 140735125281104 18446744073709551615 233510317638 0 0 16781312 514 6366660512280707330 0 0 17 1 0 0 0
/proc/10343/statm: 98817 5716 348 364 0 34720 0
[pid=10345] ppid=10343 vsize=1560 CPUtime=2.14 cores=0-7
/proc/10345/stat : 10345 (march_hi) R 10343 10345 10345 0 -1 4202496 503 0 0 0 214 0 0 0 25 0 1 0 649169480 1597440 242 33554432000 4194304 4955518 140735630500528 18446744073709551615 4208624 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/10345/statm: 390 242 106 186 0 203 0
[pid=10346] ppid=10322 vsize=405524 CPUtime=0 cores=0-7
/proc/10346/stat : 10346 (python) S 10322 10322 9715 0 -1 4202560 752 0 0 0 0 0 0 0 15 0 1 0 649169481 415256576 5840 33554432000 4194304 5683644 140735125281104 18446744073709551615 233510317638 0 0 16781312 514 14596764530928222466 0 0 17 0 0 0 0
/proc/10346/statm: 101381 5840 348 364 0 37281 0
[pid=10348] ppid=10346 vsize=1560 CPUtime=1.21 cores=0-7
/proc/10348/stat : 10348 (march_hi) R 10346 10348 10348 0 -1 4202496 503 0 0 0 121 0 0 0 25 0 1 0 649169482 1597440 241 33554432000 4194304 4955518 140737076708416 18446744073709551615 4237810 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/10348/statm: 390 241 106 186 0 203 0
[pid=10349] ppid=10322 vsize=415780 CPUtime=0 cores=0-7
/proc/10349/stat : 10349 (python) S 10322 10322 9715 0 -1 4202560 791 0 0 0 0 0 0 0 15 0 1 0 649169488 425758720 5749 33554432000 4194304 5683644 140735125281104 18446744073709551615 233510317638 0 0 16781312 514 8734304871068434689 0 0 17 2 0 0 0
/proc/10349/statm: 103945 5749 354 364 0 39842 0
[pid=10351] ppid=10349 vsize=1560 CPUtime=0.9 cores=0-7
/proc/10351/stat : 10351 (march_hi) R 10349 10351 10351 0 -1 4202496 504 0 0 0 90 0 0 0 25 0 1 0 649169499 1597440 240 33554432000 4194304 4955518 140733316228512 18446744073709551615 4208667 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/10351/statm: 390 240 106 186 0 203 0
[pid=10322/tid=10329] ppid=10320 vsize=426024 CPUtime=0 cores=0-7
/proc/10322/task/10329/stat : 10329 (python) S 10320 10322 9715 0 -1 4202560 30 934 0 0 0 0 0 0 18 0 9 0 649169463 436248576 7288 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 3 0 0 0
[pid=10322/tid=10332] ppid=10320 vsize=426024 CPUtime=0 cores=0-7
/proc/10322/task/10332/stat : 10332 (python) S 10320 10322 9715 0 -1 4202560 68 934 0 0 0 0 0 0 19 0 9 0 649169466 436248576 7288 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 0 0 0 0
[pid=10322/tid=10335] ppid=10320 vsize=426024 CPUtime=0 cores=0-7
/proc/10322/task/10335/stat : 10335 (python) S 10320 10322 9715 0 -1 4202560 32 934 0 0 0 0 0 0 18 0 9 0 649169469 436248576 7288 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 5 0 0 0
[pid=10322/tid=10338] ppid=10320 vsize=426024 CPUtime=0 cores=0-7
/proc/10322/task/10338/stat : 10338 (python) S 10320 10322 9715 0 -1 4202560 30 934 0 0 0 0 0 0 19 0 9 0 649169472 436248576 7288 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 0 0 0 0
[pid=10322/tid=10341] ppid=10320 vsize=426024 CPUtime=0 cores=0-7
/proc/10322/task/10341/stat : 10341 (python) S 10320 10322 9715 0 -1 4202560 31 934 0 0 0 0 0 0 20 0 9 0 649169475 436248576 7288 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 7 0 0 0
[pid=10322/tid=10344] ppid=10320 vsize=426024 CPUtime=0 cores=0-7
/proc/10322/task/10344/stat : 10344 (python) S 10320 10322 9715 0 -1 4202560 31 934 0 0 0 0 0 0 19 0 9 0 649169478 436248576 7288 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 1 0 0 0
[pid=10322/tid=10347] ppid=10320 vsize=426024 CPUtime=0 cores=0-7
/proc/10322/task/10347/stat : 10347 (python) S 10320 10322 9715 0 -1 4202560 33 934 0 0 0 0 0 0 20 0 9 0 649169481 436248576 7288 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 1 0 0 0
[pid=10322/tid=10350] ppid=10320 vsize=426024 CPUtime=0 cores=0-7
/proc/10322/task/10350/stat : 10350 (python) S 10320 10322 9715 0 -1 4202560 33 934 0 0 0 0 0 0 22 0 9 0 649169489 436248576 7288 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 16.17

################
# More data... #
################

/proc/10333/statm: 3379 2836 106 186 0 3192 0
[pid=10331/tid=10366] ppid=10322 vsize=354244 CPUtime=0 cores=0-7
/proc/10331/task/10366/stat : 10366 (python) S 10322 10322 9715 0 -1 4202560 99 0 0 0 0 0 0 0 15 0 2 0 649189178 362745856 5776 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 514 18446604450792517760 0 0 -1 6 0 0 0
[pid=10334] ppid=10322 vsize=364500 CPUtime=0.37 cores=0-7
/proc/10334/stat : 10334 (python) S 10322 10322 9715 0 -1 4202560 959 0 0 0 20 17 0 0 19 0 2 0 649169469 373248000 5739 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522124235 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/10334/statm: 91125 5739 400 364 0 27037 0
[pid=10336] ppid=10334 vsize=1017800 CPUtime=98.3 cores=0-7
/proc/10336/stat : 10336 (gnovelty+2) T 10334 10336 10336 0 -1 4202496 427 0 0 0 9830 0 0 0 25 0 1 0 649169469 1042227200 172 33554432000 134512640 135663704 4291462864 18446744073709551615 134545619 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/10336/statm: 254450 172 113 282 0 254166 0
[pid=10334/tid=10360] ppid=10322 vsize=364500 CPUtime=0 cores=0-7
/proc/10334/task/10360/stat : 10360 (python) S 10322 10322 9715 0 -1 4202560 63 0 0 0 0 0 0 0 19 0 2 0 649181808 373248000 5739 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 514 18446604446255902976 0 0 -1 0 0 0 0
[pid=10337] ppid=10322 vsize=374756 CPUtime=1.22 cores=0-7
/proc/10337/stat : 10337 (python) S 10322 10322 9715 0 -1 4202560 815 0 0 0 67 55 0 0 15 0 1 0 649169472 383750144 5731 33554432000 4194304 5683644 140735125281104 18446744073709551615 233510317638 0 0 16781312 514 16890842765724713218 0 0 17 7 0 0 0
/proc/10337/statm: 93689 5731 371 364 0 29598 0
[pid=10339] ppid=10337 vsize=13516 CPUtime=318.75 cores=0-7
/proc/10339/stat : 10339 (march_hi) R 10337 10339 10339 0 -1 4202496 2603 0 0 0 31874 1 0 0 25 0 1 0 649169472 13840384 2350 33554432000 4194304 4955518 140737238699104 18446744073709551615 4208656 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/10339/statm: 3379 2350 106 186 0 3192 0
[pid=10340] ppid=10322 vsize=385012 CPUtime=1.22 cores=0-7
/proc/10340/stat : 10340 (python) S 10322 10322 9715 0 -1 4202560 817 0 0 0 63 59 0 0 15 0 1 0 649169475 394252288 5752 33554432000 4194304 5683644 140735125281104 18446744073709551615 233510317638 0 0 16781312 514 16890851561817735427 0 0 17 0 0 0 0
/proc/10340/statm: 96253 5752 371 364 0 32159 0
[pid=10342] ppid=10340 vsize=9456 CPUtime=294.73 cores=0-7
/proc/10342/stat : 10342 (march_hi) R 10340 10342 10342 0 -1 4202496 2277 0 0 0 29471 2 0 0 25 0 1 0 649169475 9682944 2018 33554432000 4194304 4955518 140735256827680 18446744073709551615 4297472 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/10342/statm: 2364 2018 106 186 0 2177 0
[pid=10343] ppid=10322 vsize=395268 CPUtime=1.21 cores=0-7
/proc/10343/stat : 10343 (python) S 10322 10322 9715 0 -1 4202560 839 0 0 0 70 51 0 0 15 0 1 0 649169478 404754432 5740 33554432000 4194304 5683644 140735125281104 18446744073709551615 233510317638 0 0 16781312 514 11246438247590494466 0 0 17 5 0 0 0
/proc/10343/statm: 98817 5740 372 364 0 34720 0
[pid=10345] ppid=10343 vsize=13516 CPUtime=339.63 cores=0-7
/proc/10345/stat : 10345 (march_hi) R 10343 10345 10345 0 -1 4202496 2902 0 0 0 33963 0 0 0 25 0 1 0 649169480 13840384 2641 33554432000 4194304 4955518 140735630500528 18446744073709551615 4307199 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/10345/statm: 3379 2641 106 186 0 3192 0
[pid=10346] ppid=10322 vsize=405524 CPUtime=1.21 cores=0-7
/proc/10346/stat : 10346 (python) S 10322 10322 9715 0 -1 4202560 838 0 0 0 60 61 0 0 15 0 1 0 649169481 415256576 5863 33554432000 4194304 5683644 140735125281104 18446744073709551615 233510317638 0 0 16781312 514 14596764530928222466 0 0 17 3 0 0 0
/proc/10346/statm: 101381 5863 371 364 0 37281 0
[pid=10348] ppid=10346 vsize=13516 CPUtime=335.03 cores=0-7
/proc/10348/stat : 10348 (march_hi) R 10346 10348 10348 0 -1 4202496 2842 0 0 0 33502 1 0 0 25 0 1 0 649169482 13840384 2580 33554432000 4194304 4955518 140737076708416 18446744073709551615 4208585 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/10348/statm: 3379 2580 106 186 0 3192 0
[pid=10349] ppid=10322 vsize=415780 CPUtime=1.18 cores=0-7
/proc/10349/stat : 10349 (python) S 10322 10322 9715 0 -1 4202560 875 0 0 0 61 57 0 0 15 0 1 0 649169488 425758720 5772 33554432000 4194304 5683644 140735125281104 18446744073709551615 233510317638 0 0 16781312 514 8734304871068434689 0 0 17 1 0 0 0
/proc/10349/statm: 103945 5772 377 364 0 39842 0
[pid=10351] ppid=10349 vsize=9456 CPUtime=258.87 cores=0-7
/proc/10351/stat : 10351 (march_hi) R 10349 10351 10351 0 -1 4202496 1949 0 0 0 25887 0 0 0 25 0 1 0 649169499 9682944 1685 33554432000 4194304 4955518 140733316228512 18446744073709551615 4208676 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/10351/statm: 2364 1685 106 186 0 2177 0
[pid=10356] ppid=10322 vsize=426036 CPUtime=0.85 cores=0-7
/proc/10356/stat : 10356 (python) S 10322 10322 9715 0 -1 4202560 850 0 0 0 42 43 0 0 15 0 1 0 649179495 436260864 5790 33554432000 4194304 5683644 140735125281104 18446744073709551615 233510317638 0 0 16781312 514 6849662776223170820 0 0 17 1 0 0 0
/proc/10356/statm: 106509 5790 371 364 0 42403 0
[pid=10358] ppid=10356 vsize=6868 CPUtime=215.35 cores=0-7
/proc/10358/stat : 10358 (march_hi) R 10356 10358 10358 0 -1 4202496 1588 0 0 0 21535 0 0 0 25 0 1 0 649179496 7032832 1325 33554432000 4194304 4955518 140735666052064 18446744073709551615 4224687 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/10358/statm: 1717 1325 106 186 0 1530 0
[pid=10361] ppid=10322 vsize=436292 CPUtime=0.73 cores=0-7
/proc/10361/stat : 10361 (python) S 10322 10322 9715 0 -1 4202560 853 0 0 0 38 35 0 0 18 0 1 0 649181811 446763008 5828 33554432000 4194304 5683644 140735125281104 18446744073709551615 233510317638 0 0 16781312 514 11246438247590494466 0 0 17 5 0 0 0
/proc/10361/statm: 109073 5828 371 364 0 44964 0
[pid=10363] ppid=10361 vsize=181248 CPUtime=147.91 cores=0-7
/proc/10363/stat : 10363 (kcnfs-2006) R 10361 10363 10363 0 -1 4202496 621 0 0 0 14791 0 0 0 25 0 1 0 649181812 185597952 359 33554432000 134512640 135163436 4289317424 18446744073709551615 134569438 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/10363/statm: 45312 359 76 159 0 45152 0
[pid=10322/tid=10329] ppid=10320 vsize=446536 CPUtime=0 cores=0-7
/proc/10322/task/10329/stat : 10329 (python) S 10320 10322 9715 0 -1 4202560 30 934 0 0 0 0 0 0 18 0 11 0 649169463 457252864 7321 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 3 0 0 0
[pid=10322/tid=10332] ppid=10320 vsize=446536 CPUtime=0 cores=0-7
/proc/10322/task/10332/stat : 10332 (python) S 10320 10322 9715 0 -1 4202560 74 934 0 0 0 0 0 0 18 0 11 0 649169466 457252864 7321 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 0 0 0 0
[pid=10322/tid=10335] ppid=10320 vsize=446536 CPUtime=0 cores=0-7
/proc/10322/task/10335/stat : 10335 (python) S 10320 10322 9715 0 -1 4202560 32 934 0 0 0 0 0 0 18 0 11 0 649169469 457252864 7321 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 5 0 0 0
[pid=10322/tid=10338] ppid=10320 vsize=446536 CPUtime=0 cores=0-7
/proc/10322/task/10338/stat : 10338 (python) S 10320 10322 9715 0 -1 4202560 30 934 0 0 0 0 0 0 19 0 11 0 649169472 457252864 7321 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 0 0 0 0
[pid=10322/tid=10341] ppid=10320 vsize=446536 CPUtime=0 cores=0-7
/proc/10322/task/10341/stat : 10341 (python) S 10320 10322 9715 0 -1 4202560 31 934 0 0 0 0 0 0 20 0 11 0 649169475 457252864 7321 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 7 0 0 0
[pid=10322/tid=10344] ppid=10320 vsize=446536 CPUtime=0 cores=0-7
/proc/10322/task/10344/stat : 10344 (python) S 10320 10322 9715 0 -1 4202560 31 934 0 0 0 0 0 0 19 0 11 0 649169478 457252864 7321 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 1 0 0 0
[pid=10322/tid=10347] ppid=10320 vsize=446536 CPUtime=0 cores=0-7
/proc/10322/task/10347/stat : 10347 (python) S 10320 10322 9715 0 -1 4202560 33 934 0 0 0 0 0 0 20 0 11 0 649169481 457252864 7321 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 1 0 0 0
[pid=10322/tid=10350] ppid=10320 vsize=446536 CPUtime=0 cores=0-7
/proc/10322/task/10350/stat : 10350 (python) S 10320 10322 9715 0 -1 4202560 33 934 0 0 0 0 0 0 22 0 11 0 649169489 457252864 7321 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 2 0 0 0
[pid=10322/tid=10357] ppid=10320 vsize=446536 CPUtime=0 cores=0-7
/proc/10322/task/10357/stat : 10357 (python) S 10320 10322 9715 0 -1 4202560 35 934 0 0 0 0 0 0 19 0 11 0 649179495 457252864 7321 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 2 0 0 0
[pid=10322/tid=10362] ppid=10320 vsize=446536 CPUtime=0 cores=0-7
/proc/10322/task/10362/stat : 10362 (python) S 10320 10322 9715 0 -1 4202560 35 934 0 0 0 0 0 0 20 0 11 0 649181812 457252864 7321 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 2470.71
Current children cumulated vsize (KiB) 6654872

[startup+355.102 s]
/proc/loadavg: 9.08 8.94 8.29 10/439 10370
/proc/meminfo: memFree=22039220/32951124 swapFree=67023148/67111528
[pid=10322] ppid=10320 vsize=446536 CPUtime=0.86 cores=0-7
/proc/10322/stat : 10322 (python) S 10320 10322 9715 0 -1 4202496 17492 934 1 0 74 12 0 0 18 0 11 0 649169385 457252864 7321 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522124235 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/10322/statm: 111634 7321 1894 364 0 47525 0
[pid=10328] ppid=10322 vsize=354232 CPUtime=0.29 cores=0-7
/proc/10328/stat : 10328 (python) S 10322 10322 9715 0 -1 4202560 1087 0 0 0 15 14 0 0 18 0 2 0 649169463 362733568 5615 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522124235 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/10328/statm: 88558 5615 410 364 0 24476 0
[pid=10330] ppid=10328 vsize=1017800 CPUtime=100.29 cores=0-7
/proc/10330/stat : 10330 (gnovelty+2) T 10328 10330 10330 0 -1 4202496 436 0 0 0 10029 0 0 0 25 0 1 0 649169463 1042227200 172 33554432000 134512640 135663704 4288905216 18446744073709551615 134525218 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/10330/statm: 254450 172 113 282 0 254166 0
[pid=10328/tid=10355] ppid=10322 vsize=354232 CPUtime=0 cores=0-7
/proc/10328/task/10355/stat : 10355 (python) S 10322 10322 9715 0 -1 4202560 49 0 0 0 0 0 0 0 18 0 2 0 649179492 362733568 5615 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 514 18446604445059588160 0 0 -1 0 0 0 0
[pid=10331] ppid=10322 vsize=354244 CPUtime=1.5 cores=0-7
/proc/10331/stat : 10331 (python) R 10322 10322 9715 0 -1 4202560 1012 0 0 0 86 64 0 0 16 0 2 0 649169466 362745856 5776 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522127679 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/10331/statm: 88561 5776 400 364 0 24476 0
[pid=10333] ppid=10331 vsize=0 CPUtime=351.61 cores=0-7
/proc/10333/stat : 10333 (march_hi) R 10331 10333 10333 0 -1 4202500 3095 0 0 0 35160 1 0 0 25 0 1 0 649169466 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 5 0 0 0
/proc/10333/statm: 0 0 0 0 0 0 0
[pid=10331/tid=10366] ppid=10322 vsize=354244 CPUtime=0 cores=0-7
/proc/10331/task/10366/stat : 10366 (python) S 10322 10322 9715 0 -1 4202560 99 0 0 0 0 0 0 0 15 0 2 0 649189178 362745856 5776 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 514 18446604450792517760 0 0 -1 6 0 0 0
[pid=10334] ppid=10322 vsize=364500 CPUtime=0.37 cores=0-7
/proc/10334/stat : 10334 (python) S 10322 10322 9715 0 -1 4202560 959 0 0 0 20 17 0 0 19 0 2 0 649169469 373248000 5739 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522124235 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/10334/statm: 91125 5739 400 364 0 27037 0
[pid=10336] ppid=10334 vsize=1017800 CPUtime=98.3 cores=0-7
/proc/10336/stat : 10336 (gnovelty+2) T 10334 10336 10336 0 -1 4202496 427 0 0 0 9830 0 0 0 25 0 1 0 649169469 1042227200 172 33554432000 134512640 135663704 4291462864 18446744073709551615 134545619 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/10336/statm: 254450 172 113 282 0 254166 0
[pid=10334/tid=10360] ppid=10322 vsize=364500 CPUtime=0 cores=0-7
/proc/10334/task/10360/stat : 10360 (python) S 10322 10322 9715 0 -1 4202560 63 0 0 0 0 0 0 0 19 0 2 0 649181808 373248000 5739 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 514 18446604446255902976 0 0 -1 0 0 0 0
[pid=10337] ppid=10322 vsize=374756 CPUtime=1.22 cores=0-7
/proc/10337/stat : 10337 (python) S 10322 10322 9715 0 -1 4202560 815 0 0 0 67 55 0 0 15 0 1 0 649169472 383750144 5731 33554432000 4194304 5683644 140735125281104 18446744073709551615 233510317638 0 0 16781312 514 16890842765724713218 0 0 17 7 0 0 0
/proc/10337/statm: 93689 5731 371 364 0 29598 0
[pid=10339] ppid=10337 vsize=13516 CPUtime=319.14 cores=0-7
/proc/10339/stat : 10339 (march_hi) R 10337 10339 10339 0 -1 4202496 2609 0 0 0 31913 1 0 0 25 0 1 0 649169472 13840384 2356 33554432000 4194304 4955518 140737238699104 18446744073709551615 4224445 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/10339/statm: 3379 2356 106 186 0 3192 0
[pid=10340] ppid=10322 vsize=385012 CPUtime=1.22 cores=0-7
/proc/10340/stat : 10340 (python) S 10322 10322 9715 0 -1 4202560 817 0 0 0 63 59 0 0 15 0 1 0 649169475 394252288 5752 33554432000 4194304 5683644 140735125281104 18446744073709551615 233510317638 0 0 16781312 514 16890851561817735427 0 0 17 0 0 0 0
/proc/10340/statm: 96253 5752 371 364 0 32159 0
[pid=10342] ppid=10340 vsize=9456 CPUtime=295.13 cores=0-7
/proc/10342/stat : 10342 (march_hi) R 10340 10342 10342 0 -1 4202496 2282 0 0 0 29511 2 0 0 25 0 1 0 649169475 9682944 2023 33554432000 4194304 4955518 140735256827680 18446744073709551615 4224642 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/10342/statm: 2364 2023 106 186 0 2177 0
[pid=10343] ppid=10322 vsize=395268 CPUtime=1.21 cores=0-7
/proc/10343/stat : 10343 (python) S 10322 10322 9715 0 -1 4202560 839 0 0 0 70 51 0 0 15 0 1 0 649169478 404754432 5740 33554432000 4194304 5683644 140735125281104 18446744073709551615 233510317638 0 0 16781312 514 6366660512280707330 0 0 17 5 0 0 0
/proc/10343/statm: 98817 5740 372 364 0 34720 0
[pid=10345] ppid=10343 vsize=13516 CPUtime=340.03 cores=0-7
/proc/10345/stat : 10345 (march_hi) R 10343 10345 10345 0 -1 4202496 2909 0 0 0 34003 0 0 0 25 0 1 0 649169480 13840384 2648 33554432000 4194304 4955518 140735630500528 18446744073709551615 4208638 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/10345/statm: 3379 2648 106 186 0 3192 0
[pid=10346] ppid=10322 vsize=405524 CPUtime=1.21 cores=0-7
/proc/10346/stat : 10346 (python) S 10322 10322 9715 0 -1 4202560 838 0 0 0 60 61 0 0 15 0 1 0 649169481 415256576 5863 33554432000 4194304 5683644 140735125281104 18446744073709551615 233510317638 0 0 16781312 514 14596764530928222466 0 0 17 3 0 0 0
/proc/10346/statm: 101381 5863 371 364 0 37281 0
[pid=10348] ppid=10346 vsize=13516 CPUtime=335.43 cores=0-7
/proc/10348/stat : 10348 (march_hi) R 10346 10348 10348 0 -1 4202496 2850 0 0 0 33542 1 0 0 25 0 1 0 649169482 13840384 2588 33554432000 4194304 4955518 140737076708416 18446744073709551615 4296760 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/10348/statm: 3379 2588 106 186 0 3192 0
[pid=10349] ppid=10322 vsize=415780 CPUtime=1.18 cores=0-7
/proc/10349/stat : 10349 (python) S 10322 10322 9715 0 -1 4202560 875 0 0 0 61 57 0 0 15 0 1 0 649169488 425758720 5772 33554432000 4194304 5683644 140735125281104 18446744073709551615 233510317638 0 0 16781312 514 8734304871068434689 0 0 17 1 0 0 0
/proc/10349/statm: 103945 5772 377 364 0 39842 0
[pid=10351] ppid=10349 vsize=9456 CPUtime=259.07 cores=0-7
/proc/10351/stat : 10351 (march_hi) R 10349 10351 10351 0 -1 4202496 1952 0 0 0 25907 0 0 0 25 0 1 0 649169499 9682944 1688 33554432000 4194304 4955518 140733316228512 18446744073709551615 4285588 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/10351/statm: 2364 1688 106 186 0 2177 0
[pid=10356] ppid=10322 vsize=426036 CPUtime=0.85 cores=0-7
/proc/10356/stat : 10356 (python) S 10322 10322 9715 0 -1 4202560 850 0 0 0 42 43 0 0 15 0 1 0 649179495 436260864 5790 33554432000 4194304 5683644 140735125281104 18446744073709551615 233510317638 0 0 16781312 514 6849662776223170820 0 0 17 1 0 0 0
/proc/10356/statm: 106509 5790 371 364 0 42403 0
[pid=10358] ppid=10356 vsize=6868 CPUtime=215.75 cores=0-7
/proc/10358/stat : 10358 (march_hi) R 10356 10358 10358 0 -1 4202496 1592 0 0 0 21575 0 0 0 25 0 1 0 649179496 7032832 1329 33554432000 4194304 4955518 140735666052064 18446744073709551615 4224543 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/10358/statm: 1717 1329 106 186 0 1530 0
[pid=10361] ppid=10322 vsize=436292 CPUtime=0.73 cores=0-7
/proc/10361/stat : 10361 (python) S 10322 10322 9715 0 -1 4202560 853 0 0 0 38 35 0 0 18 0 1 0 649181811 446763008 5828 33554432000 4194304 5683644 140735125281104 18446744073709551615 233510317638 0 0 16781312 514 11246438247590494466 0 0 17 5 0 0 0
/proc/10361/statm: 109073 5828 371 364 0 44964 0
[pid=10363] ppid=10361 vsize=181248 CPUtime=148.11 cores=0-7
/proc/10363/stat : 10363 (kcnfs-2006) R 10361 10363 10363 0 -1 4202496 621 0 0 0 14811 0 0 0 25 0 1 0 649181812 185597952 359 33554432000 134512640 135163436 4289317424 18446744073709551615 134712495 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/10363/statm: 45312 359 76 159 0 45152 0
[pid=10322/tid=10329] ppid=10320 vsize=446536 CPUtime=0 cores=0-7
/proc/10322/task/10329/stat : 10329 (python) S 10320 10322 9715 0 -1 4202560 30 934 0 0 0 0 0 0 18 0 11 0 649169463 457252864 7321 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 3 0 0 0
[pid=10322/tid=10332] ppid=10320 vsize=446536 CPUtime=0 cores=0-7
/proc/10322/task/10332/stat : 10332 (python) S 10320 10322 9715 0 -1 4202560 74 934 0 0 0 0 0 0 18 0 11 0 649169466 457252864 7321 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 0 0 0 0
[pid=10322/tid=10335] ppid=10320 vsize=446536 CPUtime=0 cores=0-7
/proc/10322/task/10335/stat : 10335 (python) S 10320 10322 9715 0 -1 4202560 32 934 0 0 0 0 0 0 18 0 11 0 649169469 457252864 7321 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 5 0 0 0
[pid=10322/tid=10338] ppid=10320 vsize=446536 CPUtime=0 cores=0-7
/proc/10322/task/10338/stat : 10338 (python) S 10320 10322 9715 0 -1 4202560 30 934 0 0 0 0 0 0 19 0 11 0 649169472 457252864 7321 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 0 0 0 0
[pid=10322/tid=10341] ppid=10320 vsize=446536 CPUtime=0 cores=0-7
/proc/10322/task/10341/stat : 10341 (python) S 10320 10322 9715 0 -1 4202560 31 934 0 0 0 0 0 0 20 0 11 0 649169475 457252864 7321 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 7 0 0 0
[pid=10322/tid=10344] ppid=10320 vsize=446536 CPUtime=0 cores=0-7
/proc/10322/task/10344/stat : 10344 (python) S 10320 10322 9715 0 -1 4202560 31 934 0 0 0 0 0 0 19 0 11 0 649169478 457252864 7321 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 1 0 0 0
[pid=10322/tid=10347] ppid=10320 vsize=446536 CPUtime=0 cores=0-7
/proc/10322/task/10347/stat : 10347 (python) S 10320 10322 9715 0 -1 4202560 33 934 0 0 0 0 0 0 20 0 11 0 649169481 457252864 7321 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 1 0 0 0
[pid=10322/tid=10350] ppid=10320 vsize=446536 CPUtime=0 cores=0-7
/proc/10322/task/10350/stat : 10350 (python) S 10320 10322 9715 0 -1 4202560 33 934 0 0 0 0 0 0 22 0 11 0 649169489 457252864 7321 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 2 0 0 0
[pid=10322/tid=10357] ppid=10320 vsize=446536 CPUtime=0 cores=0-7
/proc/10322/task/10357/stat : 10357 (python) S 10320 10322 9715 0 -1 4202560 35 934 0 0 0 0 0 0 19 0 11 0 649179495 457252864 7321 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 2 0 0 0
[pid=10322/tid=10362] ppid=10320 vsize=446536 CPUtime=0 cores=0-7
/proc/10322/task/10362/stat : 10362 (python) S 10320 10322 9715 0 -1 4202560 35 934 0 0 0 0 0 0 20 0 11 0 649181812 457252864 7321 33554432000 4194304 5683644 140735125281104 18446744073709551615 233522121137 0 0 16781312 2 18446604443661792512 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 2473.5
Current children cumulated vsize (KiB) 6641356

Child status: 0
Real time (s): 355.174
CPU time (s): 2474.04
CPU user time (s): 2469.18
CPU system time (s): 4.85326
CPU usage (%): 696.57
Max. virtual memory (cumulated for all children) (KiB): 6654876

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2469.18
system time used= 4.85326
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 50762
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 8624
involuntary context switches= 20311

runsolver used 2.25766 second user time and 4.41433 second system time

The end

Launcher Data 

Begin job on node145 at 2011-05-16 19:38:55
IDJOB=3385504
IDBENCH=82769
IDSOLVER=1870
FILE ID=node145/3385504-1305567535
RUNJOBID= node145-1305564961-9729
PBS_JOBID= 13350105
Free space on /tmp= 73644 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/medium/unif-k5-r21.3-v100-c2130-S2098066936-001.UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3385504-1305567535/watcher-3385504-1305567535 -o /tmp/evaluation-result-3385504-1305567535/solver-3385504-1305567535 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385504-1305567535.cnf -seed 1468926289 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 7a1181bb5cb0b1221253c27ad7ead33e
RANDOM SEED=1468926289

node145.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.57
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      22156652 kB
Buffers:         95112 kB
Cached:         940436 kB
SwapCached:       4796 kB
Active:        6995172 kB
Inactive:      3542696 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      22156652 kB
SwapTotal:    67111528 kB
SwapFree:     67023148 kB
Dirty:           31868 kB
Writeback:           4 kB
AnonPages:     9497628 kB
Mapped:          30804 kB
Slab:           137104 kB
PageTables:      48616 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 108227168 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73612 MiB
End job on node145 at 2011-05-16 19:44:51