Trace number 3378592

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)UNSAT 1810.08 454.616

General information on the benchmark

Namerandom/medium/
unif-k3-r4.26-v500-c2130-S247862791-097.UNKNOWN.cnf
MD5SUM6b69f72e1046222baee8c8d817a4adac
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark485.529
Satisfiable
(Un)Satisfiability was proved
Number of variables500
Number of clauses2130
Sum of the clauses size6390
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 32130
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.32/0.46	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.32/0.46	c INFO: solving HOME/instance-3378592-1305466328.cnf
0.32/0.46	c INFO: computing task features
0.32/0.47	c INFO: computed variable-clause graph statistics
0.32/0.47	c INFO: computed clause balance statistics
0.41/0.51	c INFO: computed variable balance statistics
0.41/0.54	c INFO: computed small-clause counts
0.41/0.54	c INFO: computed Horn-clause counts
0.41/0.55	c INFO: computed variable graph statistics
0.41/0.55	c INFO: collected features for HOME/instance-3378592-1305466328.cnf in 0.08 s
0.41/0.56	c INFO: running gnovelty+2@0 for 97 with 1199 remaining (b = 0.57)
0.41/0.57	c INFO: running march_hi@0 for 388 with 1199 remaining (b = 0.35)
0.41/0.58	c INFO: running gnovelty+2@0 for 97 with 1199 remaining (b = 0.06)
0.41/0.59	c INFO: running march_hi@0 for 388 with 1199 remaining (b = 0.33)
399.91/100.88	c INFO: running march_hi@0 for 679 with 1099 remaining (b = 0.39)
400.31/100.94	c INFO: running march_hi@0 for 679 with 1099 remaining (b = 0.35)
1565.22/393.24	c INFO: running march_hi@100 for 97 with 807 remaining (b = 0.07)
1566.83/393.69	c INFO: running march_hi@0 for 291 with 806 remaining (b = 0.18)
1809.56/454.59	s UNSATISFIABLE

Verifier Data 

No possible verification on an UNSAT instance

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3378592-1305466328/watcher-3378592-1305466328 -o /tmp/evaluation-result-3378592-1305466328/solver-3378592-1305466328 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378592-1305466328.cnf -seed 1479294503 -budget 4800 -cores 4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.19 8.19 8.02 6/449 22778
/proc/meminfo: memFree=21921660/32951124 swapFree=51873460/67111528
[pid=22778] ppid=22776 vsize=16244 CPUtime=0.01 cores=1,3,5,7
/proc/22778/stat : 22778 (python) R 22776 22778 20819 0 -1 4202496 1049 0 0 0 1 0 0 0 17 0 1 0 639049106 16633856 922 33554432000 4194304 5683644 140735637523440 18446744073709551615 212039655168 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/22778/statm: 4061 922 381 364 0 548 0

[startup+0.0759321 s]
/proc/loadavg: 8.19 8.19 8.02 6/449 22778
/proc/meminfo: memFree=21921660/32951124 swapFree=51873460/67111528
[pid=22778] ppid=22776 vsize=84608 CPUtime=0.05 cores=1,3,5,7
/proc/22778/stat : 22778 (python) S 22776 22778 20819 0 -1 4202496 2408 0 0 0 4 1 0 0 17 0 1 0 639049106 86638592 2136 33554432000 4194304 5683644 140735637523440 18446744073709551615 212039653376 0 2147483391 16781312 2 18446744071699199166 0 0 17 7 0 0 0
/proc/22778/statm: 21152 2136 707 364 0 1369 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 84608

[startup+0.100789 s]
/proc/loadavg: 8.19 8.19 8.02 6/449 22778
/proc/meminfo: memFree=21921660/32951124 swapFree=51873460/67111528
[pid=22778] ppid=22776 vsize=104228 CPUtime=0.06 cores=1,3,5,7
/proc/22778/stat : 22778 (python) S 22776 22778 20819 0 -1 4202496 3069 0 0 0 5 1 0 0 17 0 1 0 639049106 106729472 2751 33554432000 4194304 5683644 140735637523440 18446744073709551615 212039653376 0 2147483391 16781312 2 18446744071699199166 0 0 17 7 0 0 0
/proc/22778/statm: 26057 2751 847 364 0 1805 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 104228

[startup+0.300897 s]
/proc/loadavg: 8.19 8.19 8.02 6/449 22778
/proc/meminfo: memFree=21921660/32951124 swapFree=51873460/67111528
[pid=22778] ppid=22776 vsize=187692 CPUtime=0.23 cores=1,3,5,7
/proc/22778/stat : 22778 (python) S 22776 22778 20819 0 -1 4202496 6828 932 1 0 17 6 0 0 17 0 1 0 639049106 192196608 4882 33554432000 4194304 5683644 140735637523440 18446744073709551615 212039653376 0 2147483391 16781312 2 18446744071699199166 0 0 17 7 0 0 0
/proc/22778/statm: 46923 4882 1399 364 0 3724 0
Current children cumulated CPU time (s) 0.23
Current children cumulated vsize (KiB) 187692

[startup+0.700657 s]
/proc/loadavg: 8.19 8.19 8.02 6/449 22778
/proc/meminfo: memFree=21921660/32951124 swapFree=51873460/67111528
[pid=22778] ppid=22776 vsize=320192 CPUtime=0.49 cores=1,3,5,7
/proc/22778/stat : 22778 (python) S 22776 22778 20819 0 -1 4202496 12196 932 1 0 41 8 0 0 25 0 5 0 639049106 327876608 7236 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051482059 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/22778/statm: 80048 7236 1873 364 0 15957 0
Current children cumulated CPU time (s) 0.49
Current children cumulated vsize (KiB) 320192

[startup+1.50051 s]
/proc/loadavg: 8.19 8.19 8.02 9/462 22795
/proc/meminfo: memFree=21882488/32951124 swapFree=51873460/67111528
[pid=22778] ppid=22776 vsize=320192 CPUtime=0.49 cores=1,3,5,7
/proc/22778/stat : 22778 (python) S 22776 22778 20819 0 -1 4202496 12196 932 1 0 41 8 0 0 25 0 5 0 639049106 327876608 7236 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051482059 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/22778/statm: 80048 7236 1873 364 0 15957 0
[pid=22784] ppid=22778 vsize=278504 CPUtime=0 cores=1,3,5,7
/proc/22784/stat : 22784 (python) S 22778 22778 20819 0 -1 4202560 719 0 0 0 0 0 0 0 18 0 1 0 639049162 285188096 5549 33554432000 4194304 5683644 140735637523440 18446744073709551615 212039675407 0 0 16781312 514 197305991509835775 0 0 17 1 0 0 0
/proc/22784/statm: 69626 5549 354 364 0 5544 0
[pid=22786] ppid=22784 vsize=1017800 CPUtime=0.93 cores=1,3,5,7
/proc/22786/stat : 22786 (gnovelty+2) R 22784 22786 22786 0 -1 4202496 445 0 0 0 93 0 0 0 21 0 1 0 639049162 1042227200 177 33554432000 134512640 135663704 4290767600 18446744073709551615 134524882 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/22786/statm: 254450 177 113 282 0 254166 0
[pid=22787] ppid=22778 vsize=289336 CPUtime=0 cores=1,3,5,7
/proc/22787/stat : 22787 (python) S 22778 22778 20819 0 -1 4202560 752 0 0 0 0 0 0 0 15 0 1 0 639049163 296280064 5691 33554432000 4194304 5683644 140735637523440 18446744073709551615 212039675462 0 0 16781312 514 5606935197674078471 0 0 17 5 0 0 0
/proc/22787/statm: 72334 5691 348 364 0 8249 0
[pid=22789] ppid=22787 vsize=2172 CPUtime=0.92 cores=1,3,5,7
/proc/22789/stat : 22789 (march_hi) R 22787 22789 22789 0 -1 4202496 620 0 0 0 92 0 0 0 25 0 1 0 639049163 2224128 356 33554432000 4194304 4955518 140736075248672 18446744073709551615 4238727 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/22789/statm: 543 356 104 186 0 356 0
[pid=22790] ppid=22778 vsize=299084 CPUtime=0 cores=1,3,5,7
/proc/22790/stat : 22790 (python) S 22778 22778 20819 0 -1 4202560 737 0 0 0 0 0 0 0 22 0 1 0 639049164 306262016 5583 33554432000 4194304 5683644 140735637523440 18446744073709551615 212039675462 0 0 16781312 514 11771622454793863167 0 0 17 7 0 0 0
/proc/22790/statm: 74771 5583 348 364 0 10683 0
[pid=22792] ppid=22790 vsize=1017800 CPUtime=0.89 cores=1,3,5,7
/proc/22792/stat : 22792 (gnovelty+2) R 22790 22792 22792 0 -1 4202496 439 0 0 0 89 0 0 0 25 0 1 0 639049164 1042227200 178 33554432000 134512640 135663704 4294894416 18446744073709551615 134545772 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/22792/statm: 254450 178 113 282 0 254166 0
[pid=22793] ppid=22778 vsize=309948 CPUtime=0 cores=1,3,5,7
/proc/22793/stat : 22793 (python) S 22778 22778 20819 0 -1 4202560 753 0 0 0 0 0 0 0 18 0 1 0 639049165 317386752 5707 33554432000 4194304 5683644 140735637523440 18446744073709551615 212039675462 0 0 16781312 514 5606935197674078470 0 0 17 1 0 0 0
/proc/22793/statm: 77487 5707 348 364 0 13396 0
[pid=22795] ppid=22793 vsize=2172 CPUtime=0.9 cores=1,3,5,7
/proc/22795/stat : 22795 (march_hi) R 22793 22795 22795 0 -1 4202496 619 0 0 0 90 0 0 0 25 0 1 0 639049165 2224128 356 33554432000 4194304 4955518 140736246989392 18446744073709551615 4235338 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/22795/statm: 543 356 104 186 0 356 0
[pid=22778/tid=22785] ppid=22776 vsize=320192 CPUtime=0 cores=1,3,5,7
/proc/22778/task/22785/stat : 22785 (python) S 22776 22778 20819 0 -1 4202560 24 932 0 0 0 0 0 0 19 0 5 0 639049162 327876608 7236 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 2 18446604455653488576 0 0 -1 1 0 0 0
[pid=22778/tid=22788] ppid=22776 vsize=320192 CPUtime=0 cores=1,3,5,7
/proc/22778/task/22788/stat : 22788 (python) S 22776 22778 20819 0 -1 4202560 25 932 0 0 0 0 0 0 21 0 5 0 639049163 327876608 7236 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 2 18446604455653488576 0 0 -1 5 0 0 0
[pid=22778/tid=22791] ppid=22776 vsize=320192 CPUtime=0 cores=1,3,5,7
/proc/22778/task/22791/stat : 22791 (python) S 22776 22778 20819 0 -1 4202560 35 932 0 0 0 0 0 0 23 0 5 0 639049164 327876608 7236 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 2 18446604455653488576 0 0 -1 1 0 0 0
[pid=22778/tid=22794] ppid=22776 vsize=320192 CPUtime=0 cores=1,3,5,7
/proc/22778/task/22794/stat : 22794 (python) S 22776 22778 20819 0 -1 4202560 27 932 0 0 0 0 0 0 25 0 5 0 639049165 327876608 7236 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 2 18446604455653488576 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 4.13
Current children cumulated vsize (KiB) 3537008

[startup+3.10123 s]
/proc/loadavg: 8.17 8.19 8.02 9/462 22795
/proc/meminfo: memFree=21881000/32951124 swapFree=51873460/67111528
[pid=22778] ppid=22776 vsize=320192 CPUtime=0.49 cores=1,3,5,7
/proc/22778/stat : 22778 (python) S 22776 22778 20819 0 -1 4202496 12196 932 1 0 41 8 0 0 25 0 5 0 639049106 327876608 7236 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051482059 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/22778/statm: 80048 7236 1873 364 0 15957 0
[pid=22784] ppid=22778 vsize=278504 CPUtime=0 cores=1,3,5,7
/proc/22784/stat : 22784 (python) S 22778 22778 20819 0 -1 4202560 719 0 0 0 0 0 0 0 18 0 1 0 639049162 285188096 5549 33554432000 4194304 5683644 140735637523440 18446744073709551615 212039675407 0 0 16781312 514 197305991509803268 0 0 17 1 0 0 0
/proc/22784/statm: 69626 5549 354 364 0 5544 0
[pid=22786] ppid=22784 vsize=1017800 CPUtime=2.53 cores=1,3,5,7
/proc/22786/stat : 22786 (gnovelty+2) R 22784 22786 22786 0 -1 4202496 445 0 0 0 253 0 0 0 25 0 1 0 639049162 1042227200 177 33554432000 134512640 135663704 4290767600 18446744073709551615 134524734 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/22786/statm: 254450 177 113 282 0 254166 0
[pid=22787] ppid=22778 vsize=289336 CPUtime=0 cores=1,3,5,7
/proc/22787/stat : 22787 (python) S 22778 22778 20819 0 -1 4202560 752 0 0 0 0 0 0 0 15 0 1 0 639049163 296280064 5691 33554432000 4194304 5683644 140735637523440 18446744073709551615 212039675462 0 0 16781312 514 909150984930820359 0 0 17 5 0 0 0
/proc/22787/statm: 72334 5691 348 364 0 8249 0
[pid=22789] ppid=22787 vsize=2304 CPUtime=2.52 cores=1,3,5,7
/proc/22789/stat : 22789 (march_hi) R 22787 22789 22789 0 -1 4202496 635 0 0 0 252 0 0 0 25 0 1 0 639049163 2359296 371 33554432000 4194304 4955518 140736075248672 18446744073709551615 4227565 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/22789/statm: 576 371 104 186 0 389 0
[pid=22790] ppid=22778 vsize=299084 CPUtime=0 cores=1,3,5,7
/proc/22790/stat : 22790 (python) S 22778 22778 20819 0 -1 4202560 737 0 0 0 0 0 0 0 18 0 1 0 639049164 306262016 5583 33554432000 4194304 5683644 140735637523440 18446744073709551615 212039675462 0 0 16781312 514 11771622454793830662 0 0 17 7 0 0 0
/proc/22790/statm: 74771 5583 348 364 0 10683 0
[pid=22792] ppid=22790 vsize=1017800 CPUtime=2.48 cores=1,3,5,7
/proc/22792/stat : 22792 (gnovelty+2) R 22790 22792 22792 0 -1 4202496 439 0 0 0 248 0 0 0 25 0 1 0 639049164 1042227200 178 33554432000 134512640 135663704 4294894416 18446744073709551615 134546746 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/22792/statm: 254450 178 113 282 0 254166 0
[pid=22793] ppid=22778 vsize=309948 CPUtime=0 cores=1,3,5,7
/proc/22793/stat : 22793 (python) S 22778 22778 20819 0 -1 4202560 755 0 0 0 0 0 0 0 15 0 1 0 639049165 317386752 5707 33554432000 4194304 5683644 140735637523440 18446744073709551615 212039675462 0 0 16781312 514 18373683729362026758 0 0 17 1 0 0 0
/proc/22793/statm: 77487 5707 348 364 0 13396 0
[pid=22795] ppid=22793 vsize=2304 CPUtime=2.5 cores=1,3,5,7
/proc/22795/stat : 22795 (march_hi) R 22793 22795 22795 0 -1 4202496 634 0 0 0 250 0 0 0 25 0 1 0 639049165 2359296 371 33554432000 4194304 4955518 140736246989392 18446744073709551615 4235304 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/22795/statm: 576 371 104 186 0 389 0
[pid=22778/tid=22785] ppid=22776 vsize=320192 CPUtime=0 cores=1,3,5,7
/proc/22778/task/22785/stat : 22785 (python) S 22776 22778 20819 0 -1 4202560 24 932 0 0 0 0 0 0 19 0 5 0 639049162 327876608 7236 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 2 18446604455653488576 0 0 -1 1 0 0 0
[pid=22778/tid=22788] ppid=22776 vsize=320192 CPUtime=0 cores=1,3,5,7
/proc/22778/task/22788/stat : 22788 (python) S 22776 22778 20819 0 -1 4202560 25 932 0 0 0 0 0 0 21 0 5 0 639049163 327876608 7236 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 2 18446604455653488576 0 0 -1 5 0 0 0
[pid=22778/tid=22791] ppid=22776 vsize=320192 CPUtime=0 cores=1,3,5,7
/proc/22778/task/22791/stat : 22791 (python) S 22776 22778 20819 0 -1 4202560 35 932 0 0 0 0 0 0 23 0 5 0 639049164 327876608 7236 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 2 18446604455653488576 0 0 -1 1 0 0 0
[pid=22778/tid=22794] ppid=22776 vsize=320192 CPUtime=0 cores=1,3,5,7
/proc/22778/task/22794/stat : 22794 (python) S 22776 22778 20819 0 -1 4202560 27 932 0 0 0 0 0 0 25 0 5 0 639049165 327876608 7236 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 2 18446604455653488576 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 10.52
Current children cumulated vsize (KiB) 3537272

[startup+6.30066 s]
/proc/loadavg: 8.17 8.19 8.02 10/462 22795
/proc/meminfo: memFree=21878760/32951124 swapFree=51873460/67111528
[pid=22778] ppid=22776 vsize=320192 CPUtime=0.49 cores=1,3,5,7
/proc/22778/stat : 22778 (python) S 22776 22778 20819 0 -1 4202496 12196 932 1 0 41 8 0 0 25 0 5 0 639049106 327876608 7236 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051482059 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/22778/statm: 80048 7236 1873 364 0 15957 0
[pid=22784] ppid=22778 vsize=278504 CPUtime=0.01 cores=1,3,5,7
/proc/22784/stat : 22784 (python) S 22778 22778 20819 0 -1 4202560 879 0 0 0 0 1 0 0 18 0 1 0 639049162 285188096 5572 33554432000 4194304 5683644 140735637523440 18446744073709551615 212039675407 0 0 16781312 514 197305991509803268 0 0 17 1 0 0 0
/proc/22784/statm: 69626 5572 377 364 0 5544 0
[pid=22786] ppid=22784 vsize=1017800 CPUtime=5.73 cores=1,3,5,7
/proc/22786/stat : 22786 (gnovelty+2) R 22784 22786 22786 0 -1 4202496 445 0 0 0 573 0 0 0 25 0 1 0 639049162 1042227200 177 33554432000 134512640 135663704 4290767600 18446744073709551615 134525448 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/22786/statm: 254450 177 113 282 0 254166 0
[pid=22787] ppid=22778 vsize=289336 CPUtime=0.01 cores=1,3,5,7
/proc/22787/stat : 22787 (python) S 22778 22778 20819 0 -1 4202560 923 0 0 0 1 0 0 0 15 0 1 0 639049163 296280064 5714 33554432000 4194304 5683644 140735637523440 18446744073709551615 212039675462 0 0 16781312 514 909150984930820359 0 0 17 5 0 0 0
/proc/22787/statm: 72334 5714 371 364 0 8249 0
[pid=22789] ppid=22787 vsize=2304 CPUtime=5.7 cores=1,3,5,7
/proc/22789/stat : 22789 (march_hi) R 22787 22789 22789 0 -1 4202496 649 0 0 0 570 0 0 0 25 0 1 0 639049163 2359296 385 33554432000 4194304 4955518 140736075248672 18446744073709551615 4209733 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/22789/statm: 576 385 104 186 0 389 0
[pid=22790] ppid=22778 vsize=299084 CPUtime=0.01 cores=1,3,5,7
/proc/22790/stat : 22790 (python) S 22778 22778 20819 0 -1 4202560 902 0 0 0 0 1 0 0 18 0 1 0 639049164 306262016 5606 33554432000 4194304 5683644 140735637523440 18446744073709551615 212039675462 0 0 16781312 514 11771622454793830662 0 0 17 7 0 0 0
/proc/22790/statm: 74771 5606 371 364 0 10683 0
[pid=22792] ppid=22790 vsize=1017800 CPUtime=5.61 cores=1,3,5,7
/proc/22792/stat : 22792 (gnovelty+2) R 22790 22792 22792 0 -1 4202496 439 0 0 0 561 0 0 0 25 0 1 0 639049164 1042227200 178 33554432000 134512640 135663704 4294894416 18446744073709551615 134525271 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/22792/statm: 254450 178 113 282 0 254166 0
[pid=22793] ppid=22778 vsize=309948 CPUtime=0.01 cores=1,3,5,7
/proc/22793/stat : 22793 (python) S 22778 22778 20819 0 -1 4202560 915 0 0 0 0 1 0 0 15 0 1 0 639049165 317386752 5730 33554432000 4194304 5683644 140735637523440 18446744073709551615 212039675462 0 0 16781312 514 3079239767364174086 0 0 17 1 0 0 0
/proc/22793/statm: 77487 5730 371 364 0 13396 0
[pid=22795] ppid=22793 vsize=2304 CPUtime=5.66 cores=1,3,5,7
/proc/22795/stat : 22795 (march_hi) R 22793 22795 22795 0 -1 4202496 648 0 0 0 566 0 0 0 25 0 1 0 639049165 2359296 385 33554432000 4194304 4955518 140736246989392 18446744073709551615 4209727 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/22795/statm: 576 385 104 186 0 389 0
[pid=22778/tid=22785] ppid=22776 vsize=320192 CPUtime=0 cores=1,3,5,7
/proc/22778/task/22785/stat : 22785 (python) S 22776 22778 20819 0 -1 4202560 24 932 0 0 0 0 0 0 19 0 5 0 639049162 327876608 7236 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 2 18446604455653488576 0 0 -1 1 0 0 0
[pid=22778/tid=22788] ppid=22776 vsize=320192 CPUtime=0 cores=1,3,5,7
/proc/22778/task/22788/stat : 22788 (python) S 22776 22778 20819 0 -1 4202560 25 932 0 0 0 0 0 0 21 0 5 0 639049163 327876608 7236 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 2 18446604455653488576 0 0 -1 5 0 0 0
[pid=22778/tid=22791] ppid=22776 vsize=320192 CPUtime=0 cores=1,3,5,7
/proc/22778/task/22791/stat : 22791 (python) S 22776 22778 20819 0 -1 4202560 35 932 0 0 0 0 0 0 23 0 5 0 639049164 327876608 7236 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 2 18446604455653488576 0 0 -1 1 0 0 0
[pid=22778/tid=22794] ppid=22776 vsize=320192 CPUtime=0 cores=1,3,5,7
/proc/22778/task/22794/stat : 22794 (python) S 22776 22778 20819 0 -1 4202560 27 932 0 0 0 0 0 0 25 0 5 0 639049165 327876608 7236 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 2 18446604455653488576 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 23.23
Current children cumulated vsize (KiB) 3537272

[startup+12.7005 s]
/proc/loadavg: 8.14 8.18 8.02 9/462 22795
/proc/meminfo: memFree=21877036/32951124 swapFree=51873460/67111528
[pid=22778] ppid=22776 vsize=320192 CPUtime=0.49 cores=1,3,5,7
/proc/22778/stat : 22778 (python) S 22776 22778 20819 0 -1 4202496 12196 932 1 0 41 8 0 0 25 0 5 0 639049106 327876608 7236 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051482059 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/22778/statm: 80048 7236 1873 364 0 15957 0
[pid=22784] ppid=22778 vsize=278504 CPUtime=0.02 cores=1,3,5,7
/proc/22784/stat : 22784 (python) S 22778 22778 20819 0 -1 4202560 882 0 0 0 1 1 0 0 18 0 1 0 639049162 285188096 5572 33554432000 4194304 5683644 140735637523440 18446744073709551615 212039675407 0 0 16781312 514 14066272156463890692 0 0 17 1 0 0 0
/proc/22784/statm: 69626 5572 377 364 0 5544 0
[pid=22786] ppid=22784 vsize=1017800 CPUtime=12.13 cores=1,3,5,7
/proc/22786/stat : 22786 (gnovelty+2) R 22784 22786 22786 0 -1 4202496 445 0 0 0 1213 0 0 0 25 0 1 0 639049162 1042227200 177 33554432000 134512640 135663704 4290767600 18446744073709551615 134525520 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/22786/statm: 254450 177 113 282 0 254166 0
[pid=22787] ppid=22778 vsize=289336 CPUtime=0.03 cores=1,3,5,7

################
# More data... #
################

[pid=22795] ppid=22793 vsize=0 CPUtime=448.55 cores=1,3,5,7
/proc/22795/stat : 22795 (march_hi) R 22793 22795 22795 0 -1 4202500 2948 0 0 0 44854 1 0 0 25 0 1 0 639049165 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 5 0 0 0
/proc/22795/statm: 0 0 0 0 0 0 0
[pid=22793/tid=22957] ppid=22778 vsize=309948 CPUtime=0 cores=1,3,5,7
/proc/22793/task/22957/stat : 22957 (python) S 22778 22778 20819 0 -1 4202560 46 0 0 0 0 0 0 0 16 0 2 0 639088429 317386752 5761 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 514 18446604462219569216 0 0 -1 1 0 0 0
[pid=22889] ppid=22778 vsize=320204 CPUtime=1.29 cores=1,3,5,7
/proc/22889/stat : 22889 (python) S 22778 22778 20819 0 -1 4202560 1019 0 0 0 67 62 0 0 15 0 1 0 639059194 327888896 5743 33554432000 4194304 5683644 140735637523440 18446744073709551615 212039675462 0 0 16781312 514 8236573549360283911 0 0 17 7 0 0 0
/proc/22889/statm: 80051 5743 380 364 0 15957 0
[pid=22891] ppid=22889 vsize=8280 CPUtime=346.24 cores=1,3,5,7
/proc/22891/stat : 22891 (march_hi) R 22889 22891 22891 0 -1 4202496 1897 0 0 0 34624 0 0 0 25 0 1 0 639059194 8478720 1638 33554432000 4194304 4955518 140735923062112 18446744073709551615 4261351 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/22891/statm: 2070 1638 104 186 0 1883 0
[pid=22893] ppid=22778 vsize=330092 CPUtime=1.29 cores=1,3,5,7
/proc/22893/stat : 22893 (python) S 22778 22778 20819 0 -1 4202560 1013 0 0 0 65 64 0 0 15 0 1 0 639059200 338014208 5667 33554432000 4194304 5683644 140735637523440 18446744073709551615 212039675462 0 0 16781312 514 11477815080788132103 0 0 17 1 0 0 0
/proc/22893/statm: 82523 5667 371 364 0 18426 0
[pid=22895] ppid=22893 vsize=8280 CPUtime=352.54 cores=1,3,5,7
/proc/22895/stat : 22895 (march_hi) R 22893 22895 22895 0 -1 4202496 1975 0 0 0 35254 0 0 0 25 0 1 0 639059201 8478720 1713 33554432000 4194304 4955518 140733935379168 18446744073709551615 4227917 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/22895/statm: 2070 1713 104 186 0 1883 0
[pid=22778/tid=22785] ppid=22776 vsize=340960 CPUtime=0 cores=1,3,5,7
/proc/22778/task/22785/stat : 22785 (python) S 22776 22778 20819 0 -1 4202560 24 932 0 0 0 0 0 0 19 0 7 0 639049162 349143040 7321 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 2 18446604455653488576 0 0 -1 1 0 0 0
[pid=22778/tid=22788] ppid=22776 vsize=340960 CPUtime=0 cores=1,3,5,7
/proc/22778/task/22788/stat : 22788 (python) S 22776 22778 20819 0 -1 4202560 30 932 0 0 0 0 0 0 18 0 7 0 639049163 349143040 7321 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 2 18446604455653488576 0 0 -1 5 0 0 0
[pid=22778/tid=22791] ppid=22776 vsize=340960 CPUtime=0 cores=1,3,5,7
/proc/22778/task/22791/stat : 22791 (python) S 22776 22778 20819 0 -1 4202560 35 932 0 0 0 0 0 0 23 0 7 0 639049164 349143040 7321 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 2 18446604455653488576 0 0 -1 1 0 0 0
[pid=22778/tid=22794] ppid=22776 vsize=340960 CPUtime=0 cores=1,3,5,7
/proc/22778/task/22794/stat : 22794 (python) S 22776 22778 20819 0 -1 4202560 33 932 0 0 0 0 0 0 18 0 7 0 639049165 349143040 7321 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 2 18446604455653488576 0 0 -1 1 0 0 0
[pid=22778/tid=22890] ppid=22776 vsize=340960 CPUtime=0 cores=1,3,5,7
/proc/22778/task/22890/stat : 22890 (python) S 22776 22778 20819 0 -1 4202560 26 932 0 0 0 0 0 0 20 0 7 0 639059194 349143040 7321 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 2 18446604455653488576 0 0 -1 1 0 0 0
[pid=22778/tid=22894] ppid=22776 vsize=340960 CPUtime=0 cores=1,3,5,7
/proc/22778/task/22894/stat : 22894 (python) S 22776 22778 20819 0 -1 4202560 28 932 0 0 0 0 0 0 21 0 7 0 639059201 349143040 7321 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 2 18446604455653488576 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1805.57
Current children cumulated vsize (KiB) 4230532

[startup+454.317 s]
/proc/loadavg: 8.33 8.27 8.12 12/480 22967
/proc/meminfo: memFree=21841976/32951124 swapFree=51873460/67111528
[pid=22778] ppid=22776 vsize=340960 CPUtime=0.53 cores=1,3,5,7
/proc/22778/stat : 22778 (python) S 22776 22778 20819 0 -1 4202496 14437 932 1 0 44 9 0 0 18 0 7 0 639049106 349143040 7321 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051482059 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/22778/statm: 85240 7321 1886 364 0 21143 0
[pid=22784] ppid=22778 vsize=288748 CPUtime=0.32 cores=1,3,5,7
/proc/22784/stat : 22784 (python) S 22778 22778 20819 0 -1 4202560 1157 0 0 0 16 16 0 0 19 0 2 0 639049162 295677952 5616 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051482059 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/22784/statm: 72187 5616 416 364 0 8105 0
[pid=22786] ppid=22784 vsize=1017800 CPUtime=100.3 cores=1,3,5,7
/proc/22786/stat : 22786 (gnovelty+2) T 22784 22786 22786 0 -1 4202496 445 0 0 0 10030 0 0 0 25 0 1 0 639049162 1042227200 177 33554432000 134512640 135663704 4290767600 18446744073709551615 134544183 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/22786/statm: 254450 177 113 282 0 254166 0
[pid=22784/tid=22888] ppid=22778 vsize=288748 CPUtime=0 cores=1,3,5,7
/proc/22784/task/22888/stat : 22888 (python) S 22778 22778 20819 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 639059193 295677952 5616 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 514 18446604470138501312 0 0 -1 1 0 0 0
[pid=22787] ppid=22778 vsize=289336 CPUtime=2.89 cores=1,3,5,7
/proc/22787/stat : 22787 (python) R 22778 22778 20819 0 -1 4202560 1173 0 0 0 194 95 0 0 17 0 2 0 639049163 296280064 5745 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051485503 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/22787/statm: 72334 5745 401 364 0 8249 0
[pid=22789] ppid=22787 vsize=0 CPUtime=450.67 cores=1,3,5,7
/proc/22789/stat : 22789 (march_hi) R 22787 22789 22789 0 -1 4202500 2949 0 0 0 45066 1 0 0 25 0 1 0 639049163 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 1 0 0 0
/proc/22789/statm: 0 0 0 0 0 0 0
[pid=22787/tid=22958] ppid=22778 vsize=289336 CPUtime=0 cores=1,3,5,7
/proc/22787/task/22958/stat : 22958 (python) S 22778 22778 20819 0 -1 4202560 44 0 0 0 0 0 0 0 16 0 2 0 639088474 296280064 5745 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 514 18446604464205956032 0 0 -1 5 0 0 0
[pid=22790] ppid=22778 vsize=299084 CPUtime=0.32 cores=1,3,5,7
/proc/22790/stat : 22790 (python) S 22778 22778 20819 0 -1 4202560 1091 0 0 0 17 15 0 0 19 0 2 0 639049164 306262016 5636 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051482059 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/22790/statm: 74771 5636 400 364 0 10683 0
[pid=22792] ppid=22790 vsize=1017800 CPUtime=98.65 cores=1,3,5,7
/proc/22792/stat : 22792 (gnovelty+2) T 22790 22792 22792 0 -1 4202496 439 0 0 0 9865 0 0 0 25 0 1 0 639049164 1042227200 178 33554432000 134512640 135663704 4294894416 18446744073709551615 134525255 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/22792/statm: 254450 178 113 282 0 254166 0
[pid=22790/tid=22892] ppid=22778 vsize=299084 CPUtime=0 cores=1,3,5,7
/proc/22790/task/22892/stat : 22892 (python) S 22778 22778 20819 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 639059200 306262016 5636 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 514 18446604455653487680 0 0 -1 7 0 0 0
[pid=22793] ppid=22778 vsize=309948 CPUtime=3.61 cores=1,3,5,7
/proc/22793/stat : 22793 (python) R 22778 22778 20819 0 -1 4202560 1181 0 0 0 258 103 0 0 18 0 2 0 639049165 317386752 5761 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051485503 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/22793/statm: 77487 5761 401 364 0 13396 0
[pid=22795] ppid=22793 vsize=0 CPUtime=448.55 cores=1,3,5,7
/proc/22795/stat : 22795 (march_hi) R 22793 22795 22795 0 -1 4202500 2948 0 0 0 44854 1 0 0 25 0 1 0 639049165 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 5 0 0 0
/proc/22795/statm: 0 0 0 0 0 0 0
[pid=22793/tid=22957] ppid=22778 vsize=309948 CPUtime=0 cores=1,3,5,7
/proc/22793/task/22957/stat : 22957 (python) S 22778 22778 20819 0 -1 4202560 46 0 0 0 0 0 0 0 16 0 2 0 639088429 317386752 5761 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 514 18446604462219569216 0 0 -1 1 0 0 0
[pid=22889] ppid=22778 vsize=320204 CPUtime=1.29 cores=1,3,5,7
/proc/22889/stat : 22889 (python) S 22778 22778 20819 0 -1 4202560 1019 0 0 0 67 62 0 0 15 0 1 0 639059194 327888896 5743 33554432000 4194304 5683644 140735637523440 18446744073709551615 212039675462 0 0 16781312 514 8236573549360283911 0 0 17 7 0 0 0
/proc/22889/statm: 80051 5743 380 364 0 15957 0
[pid=22891] ppid=22889 vsize=8280 CPUtime=347.04 cores=1,3,5,7
/proc/22891/stat : 22891 (march_hi) R 22889 22891 22891 0 -1 4202496 1902 0 0 0 34704 0 0 0 25 0 1 0 639059194 8478720 1643 33554432000 4194304 4955518 140735923062112 18446744073709551615 4224697 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/22891/statm: 2070 1643 104 186 0 1883 0
[pid=22893] ppid=22778 vsize=330092 CPUtime=1.29 cores=1,3,5,7
/proc/22893/stat : 22893 (python) S 22778 22778 20819 0 -1 4202560 1013 0 0 0 65 64 0 0 15 0 1 0 639059200 338014208 5667 33554432000 4194304 5683644 140735637523440 18446744073709551615 212039675462 0 0 16781312 514 11477815080788132103 0 0 17 1 0 0 0
/proc/22893/statm: 82523 5667 371 364 0 18426 0
[pid=22895] ppid=22893 vsize=8280 CPUtime=353.36 cores=1,3,5,7
/proc/22895/stat : 22895 (march_hi) R 22893 22895 22895 0 -1 4202496 1978 0 0 0 35336 0 0 0 25 0 1 0 639059201 8478720 1716 33554432000 4194304 4955518 140733935379168 18446744073709551615 4227947 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/22895/statm: 2070 1716 104 186 0 1883 0
[pid=22778/tid=22785] ppid=22776 vsize=340960 CPUtime=0 cores=1,3,5,7
/proc/22778/task/22785/stat : 22785 (python) S 22776 22778 20819 0 -1 4202560 24 932 0 0 0 0 0 0 19 0 7 0 639049162 349143040 7321 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 2 18446604455653488576 0 0 -1 1 0 0 0
[pid=22778/tid=22788] ppid=22776 vsize=340960 CPUtime=0 cores=1,3,5,7
/proc/22778/task/22788/stat : 22788 (python) S 22776 22778 20819 0 -1 4202560 30 932 0 0 0 0 0 0 18 0 7 0 639049163 349143040 7321 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 2 18446604455653488576 0 0 -1 5 0 0 0
[pid=22778/tid=22791] ppid=22776 vsize=340960 CPUtime=0 cores=1,3,5,7
/proc/22778/task/22791/stat : 22791 (python) S 22776 22778 20819 0 -1 4202560 35 932 0 0 0 0 0 0 23 0 7 0 639049164 349143040 7321 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 2 18446604455653488576 0 0 -1 1 0 0 0
[pid=22778/tid=22794] ppid=22776 vsize=340960 CPUtime=0 cores=1,3,5,7
/proc/22778/task/22794/stat : 22794 (python) S 22776 22778 20819 0 -1 4202560 33 932 0 0 0 0 0 0 18 0 7 0 639049165 349143040 7321 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 2 18446604455653488576 0 0 -1 1 0 0 0
[pid=22778/tid=22890] ppid=22776 vsize=340960 CPUtime=0 cores=1,3,5,7
/proc/22778/task/22890/stat : 22890 (python) S 22776 22778 20819 0 -1 4202560 26 932 0 0 0 0 0 0 20 0 7 0 639059194 349143040 7321 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 2 18446604455653488576 0 0 -1 1 0 0 0
[pid=22778/tid=22894] ppid=22776 vsize=340960 CPUtime=0 cores=1,3,5,7
/proc/22778/task/22894/stat : 22894 (python) S 22776 22778 20819 0 -1 4202560 28 932 0 0 0 0 0 0 21 0 7 0 639059201 349143040 7321 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 2 18446604455653488576 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1808.82
Current children cumulated vsize (KiB) 4230532

[startup+454.502 s]
/proc/loadavg: 8.33 8.27 8.12 12/480 22967
/proc/meminfo: memFree=21841976/32951124 swapFree=51873460/67111528
[pid=22778] ppid=22776 vsize=340960 CPUtime=0.53 cores=1,3,5,7
/proc/22778/stat : 22778 (python) S 22776 22778 20819 0 -1 4202496 14437 932 1 0 44 9 0 0 18 0 7 0 639049106 349143040 7321 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051482059 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/22778/statm: 85240 7321 1886 364 0 21143 0
[pid=22784] ppid=22778 vsize=288748 CPUtime=0.32 cores=1,3,5,7
/proc/22784/stat : 22784 (python) S 22778 22778 20819 0 -1 4202560 1157 0 0 0 16 16 0 0 19 0 2 0 639049162 295677952 5616 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051482059 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/22784/statm: 72187 5616 416 364 0 8105 0
[pid=22786] ppid=22784 vsize=1017800 CPUtime=100.3 cores=1,3,5,7
/proc/22786/stat : 22786 (gnovelty+2) T 22784 22786 22786 0 -1 4202496 445 0 0 0 10030 0 0 0 25 0 1 0 639049162 1042227200 177 33554432000 134512640 135663704 4290767600 18446744073709551615 134544183 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/22786/statm: 254450 177 113 282 0 254166 0
[pid=22784/tid=22888] ppid=22778 vsize=288748 CPUtime=0 cores=1,3,5,7
/proc/22784/task/22888/stat : 22888 (python) S 22778 22778 20819 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 639059193 295677952 5616 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 514 18446604470138501312 0 0 -1 1 0 0 0
[pid=22787] ppid=22778 vsize=289336 CPUtime=3.08 cores=1,3,5,7
/proc/22787/stat : 22787 (python) R 22778 22778 20819 0 -1 4202560 1173 0 0 0 211 97 0 0 17 0 2 0 639049163 296280064 5745 33554432000 4194304 5683644 140735637523440 18446744073709551615 212039675462 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/22787/statm: 72334 5745 401 364 0 8249 0
[pid=22789] ppid=22787 vsize=0 CPUtime=450.67 cores=1,3,5,7
/proc/22789/stat : 22789 (march_hi) R 22787 22789 22789 0 -1 4202500 2949 0 0 0 45066 1 0 0 25 0 1 0 639049163 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 1 0 0 0
/proc/22789/statm: 0 0 0 0 0 0 0
[pid=22787/tid=22958] ppid=22778 vsize=289336 CPUtime=0 cores=1,3,5,7
/proc/22787/task/22958/stat : 22958 (python) S 22778 22778 20819 0 -1 4202560 44 0 0 0 0 0 0 0 16 0 2 0 639088474 296280064 5745 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 514 18446604464205956032 0 0 -1 5 0 0 0
[pid=22790] ppid=22778 vsize=299084 CPUtime=0.32 cores=1,3,5,7
/proc/22790/stat : 22790 (python) S 22778 22778 20819 0 -1 4202560 1091 0 0 0 17 15 0 0 19 0 2 0 639049164 306262016 5636 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051482059 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/22790/statm: 74771 5636 400 364 0 10683 0
[pid=22792] ppid=22790 vsize=1017800 CPUtime=98.65 cores=1,3,5,7
/proc/22792/stat : 22792 (gnovelty+2) T 22790 22792 22792 0 -1 4202496 439 0 0 0 9865 0 0 0 25 0 1 0 639049164 1042227200 178 33554432000 134512640 135663704 4294894416 18446744073709551615 134525255 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/22792/statm: 254450 178 113 282 0 254166 0
[pid=22790/tid=22892] ppid=22778 vsize=299084 CPUtime=0 cores=1,3,5,7
/proc/22790/task/22892/stat : 22892 (python) S 22778 22778 20819 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 639059200 306262016 5636 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 514 18446604455653487680 0 0 -1 7 0 0 0
[pid=22793] ppid=22778 vsize=309948 CPUtime=3.79 cores=1,3,5,7
/proc/22793/stat : 22793 (python) R 22778 22778 20819 0 -1 4202560 1181 0 0 0 275 104 0 0 18 0 2 0 639049165 317386752 5761 33554432000 4194304 5683644 140735637523440 18446744073709551615 212039675462 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/22793/statm: 77487 5761 401 364 0 13396 0
[pid=22795] ppid=22793 vsize=0 CPUtime=448.55 cores=1,3,5,7
/proc/22795/stat : 22795 (march_hi) R 22793 22795 22795 0 -1 4202500 2948 0 0 0 44854 1 0 0 25 0 1 0 639049165 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 5 0 0 0
/proc/22795/statm: 0 0 0 0 0 0 0
[pid=22793/tid=22957] ppid=22778 vsize=309948 CPUtime=0 cores=1,3,5,7
/proc/22793/task/22957/stat : 22957 (python) S 22778 22778 20819 0 -1 4202560 46 0 0 0 0 0 0 0 16 0 2 0 639088429 317386752 5761 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 514 18446604462219569216 0 0 -1 1 0 0 0
[pid=22889] ppid=22778 vsize=320204 CPUtime=1.29 cores=1,3,5,7
/proc/22889/stat : 22889 (python) S 22778 22778 20819 0 -1 4202560 1019 0 0 0 67 62 0 0 15 0 1 0 639059194 327888896 5743 33554432000 4194304 5683644 140735637523440 18446744073709551615 212039675462 0 0 16781312 514 8236573549360283911 0 0 17 7 0 0 0
/proc/22889/statm: 80051 5743 380 364 0 15957 0
[pid=22891] ppid=22889 vsize=8280 CPUtime=347.23 cores=1,3,5,7
/proc/22891/stat : 22891 (march_hi) R 22889 22891 22891 0 -1 4202496 1903 0 0 0 34723 0 0 0 25 0 1 0 639059194 8478720 1644 33554432000 4194304 4955518 140735923062112 18446744073709551615 4227651 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/22891/statm: 2070 1644 104 186 0 1883 0
[pid=22893] ppid=22778 vsize=330092 CPUtime=1.29 cores=1,3,5,7
/proc/22893/stat : 22893 (python) S 22778 22778 20819 0 -1 4202560 1013 0 0 0 65 64 0 0 15 0 1 0 639059200 338014208 5667 33554432000 4194304 5683644 140735637523440 18446744073709551615 212039675462 0 0 16781312 514 11477815080788132103 0 0 17 1 0 0 0
/proc/22893/statm: 82523 5667 371 364 0 18426 0
[pid=22895] ppid=22893 vsize=8280 CPUtime=353.54 cores=1,3,5,7
/proc/22895/stat : 22895 (march_hi) R 22893 22895 22895 0 -1 4202496 1978 0 0 0 35354 0 0 0 25 0 1 0 639059201 8478720 1716 33554432000 4194304 4955518 140733935379168 18446744073709551615 4238696 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/22895/statm: 2070 1716 104 186 0 1883 0
[pid=22778/tid=22785] ppid=22776 vsize=340960 CPUtime=0 cores=1,3,5,7
/proc/22778/task/22785/stat : 22785 (python) S 22776 22778 20819 0 -1 4202560 24 932 0 0 0 0 0 0 19 0 7 0 639049162 349143040 7321 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 2 18446604455653488576 0 0 -1 1 0 0 0
[pid=22778/tid=22788] ppid=22776 vsize=340960 CPUtime=0 cores=1,3,5,7
/proc/22778/task/22788/stat : 22788 (python) S 22776 22778 20819 0 -1 4202560 30 932 0 0 0 0 0 0 18 0 7 0 639049163 349143040 7321 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 2 18446604455653488576 0 0 -1 5 0 0 0
[pid=22778/tid=22791] ppid=22776 vsize=340960 CPUtime=0 cores=1,3,5,7
/proc/22778/task/22791/stat : 22791 (python) S 22776 22778 20819 0 -1 4202560 35 932 0 0 0 0 0 0 23 0 7 0 639049164 349143040 7321 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 2 18446604455653488576 0 0 -1 1 0 0 0
[pid=22778/tid=22794] ppid=22776 vsize=340960 CPUtime=0 cores=1,3,5,7
/proc/22778/task/22794/stat : 22794 (python) S 22776 22778 20819 0 -1 4202560 33 932 0 0 0 0 0 0 18 0 7 0 639049165 349143040 7321 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 2 18446604455653488576 0 0 -1 1 0 0 0
[pid=22778/tid=22890] ppid=22776 vsize=340960 CPUtime=0 cores=1,3,5,7
/proc/22778/task/22890/stat : 22890 (python) S 22776 22778 20819 0 -1 4202560 26 932 0 0 0 0 0 0 20 0 7 0 639059194 349143040 7321 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 2 18446604455653488576 0 0 -1 1 0 0 0
[pid=22778/tid=22894] ppid=22776 vsize=340960 CPUtime=0 cores=1,3,5,7
/proc/22778/task/22894/stat : 22894 (python) S 22776 22778 20819 0 -1 4202560 28 932 0 0 0 0 0 0 21 0 7 0 639059201 349143040 7321 33554432000 4194304 5683644 140735637523440 18446744073709551615 212051478961 0 0 16781312 2 18446604455653488576 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1809.56
Current children cumulated vsize (KiB) 4230532

[startup+454.602 s]
/proc/loadavg: 8.33 8.27 8.12 12/480 22967
/proc/meminfo: memFree=21841976/32951124 swapFree=51873460/67111528
[pid=22778] ppid=22776 vsize=310144 CPUtime=1810.04 cores=1,3,5,7
/proc/22778/stat : 22778 (python) R 22776 22778 20819 0 -1 4202496 16307 18511 1 0 45 9 180581 369 17 0 1 0 639049106 317587456 7296 33554432000 4194304 5683644 140735637523440 18446744073709551615 5114087 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/22778/statm: 77536 7296 1873 364 0 13460 0
Current children cumulated CPU time (s) 1810.04
Current children cumulated vsize (KiB) 310144

Child status: 0
Real time (s): 454.616
CPU time (s): 1810.08
CPU user time (s): 1806.28
CPU system time (s): 3.79442
CPU usage (%): 398.155
Max. virtual memory (cumulated for all children) (KiB): 4260956

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1806.28
system time used= 3.79442
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 37015
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 4786
involuntary context switches= 15807

runsolver used 2.01269 second user time and 4.52331 second system time

The end

Launcher Data 

Begin job on node146 at 2011-05-15 15:32:08
IDJOB=3378592
IDBENCH=83001
IDSOLVER=1859
FILE ID=node146/3378592-1305466328
RUNJOBID= node146-1305463158-20838
PBS_JOBID= 13324984
Free space on /tmp= 73260 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/medium/unif-k3-r4.26-v500-c2130-S247862791-097.UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3378592-1305466328/watcher-3378592-1305466328 -o /tmp/evaluation-result-3378592-1305466328/solver-3378592-1305466328 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378592-1305466328.cnf -seed 1479294503 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 6b69f72e1046222baee8c8d817a4adac
RANDOM SEED=1479294503

node146.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      21921816 kB
Buffers:        155268 kB
Cached:        1083952 kB
SwapCached:      56148 kB
Active:        8241832 kB
Inactive:      2519904 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      21921816 kB
SwapTotal:    67111528 kB
SwapFree:     51873460 kB
Dirty:           64968 kB
Writeback:           0 kB
AnonPages:     9472876 kB
Mapped:          29672 kB
Slab:           115272 kB
PageTables:      84116 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 120200988 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73228 MiB
End job on node146 at 2011-05-15 15:39:43