Trace number 3378443

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)UNSAT 1205.97 303.223

General information on the benchmark

Namerandom/medium/
unif-k5-r21.3-v100-c2130-S2051088205-036.UNKNOWN.cnf
MD5SUM901aadbc76f4961d582bc9856b3c63b3
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark300.4
Satisfiable
(Un)Satisfiability was proved
Number of variables100
Number of clauses2130
Sum of the clauses size10650
Maximum clause length5
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 52130
Number of clauses of size over 50

Solver Data 

0.29/0.46	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.29/0.46	c INFO: solving HOME/instance-3378443-1305465567.cnf
0.29/0.46	c INFO: computing task features
0.29/0.47	c INFO: computed variable-clause graph statistics
0.29/0.47	c INFO: computed clause balance statistics
0.38/0.53	c INFO: computed variable balance statistics
0.38/0.56	c INFO: computed small-clause counts
0.38/0.56	c INFO: computed Horn-clause counts
0.38/0.56	c INFO: computed variable graph statistics
0.38/0.56	c INFO: collected features for HOME/instance-3378443-1305465567.cnf in 0.10 s
0.38/0.57	c INFO: running gnovelty+2@0 for 97 with 1199 remaining (b = 0.62)
0.38/0.58	c INFO: running march_hi@0 for 388 with 1199 remaining (b = 0.33)
0.38/0.59	c INFO: running march_hi@0 for 388 with 1199 remaining (b = 0.29)
0.48/0.60	c INFO: running gnovelty+2@0 for 97 with 1199 remaining (b = 0.07)
396.20/99.95	c INFO: running march_hi@0 for 679 with 1100 remaining (b = 0.38)
400.17/100.90	c INFO: running march_hi@0 for 679 with 1099 remaining (b = 0.34)
1005.06/303.20	s UNSATISFIABLE

Verifier Data 

No possible verification on an UNSAT instance

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3378443-1305465567/watcher-3378443-1305465567 -o /tmp/evaluation-result-3378443-1305465567/solver-3378443-1305465567 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378443-1305465567.cnf -seed 1000761321 -budget 4800 -cores 4 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.02 8.24 7.90 6/465 9308
/proc/meminfo: memFree=21447320/32951124 swapFree=56449232/67111528
[pid=9308] ppid=9306 vsize=15780 CPUtime=0 cores=0,2,4,6
/proc/9308/stat : 9308 (python) R 9306 9308 8588 0 -1 4202496 910 0 0 0 0 0 0 0 25 0 1 0 658157112 16158720 791 33554432000 4194304 5683644 140734548234512 18446744073709551615 4540219 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/9308/statm: 3945 792 346 364 0 432 0

[startup+0.0258971 s]
/proc/loadavg: 8.02 8.24 7.90 6/465 9308
/proc/meminfo: memFree=21447320/32951124 swapFree=56449232/67111528
[pid=9308] ppid=9306 vsize=16812 CPUtime=0.01 cores=0,2,4,6
/proc/9308/stat : 9308 (python) R 9306 9308 8588 0 -1 4202496 1226 0 0 0 1 0 0 0 18 0 1 0 658157112 17215488 1084 33554432000 4194304 5683644 140734548234512 18446744073709551615 4868539 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/9308/statm: 4203 1084 417 364 0 690 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 16812

[startup+0.100839 s]
/proc/loadavg: 8.02 8.24 7.90 6/465 9308
/proc/meminfo: memFree=21447320/32951124 swapFree=56449232/67111528
[pid=9308] ppid=9306 vsize=104232 CPUtime=0.05 cores=0,2,4,6
/proc/9308/stat : 9308 (python) R 9306 9308 8588 0 -1 4202496 3087 0 0 0 4 1 0 0 17 0 1 0 658157112 106733568 2768 33554432000 4194304 5683644 140734548234512 18446744073709551615 253640520389 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/9308/statm: 26058 2768 847 364 0 1806 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 104232

[startup+0.300846 s]
/proc/loadavg: 8.02 8.24 7.90 6/465 9308
/proc/meminfo: memFree=21447320/32951124 swapFree=56449232/67111528
[pid=9308] ppid=9306 vsize=194416 CPUtime=0.21 cores=0,2,4,6
/proc/9308/stat : 9308 (python) R 9306 9308 8588 0 -1 4202496 6907 934 1 0 16 5 0 0 17 0 1 0 658157112 199081984 4960 33554432000 4194304 5683644 140734548234512 18446744073709551615 253640857600 0 2147483391 16781312 2 0 0 0 17 2 0 0 0
/proc/9308/statm: 48604 4960 1431 364 0 3723 0
Current children cumulated CPU time (s) 0.21
Current children cumulated vsize (KiB) 194416

[startup+0.70073 s]
/proc/loadavg: 8.02 8.24 7.90 6/465 9308
/proc/meminfo: memFree=21447320/32951124 swapFree=56449232/67111528
[pid=9308] ppid=9306 vsize=320036 CPUtime=0.48 cores=0,2,4,6
/proc/9308/stat : 9308 (python) S 9306 9308 8588 0 -1 4202496 12163 934 1 0 40 8 0 0 22 0 5 0 658157112 327716864 7214 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652686283 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/9308/statm: 80009 7214 1873 364 0 15918 0
Current children cumulated CPU time (s) 0.48
Current children cumulated vsize (KiB) 320036

[startup+1.50058 s]
/proc/loadavg: 8.02 8.24 7.90 9/478 9325
/proc/meminfo: memFree=21407288/32951124 swapFree=56449232/67111528
[pid=9308] ppid=9306 vsize=320036 CPUtime=0.48 cores=0,2,4,6
/proc/9308/stat : 9308 (python) S 9306 9308 8588 0 -1 4202496 12163 934 1 0 40 8 0 0 22 0 5 0 658157112 327716864 7214 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652686283 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/9308/statm: 80009 7214 1873 364 0 15918 0
[pid=9314] ppid=9308 vsize=278456 CPUtime=0 cores=0,2,4,6
/proc/9314/stat : 9314 (python) S 9308 9308 8588 0 -1 4202560 730 0 0 0 0 0 0 0 18 0 1 0 658157169 285138944 5529 33554432000 4194304 5683644 140734548234512 18446744073709551615 253640879631 0 0 16781312 514 11033525796625973247 0 0 17 2 0 0 0
/proc/9314/statm: 69614 5529 354 364 0 5532 0
[pid=9316] ppid=9314 vsize=1017800 CPUtime=0.92 cores=0,2,4,6
/proc/9316/stat : 9316 (gnovelty+2) R 9314 9316 9316 0 -1 4202496 433 0 0 0 92 0 0 0 21 0 1 0 658157169 1042227200 171 33554432000 134512640 135663704 4287484352 18446744073709551615 134525684 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/9316/statm: 254450 171 113 282 0 254166 0
[pid=9317] ppid=9308 vsize=289280 CPUtime=0 cores=0,2,4,6
/proc/9317/stat : 9317 (python) S 9308 9308 8588 0 -1 4202560 748 0 0 0 0 0 0 0 15 0 1 0 658157170 296222720 5677 33554432000 4194304 5683644 140734548234512 18446744073709551615 253640879686 0 0 16781312 514 6195897560394072323 0 0 17 2 0 0 0
/proc/9317/statm: 72320 5677 348 364 0 8235 0
[pid=9319] ppid=9317 vsize=1560 CPUtime=0.89 cores=0,2,4,6
/proc/9319/stat : 9319 (march_hi) R 9317 9319 9319 0 -1 4202496 505 0 0 0 89 0 0 0 25 0 1 0 658157170 1597440 240 33554432000 4194304 4955518 140736131371296 18446744073709551615 4285588 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/9319/statm: 390 240 106 186 0 203 0
[pid=9320] ppid=9308 vsize=299536 CPUtime=0 cores=0,2,4,6
/proc/9320/stat : 9320 (python) S 9308 9308 8588 0 -1 4202560 752 0 0 0 0 0 0 0 15 0 1 0 658157171 306724864 5681 33554432000 4194304 5683644 140734548234512 18446744073709551615 253640879686 0 0 16781312 514 8903374575048294658 0 0 17 0 0 0 0
/proc/9320/statm: 74884 5681 348 364 0 10796 0
[pid=9322] ppid=9320 vsize=1560 CPUtime=0.9 cores=0,2,4,6
/proc/9322/stat : 9322 (march_hi) R 9320 9322 9322 0 -1 4202496 499 0 0 0 90 0 0 0 23 0 1 0 658157171 1597440 240 33554432000 4194304 4955518 140736077671408 18446744073709551615 4224450 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/9322/statm: 390 240 106 186 0 203 0
[pid=9323] ppid=9308 vsize=309792 CPUtime=0 cores=0,2,4,6
/proc/9323/stat : 9323 (python) S 9308 9308 8588 0 -1 4202560 754 0 0 0 0 0 0 0 21 0 1 0 658157172 317227008 5685 33554432000 4194304 5683644 140734548234512 18446744073709551615 253640879686 0 0 16781312 514 12556124276089946111 0 0 17 6 0 0 0
/proc/9323/statm: 77448 5685 348 364 0 13357 0
[pid=9325] ppid=9323 vsize=1017800 CPUtime=0.89 cores=0,2,4,6
/proc/9325/stat : 9325 (gnovelty+2) R 9323 9325 9325 0 -1 4202496 433 0 0 0 89 0 0 0 25 0 1 0 658157172 1042227200 171 33554432000 134512640 135663704 4290941840 18446744073709551615 134545583 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/9325/statm: 254450 171 113 282 0 254166 0
[pid=9308/tid=9315] ppid=9306 vsize=320036 CPUtime=0 cores=0,2,4,6
/proc/9308/task/9315/stat : 9315 (python) S 9306 9308 8588 0 -1 4202560 24 934 0 0 0 0 0 0 19 0 5 0 658157169 327716864 7214 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 2 18446604452100827904 0 0 -1 4 0 0 0
[pid=9308/tid=9318] ppid=9306 vsize=320036 CPUtime=0 cores=0,2,4,6
/proc/9308/task/9318/stat : 9318 (python) S 9306 9308 8588 0 -1 4202560 24 934 0 0 0 0 0 0 21 0 5 0 658157170 327716864 7214 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 2 18446604452100827904 0 0 -1 2 0 0 0
[pid=9308/tid=9321] ppid=9306 vsize=320036 CPUtime=0 cores=0,2,4,6
/proc/9308/task/9321/stat : 9321 (python) S 9306 9308 8588 0 -1 4202560 22 934 0 0 0 0 0 0 20 0 5 0 658157171 327716864 7214 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 2 18446604452100827904 0 0 -1 0 0 0 0
[pid=9308/tid=9324] ppid=9306 vsize=320036 CPUtime=0 cores=0,2,4,6
/proc/9308/task/9324/stat : 9324 (python) S 9306 9308 8588 0 -1 4202560 27 934 0 0 0 0 0 0 22 0 5 0 658157172 327716864 7214 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 2 18446604452100827904 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4.08
Current children cumulated vsize (KiB) 3535820

[startup+3.10129 s]
/proc/loadavg: 8.02 8.24 7.90 10/478 9325
/proc/meminfo: memFree=21405552/32951124 swapFree=56449232/67111528
[pid=9308] ppid=9306 vsize=320036 CPUtime=0.48 cores=0,2,4,6
/proc/9308/stat : 9308 (python) S 9306 9308 8588 0 -1 4202496 12163 934 1 0 40 8 0 0 22 0 5 0 658157112 327716864 7214 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652686283 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/9308/statm: 80009 7214 1873 364 0 15918 0
[pid=9314] ppid=9308 vsize=278456 CPUtime=0 cores=0,2,4,6
/proc/9314/stat : 9314 (python) S 9308 9308 8588 0 -1 4202560 731 0 0 0 0 0 0 0 18 0 1 0 658157169 285138944 5529 33554432000 4194304 5683644 140734548234512 18446744073709551615 253640879631 0 0 16781312 514 11033525796625940737 0 0 17 2 0 0 0
/proc/9314/statm: 69614 5529 354 364 0 5532 0
[pid=9316] ppid=9314 vsize=1017800 CPUtime=2.52 cores=0,2,4,6
/proc/9316/stat : 9316 (gnovelty+2) R 9314 9316 9316 0 -1 4202496 433 0 0 0 252 0 0 0 25 0 1 0 658157169 1042227200 171 33554432000 134512640 135663704 4287484352 18446744073709551615 134545631 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/9316/statm: 254450 171 113 282 0 254166 0
[pid=9317] ppid=9308 vsize=289280 CPUtime=0 cores=0,2,4,6
/proc/9317/stat : 9317 (python) S 9308 9308 8588 0 -1 4202560 748 0 0 0 0 0 0 0 15 0 1 0 658157170 296222720 5677 33554432000 4194304 5683644 140734548234512 18446744073709551615 253640879686 0 0 16781312 514 6195897560394072323 0 0 17 2 0 0 0
/proc/9317/statm: 72320 5677 348 364 0 8235 0
[pid=9319] ppid=9317 vsize=1560 CPUtime=2.48 cores=0,2,4,6
/proc/9319/stat : 9319 (march_hi) R 9317 9319 9319 0 -1 4202496 506 0 0 0 248 0 0 0 25 0 1 0 658157170 1597440 241 33554432000 4194304 4955518 140736131371296 18446744073709551615 4208667 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/9319/statm: 390 241 106 186 0 203 0
[pid=9320] ppid=9308 vsize=299536 CPUtime=0 cores=0,2,4,6
/proc/9320/stat : 9320 (python) S 9308 9308 8588 0 -1 4202560 752 0 0 0 0 0 0 0 15 0 1 0 658157171 306724864 5681 33554432000 4194304 5683644 140734548234512 18446744073709551615 253640879686 0 0 16781312 514 8903374575048294658 0 0 17 0 0 0 0
/proc/9320/statm: 74884 5681 348 364 0 10796 0
[pid=9322] ppid=9320 vsize=1560 CPUtime=2.5 cores=0,2,4,6
/proc/9322/stat : 9322 (march_hi) R 9320 9322 9322 0 -1 4202496 500 0 0 0 250 0 0 0 25 0 1 0 658157171 1597440 241 33554432000 4194304 4955518 140736077671408 18446744073709551615 4286092 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/9322/statm: 390 241 106 186 0 203 0
[pid=9323] ppid=9308 vsize=309792 CPUtime=0 cores=0,2,4,6
/proc/9323/stat : 9323 (python) S 9308 9308 8588 0 -1 4202560 754 0 0 0 0 0 0 0 18 0 1 0 658157172 317227008 5685 33554432000 4194304 5683644 140734548234512 18446744073709551615 253640879686 0 0 16781312 514 12556124276089913604 0 0 17 6 0 0 0
/proc/9323/statm: 77448 5685 348 364 0 13357 0
[pid=9325] ppid=9323 vsize=1017800 CPUtime=2.49 cores=0,2,4,6
/proc/9325/stat : 9325 (gnovelty+2) R 9323 9325 9325 0 -1 4202496 433 0 0 0 249 0 0 0 25 0 1 0 658157172 1042227200 171 33554432000 134512640 135663704 4290941840 18446744073709551615 134544268 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/9325/statm: 254450 171 113 282 0 254166 0
[pid=9308/tid=9315] ppid=9306 vsize=320036 CPUtime=0 cores=0,2,4,6
/proc/9308/task/9315/stat : 9315 (python) S 9306 9308 8588 0 -1 4202560 24 934 0 0 0 0 0 0 19 0 5 0 658157169 327716864 7214 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 2 18446604452100827904 0 0 -1 4 0 0 0
[pid=9308/tid=9318] ppid=9306 vsize=320036 CPUtime=0 cores=0,2,4,6
/proc/9308/task/9318/stat : 9318 (python) S 9306 9308 8588 0 -1 4202560 24 934 0 0 0 0 0 0 21 0 5 0 658157170 327716864 7214 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 2 18446604452100827904 0 0 -1 2 0 0 0
[pid=9308/tid=9321] ppid=9306 vsize=320036 CPUtime=0 cores=0,2,4,6
/proc/9308/task/9321/stat : 9321 (python) S 9306 9308 8588 0 -1 4202560 22 934 0 0 0 0 0 0 20 0 5 0 658157171 327716864 7214 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 2 18446604452100827904 0 0 -1 0 0 0 0
[pid=9308/tid=9324] ppid=9306 vsize=320036 CPUtime=0 cores=0,2,4,6
/proc/9308/task/9324/stat : 9324 (python) S 9306 9308 8588 0 -1 4202560 27 934 0 0 0 0 0 0 22 0 5 0 658157172 327716864 7214 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 2 18446604452100827904 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 10.47
Current children cumulated vsize (KiB) 3535820

[startup+6.30071 s]
/proc/loadavg: 8.10 8.25 7.91 9/478 9325
/proc/meminfo: memFree=21401328/32951124 swapFree=56449232/67111528
[pid=9308] ppid=9306 vsize=320036 CPUtime=0.48 cores=0,2,4,6
/proc/9308/stat : 9308 (python) S 9306 9308 8588 0 -1 4202496 12163 934 1 0 40 8 0 0 22 0 5 0 658157112 327716864 7214 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652686283 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/9308/statm: 80009 7214 1873 364 0 15918 0
[pid=9314] ppid=9308 vsize=278456 CPUtime=0.01 cores=0,2,4,6
/proc/9314/stat : 9314 (python) S 9308 9308 8588 0 -1 4202560 885 0 0 0 0 1 0 0 18 0 1 0 658157169 285138944 5552 33554432000 4194304 5683644 140734548234512 18446744073709551615 253640879631 0 0 16781312 514 11033525796625940737 0 0 17 2 0 0 0
/proc/9314/statm: 69614 5552 377 364 0 5532 0
[pid=9316] ppid=9314 vsize=1017800 CPUtime=5.72 cores=0,2,4,6
/proc/9316/stat : 9316 (gnovelty+2) R 9314 9316 9316 0 -1 4202496 433 0 0 0 572 0 0 0 25 0 1 0 658157169 1042227200 171 33554432000 134512640 135663704 4287484352 18446744073709551615 134524882 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/9316/statm: 254450 171 113 282 0 254166 0
[pid=9317] ppid=9308 vsize=289280 CPUtime=0.01 cores=0,2,4,6
/proc/9317/stat : 9317 (python) S 9308 9308 8588 0 -1 4202560 918 0 0 0 1 0 0 0 15 0 1 0 658157170 296222720 5700 33554432000 4194304 5683644 140734548234512 18446744073709551615 253640879686 0 0 16781312 514 6195897560394072323 0 0 17 6 0 0 0
/proc/9317/statm: 72320 5700 371 364 0 8235 0
[pid=9319] ppid=9317 vsize=1776 CPUtime=5.59 cores=0,2,4,6
/proc/9319/stat : 9319 (march_hi) R 9317 9319 9319 0 -1 4202496 533 0 0 0 559 0 0 0 25 0 1 0 658157170 1818624 268 33554432000 4194304 4955518 140736131371296 18446744073709551615 4208667 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/9319/statm: 444 268 106 186 0 257 0
[pid=9320] ppid=9308 vsize=299536 CPUtime=0.02 cores=0,2,4,6
/proc/9320/stat : 9320 (python) S 9308 9308 8588 0 -1 4202560 915 0 0 0 1 1 0 0 15 0 1 0 658157171 306724864 5704 33554432000 4194304 5683644 140734548234512 18446744073709551615 253640879686 0 0 16781312 514 8903374575048294658 0 0 17 0 0 0 0
/proc/9320/statm: 74884 5704 371 364 0 10796 0
[pid=9322] ppid=9320 vsize=1776 CPUtime=5.67 cores=0,2,4,6
/proc/9322/stat : 9322 (march_hi) R 9320 9322 9322 0 -1 4202496 527 0 0 0 567 0 0 0 25 0 1 0 658157171 1818624 268 33554432000 4194304 4955518 140736077671408 18446744073709551615 4224468 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/9322/statm: 444 268 106 186 0 257 0
[pid=9323] ppid=9308 vsize=309792 CPUtime=0.01 cores=0,2,4,6
/proc/9323/stat : 9323 (python) S 9308 9308 8588 0 -1 4202560 922 0 0 0 0 1 0 0 18 0 1 0 658157172 317227008 5708 33554432000 4194304 5683644 140734548234512 18446744073709551615 253640879686 0 0 16781312 514 12556124276089913604 0 0 17 6 0 0 0
/proc/9323/statm: 77448 5708 371 364 0 13357 0
[pid=9325] ppid=9323 vsize=1017800 CPUtime=5.67 cores=0,2,4,6
/proc/9325/stat : 9325 (gnovelty+2) R 9323 9325 9325 0 -1 4202496 433 0 0 0 567 0 0 0 25 0 1 0 658157172 1042227200 171 33554432000 134512640 135663704 4290941840 18446744073709551615 134524732 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/9325/statm: 254450 171 113 282 0 254166 0
[pid=9308/tid=9315] ppid=9306 vsize=320036 CPUtime=0 cores=0,2,4,6
/proc/9308/task/9315/stat : 9315 (python) S 9306 9308 8588 0 -1 4202560 24 934 0 0 0 0 0 0 19 0 5 0 658157169 327716864 7214 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 2 18446604452100827904 0 0 -1 4 0 0 0
[pid=9308/tid=9318] ppid=9306 vsize=320036 CPUtime=0 cores=0,2,4,6
/proc/9308/task/9318/stat : 9318 (python) S 9306 9308 8588 0 -1 4202560 24 934 0 0 0 0 0 0 21 0 5 0 658157170 327716864 7214 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 2 18446604452100827904 0 0 -1 2 0 0 0
[pid=9308/tid=9321] ppid=9306 vsize=320036 CPUtime=0 cores=0,2,4,6
/proc/9308/task/9321/stat : 9321 (python) S 9306 9308 8588 0 -1 4202560 22 934 0 0 0 0 0 0 20 0 5 0 658157171 327716864 7214 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 2 18446604452100827904 0 0 -1 0 0 0 0
[pid=9308/tid=9324] ppid=9306 vsize=320036 CPUtime=0 cores=0,2,4,6
/proc/9308/task/9324/stat : 9324 (python) S 9306 9308 8588 0 -1 4202560 27 934 0 0 0 0 0 0 22 0 5 0 658157172 327716864 7214 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 2 18446604452100827904 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 23.18
Current children cumulated vsize (KiB) 3536252

[startup+12.7005 s]
/proc/loadavg: 8.09 8.25 7.91 9/478 9325
/proc/meminfo: memFree=21395724/32951124 swapFree=56449232/67111528
[pid=9308] ppid=9306 vsize=320036 CPUtime=0.48 cores=0,2,4,6
/proc/9308/stat : 9308 (python) S 9306 9308 8588 0 -1 4202496 12163 934 1 0 40 8 0 0 22 0 5 0 658157112 327716864 7214 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652686283 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/9308/statm: 80009 7214 1873 364 0 15918 0
[pid=9314] ppid=9308 vsize=278456 CPUtime=0.03 cores=0,2,4,6
/proc/9314/stat : 9314 (python) S 9308 9308 8588 0 -1 4202560 888 0 0 0 2 1 0 0 18 0 1 0 658157169 285138944 5552 33554432000 4194304 5683644 140734548234512 18446744073709551615 253640879631 0 0 16781312 514 17400466400903921921 0 0 17 2 0 0 0
/proc/9314/statm: 69614 5552 377 364 0 5532 0
[pid=9316] ppid=9314 vsize=1017800 CPUtime=12.12 cores=0,2,4,6
/proc/9316/stat : 9316 (gnovelty+2) R 9314 9316 9316 0 -1 4202496 433 0 0 0 1212 0 0 0 25 0 1 0 658157169 1042227200 171 33554432000 134512640 135663704 4287484352 18446744073709551615 134524509 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/9316/statm: 254450 171 113 282 0 254166 0
[pid=9317] ppid=9308 vsize=289280 CPUtime=0.03 cores=0,2,4,6

################
# More data... #
################

[pid=9308] ppid=9306 vsize=340832 CPUtime=0.49 cores=0,2,4,6
/proc/9308/stat : 9308 (python) S 9306 9308 8588 0 -1 4202496 13784 934 1 0 41 8 0 0 20 0 7 0 658157112 349011968 7282 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652686283 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/9308/statm: 85208 7282 1878 364 0 21111 0
[pid=9314] ppid=9308 vsize=288700 CPUtime=0.33 cores=0,2,4,6
/proc/9314/stat : 9314 (python) S 9308 9308 8588 0 -1 4202560 1169 0 0 0 17 16 0 0 19 0 2 0 658157169 295628800 5596 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652686283 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/9314/statm: 72175 5596 416 364 0 8093 0
[pid=9316] ppid=9314 vsize=1017800 CPUtime=100.05 cores=0,2,4,6
/proc/9316/stat : 9316 (gnovelty+2) T 9314 9316 9316 0 -1 4202496 433 0 0 0 10005 0 0 0 25 0 1 0 658157169 1042227200 171 33554432000 134512640 135663704 4287484352 18446744073709551615 134545605 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/9316/statm: 254450 171 113 282 0 254166 0
[pid=9314/tid=9333] ppid=9308 vsize=288700 CPUtime=0 cores=0,2,4,6
/proc/9314/task/9333/stat : 9333 (python) S 9308 9308 8588 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 658167202 295628800 5596 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 514 18446604451186668864 0 0 -1 2 0 0 0
[pid=9317] ppid=9308 vsize=289280 CPUtime=1.13 cores=0,2,4,6
/proc/9317/stat : 9317 (python) S 9308 9308 8588 0 -1 4202560 1011 0 0 0 60 53 0 0 15 0 1 0 658157170 296222720 5700 33554432000 4194304 5683644 140734548234512 18446744073709551615 253640879686 0 0 16781312 514 6195897560394072323 0 0 17 6 0 0 0
/proc/9317/statm: 72320 5700 371 364 0 8235 0
[pid=9319] ppid=9317 vsize=13508 CPUtime=296.59 cores=0,2,4,6
/proc/9319/stat : 9319 (march_hi) R 9317 9319 9319 0 -1 4202496 2961 0 0 0 29658 1 0 0 25 0 1 0 658157170 13832192 2677 33554432000 4194304 4955518 140736131371296 18446744073709551615 4298055 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/9319/statm: 3377 2677 106 186 0 3190 0
[pid=9320] ppid=9308 vsize=299536 CPUtime=1.15 cores=0,2,4,6
/proc/9320/stat : 9320 (python) S 9308 9308 8588 0 -1 4202560 1009 0 0 0 62 53 0 0 15 0 1 0 658157171 306724864 5704 33554432000 4194304 5683644 140734548234512 18446744073709551615 253640879686 0 0 16781312 514 8903374575048294658 0 0 17 0 0 0 0
/proc/9320/statm: 74884 5704 371 364 0 10796 0
[pid=9322] ppid=9320 vsize=13508 CPUtime=301.76 cores=0,2,4,6
/proc/9322/stat : 9322 (march_hi) R 9320 9322 9322 0 -1 4202496 3072 0 0 0 30175 1 0 0 25 0 1 0 658157171 13832192 2794 33554432000 4194304 4955518 140736077671408 18446744073709551615 4297502 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/9322/statm: 3377 2794 106 186 0 3190 0
[pid=9323] ppid=9308 vsize=309792 CPUtime=0.33 cores=0,2,4,6
/proc/9323/stat : 9323 (python) S 9308 9308 8588 0 -1 4202560 1105 0 0 0 17 16 0 0 19 0 2 0 658157172 317227008 5738 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652686283 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/9323/statm: 77448 5738 400 364 0 13357 0
[pid=9325] ppid=9323 vsize=1017800 CPUtime=98.94 cores=0,2,4,6
/proc/9325/stat : 9325 (gnovelty+2) T 9323 9325 9325 0 -1 4202496 433 0 0 0 9893 1 0 0 25 0 1 0 658157172 1042227200 171 33554432000 134512640 135663704 4290941840 18446744073709551615 134546982 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/9325/statm: 254450 171 113 282 0 254166 0
[pid=9323/tid=9329] ppid=9308 vsize=309792 CPUtime=0 cores=0,2,4,6
/proc/9323/task/9329/stat : 9329 (python) S 9308 9308 8588 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 658167106 317227008 5738 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 514 18446604447246209152 0 0 -1 4 0 0 0
[pid=9330] ppid=9308 vsize=320180 CPUtime=0.75 cores=0,2,4,6
/proc/9330/stat : 9330 (python) S 9308 9308 8588 0 -1 4202560 997 0 0 0 41 34 0 0 15 0 1 0 658167107 327864320 5729 33554432000 4194304 5683644 140734548234512 18446744073709551615 253640879686 0 0 16781312 514 10809501127100301569 0 0 17 4 0 0 0
/proc/9330/statm: 80045 5729 371 364 0 15951 0
[pid=9332] ppid=9330 vsize=6860 CPUtime=202 cores=0,2,4,6
/proc/9332/stat : 9332 (march_hi) R 9330 9332 9332 0 -1 4202496 1790 0 0 0 20200 0 0 0 25 0 1 0 658167107 7024640 1512 33554432000 4194304 4955518 140736502564000 18446744073709551615 4235583 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/9332/statm: 1715 1512 106 186 0 1528 0
[pid=9334] ppid=9308 vsize=330588 CPUtime=0.76 cores=0,2,4,6
/proc/9334/stat : 9334 (python) S 9308 9308 8588 0 -1 4202560 986 0 0 0 40 36 0 0 15 0 1 0 658167203 338522112 5771 33554432000 4194304 5683644 140734548234512 18446744073709551615 253640879686 0 0 16781312 514 3584680589728383235 0 0 17 6 0 0 0
/proc/9334/statm: 82647 5771 371 364 0 18550 0
[pid=9336] ppid=9334 vsize=6860 CPUtime=200.09 cores=0,2,4,6
/proc/9336/stat : 9336 (march_hi) R 9334 9336 9336 0 -1 4202496 1760 0 0 0 20007 2 0 0 25 0 1 0 658167203 7024640 1481 33554432000 4194304 4955518 140734344928256 18446744073709551615 4237810 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/9336/statm: 1715 1481 106 186 0 1528 0
[pid=9308/tid=9315] ppid=9306 vsize=340832 CPUtime=0 cores=0,2,4,6
/proc/9308/task/9315/stat : 9315 (python) S 9306 9308 8588 0 -1 4202560 24 934 0 0 0 0 0 0 19 0 7 0 658157169 349011968 7282 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 2 18446604452100827904 0 0 -1 4 0 0 0
[pid=9308/tid=9318] ppid=9306 vsize=340832 CPUtime=0 cores=0,2,4,6
/proc/9308/task/9318/stat : 9318 (python) S 9306 9308 8588 0 -1 4202560 24 934 0 0 0 0 0 0 21 0 7 0 658157170 349011968 7282 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 2 18446604452100827904 0 0 -1 2 0 0 0
[pid=9308/tid=9321] ppid=9306 vsize=340832 CPUtime=0 cores=0,2,4,6
/proc/9308/task/9321/stat : 9321 (python) S 9306 9308 8588 0 -1 4202560 22 934 0 0 0 0 0 0 20 0 7 0 658157171 349011968 7282 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 2 18446604452100827904 0 0 -1 0 0 0 0
[pid=9308/tid=9324] ppid=9306 vsize=340832 CPUtime=0 cores=0,2,4,6
/proc/9308/task/9324/stat : 9324 (python) S 9306 9308 8588 0 -1 4202560 27 934 0 0 0 0 0 0 22 0 7 0 658157172 349011968 7282 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 2 18446604452100827904 0 0 -1 6 0 0 0
[pid=9308/tid=9331] ppid=9306 vsize=340832 CPUtime=0 cores=0,2,4,6
/proc/9308/task/9331/stat : 9331 (python) S 9306 9308 8588 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 7 0 658167107 349011968 7282 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 2 18446604452100827904 0 0 -1 6 0 0 0
[pid=9308/tid=9335] ppid=9306 vsize=340832 CPUtime=0 cores=0,2,4,6
/proc/9308/task/9335/stat : 9335 (python) S 9306 9308 8588 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 7 0 658167203 349011968 7282 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 2 18446604452100827904 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 1204.37
Current children cumulated vsize (KiB) 4255244

[startup+303.101 s]
/proc/loadavg: 8.30 8.26 8.00 10/487 9342
/proc/meminfo: memFree=21149080/32951124 swapFree=56449232/67111528
[pid=9308] ppid=9306 vsize=340832 CPUtime=0.49 cores=0,2,4,6
/proc/9308/stat : 9308 (python) S 9306 9308 8588 0 -1 4202496 13784 934 1 0 41 8 0 0 20 0 7 0 658157112 349011968 7282 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652686283 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/9308/statm: 85208 7282 1878 364 0 21111 0
[pid=9314] ppid=9308 vsize=288700 CPUtime=0.33 cores=0,2,4,6
/proc/9314/stat : 9314 (python) S 9308 9308 8588 0 -1 4202560 1169 0 0 0 17 16 0 0 19 0 2 0 658157169 295628800 5596 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652686283 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/9314/statm: 72175 5596 416 364 0 8093 0
[pid=9316] ppid=9314 vsize=1017800 CPUtime=100.05 cores=0,2,4,6
/proc/9316/stat : 9316 (gnovelty+2) T 9314 9316 9316 0 -1 4202496 433 0 0 0 10005 0 0 0 25 0 1 0 658157169 1042227200 171 33554432000 134512640 135663704 4287484352 18446744073709551615 134545605 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/9316/statm: 254450 171 113 282 0 254166 0
[pid=9314/tid=9333] ppid=9308 vsize=288700 CPUtime=0 cores=0,2,4,6
/proc/9314/task/9333/stat : 9333 (python) S 9308 9308 8588 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 658167202 295628800 5596 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 514 18446604451186668864 0 0 -1 2 0 0 0
[pid=9317] ppid=9308 vsize=289280 CPUtime=1.13 cores=0,2,4,6
/proc/9317/stat : 9317 (python) S 9308 9308 8588 0 -1 4202560 1011 0 0 0 60 53 0 0 15 0 1 0 658157170 296222720 5700 33554432000 4194304 5683644 140734548234512 18446744073709551615 253640879686 0 0 16781312 514 6195897560394072323 0 0 17 6 0 0 0
/proc/9317/statm: 72320 5700 371 364 0 8235 0
[pid=9319] ppid=9317 vsize=13508 CPUtime=296.99 cores=0,2,4,6
/proc/9319/stat : 9319 (march_hi) R 9317 9319 9319 0 -1 4202496 2968 0 0 0 29698 1 0 0 25 0 1 0 658157170 13832192 2684 33554432000 4194304 4955518 140736131371296 18446744073709551615 4208624 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/9319/statm: 3377 2684 106 186 0 3190 0
[pid=9320] ppid=9308 vsize=299536 CPUtime=1.15 cores=0,2,4,6
/proc/9320/stat : 9320 (python) S 9308 9308 8588 0 -1 4202560 1009 0 0 0 62 53 0 0 15 0 1 0 658157171 306724864 5704 33554432000 4194304 5683644 140734548234512 18446744073709551615 253640879686 0 0 16781312 514 5606933221989122306 0 0 17 0 0 0 0
/proc/9320/statm: 74884 5704 371 364 0 10796 0
[pid=9322] ppid=9320 vsize=13508 CPUtime=302.16 cores=0,2,4,6
/proc/9322/stat : 9322 (march_hi) R 9320 9322 9322 0 -1 4202496 3080 0 0 0 30215 1 0 0 25 0 1 0 658157171 13832192 2802 33554432000 4194304 4955518 140736077671408 18446744073709551615 4237750 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/9322/statm: 3377 2802 106 186 0 3190 0
[pid=9323] ppid=9308 vsize=309792 CPUtime=0.33 cores=0,2,4,6
/proc/9323/stat : 9323 (python) S 9308 9308 8588 0 -1 4202560 1105 0 0 0 17 16 0 0 19 0 2 0 658157172 317227008 5738 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652686283 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/9323/statm: 77448 5738 400 364 0 13357 0
[pid=9325] ppid=9323 vsize=1017800 CPUtime=98.94 cores=0,2,4,6
/proc/9325/stat : 9325 (gnovelty+2) T 9323 9325 9325 0 -1 4202496 433 0 0 0 9893 1 0 0 25 0 1 0 658157172 1042227200 171 33554432000 134512640 135663704 4290941840 18446744073709551615 134546982 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/9325/statm: 254450 171 113 282 0 254166 0
[pid=9323/tid=9329] ppid=9308 vsize=309792 CPUtime=0 cores=0,2,4,6
/proc/9323/task/9329/stat : 9329 (python) S 9308 9308 8588 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 658167106 317227008 5738 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 514 18446604447246209152 0 0 -1 4 0 0 0
[pid=9330] ppid=9308 vsize=320180 CPUtime=0.75 cores=0,2,4,6
/proc/9330/stat : 9330 (python) S 9308 9308 8588 0 -1 4202560 997 0 0 0 41 34 0 0 15 0 1 0 658167107 327864320 5729 33554432000 4194304 5683644 140734548234512 18446744073709551615 253640879686 0 0 16781312 514 10809501127100301569 0 0 17 4 0 0 0
/proc/9330/statm: 80045 5729 371 364 0 15951 0
[pid=9332] ppid=9330 vsize=6860 CPUtime=202.4 cores=0,2,4,6
/proc/9332/stat : 9332 (march_hi) R 9330 9332 9332 0 -1 4202496 1792 0 0 0 20240 0 0 0 25 0 1 0 658167107 7024640 1514 33554432000 4194304 4955518 140736502564000 18446744073709551615 4208722 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/9332/statm: 1715 1514 106 186 0 1528 0
[pid=9334] ppid=9308 vsize=330588 CPUtime=0.76 cores=0,2,4,6
/proc/9334/stat : 9334 (python) S 9308 9308 8588 0 -1 4202560 986 0 0 0 40 36 0 0 15 0 1 0 658167203 338522112 5771 33554432000 4194304 5683644 140734548234512 18446744073709551615 253640879686 0 0 16781312 514 3584680589728383235 0 0 17 6 0 0 0
/proc/9334/statm: 82647 5771 371 364 0 18550 0
[pid=9336] ppid=9334 vsize=6860 CPUtime=200.49 cores=0,2,4,6
/proc/9336/stat : 9336 (march_hi) R 9334 9336 9336 0 -1 4202496 1763 0 0 0 20047 2 0 0 25 0 1 0 658167203 7024640 1484 33554432000 4194304 4955518 140734344928256 18446744073709551615 4299084 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/9336/statm: 1715 1484 106 186 0 1528 0
[pid=9308/tid=9315] ppid=9306 vsize=340832 CPUtime=0 cores=0,2,4,6
/proc/9308/task/9315/stat : 9315 (python) S 9306 9308 8588 0 -1 4202560 24 934 0 0 0 0 0 0 19 0 7 0 658157169 349011968 7282 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 2 18446604452100827904 0 0 -1 4 0 0 0
[pid=9308/tid=9318] ppid=9306 vsize=340832 CPUtime=0 cores=0,2,4,6
/proc/9308/task/9318/stat : 9318 (python) S 9306 9308 8588 0 -1 4202560 24 934 0 0 0 0 0 0 21 0 7 0 658157170 349011968 7282 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 2 18446604452100827904 0 0 -1 2 0 0 0
[pid=9308/tid=9321] ppid=9306 vsize=340832 CPUtime=0 cores=0,2,4,6
/proc/9308/task/9321/stat : 9321 (python) S 9306 9308 8588 0 -1 4202560 22 934 0 0 0 0 0 0 20 0 7 0 658157171 349011968 7282 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 2 18446604452100827904 0 0 -1 0 0 0 0
[pid=9308/tid=9324] ppid=9306 vsize=340832 CPUtime=0 cores=0,2,4,6
/proc/9308/task/9324/stat : 9324 (python) S 9306 9308 8588 0 -1 4202560 27 934 0 0 0 0 0 0 22 0 7 0 658157172 349011968 7282 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 2 18446604452100827904 0 0 -1 6 0 0 0
[pid=9308/tid=9331] ppid=9306 vsize=340832 CPUtime=0 cores=0,2,4,6
/proc/9308/task/9331/stat : 9331 (python) S 9306 9308 8588 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 7 0 658167107 349011968 7282 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 2 18446604452100827904 0 0 -1 6 0 0 0
[pid=9308/tid=9335] ppid=9306 vsize=340832 CPUtime=0 cores=0,2,4,6
/proc/9308/task/9335/stat : 9335 (python) S 9306 9308 8588 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 7 0 658167203 349011968 7282 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 2 18446604452100827904 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 1205.97
Current children cumulated vsize (KiB) 4255244

[startup+303.201 s]
/proc/loadavg: 8.30 8.26 8.00 10/487 9342
/proc/meminfo: memFree=21149080/32951124 swapFree=56449232/67111528
[pid=9308] ppid=9306 vsize=340832 CPUtime=403.44 cores=0,2,4,6
/proc/9308/stat : 9308 (python) S 9306 9308 8588 0 -1 4202496 13825 6957 1 0 41 8 40226 69 17 0 7 0 658157112 349011968 7282 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652689727 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/9308/statm: 85208 7282 1878 364 0 21111 0
[pid=9317] ppid=9308 vsize=289280 CPUtime=1.13 cores=0,2,4,6
/proc/9317/stat : 9317 (python) S 9308 9308 8588 0 -1 4202560 1011 0 0 0 60 53 0 0 15 0 1 0 658157170 296222720 5700 33554432000 4194304 5683644 140734548234512 18446744073709551615 253640879686 0 0 16781312 514 6195897560394072323 0 0 17 6 0 0 0
/proc/9317/statm: 72320 5700 371 364 0 8235 0
[pid=9319] ppid=9317 vsize=13508 CPUtime=297.09 cores=0,2,4,6
/proc/9319/stat : 9319 (march_hi) R 9317 9319 9319 0 -1 4202496 2970 0 0 0 29708 1 0 0 25 0 1 0 658157170 13832192 2686 33554432000 4194304 4955518 140736131371296 18446744073709551615 4297337 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/9319/statm: 3377 2686 106 186 0 3190 0
[pid=9320] ppid=9308 vsize=0 CPUtime=303.4 cores=0,2,4,6
/proc/9320/stat : 9320 (python) Z 9308 9308 8588 0 -1 4202588 1226 3083 0 0 62 53 30224 1 15 0 1 0 658157171 0 0 33554432000 0 0 0 0 0 0 0 16781824 2 18446744073709551615 0 0 17 0 0 0 0
/proc/9320/statm: 0 0 0 0 0 0 0
[pid=9308/tid=9315] ppid=9306 vsize=340832 CPUtime=604.31 cores=0,2,4,6
/proc/9308/task/9315/stat : 9315 (python) S 9306 9308 8588 0 -1 4202560 24 9753 0 0 0 0 60323 108 19 0 7 0 658157169 349011968 7282 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 2 18446604452100827904 0 0 -1 4 0 0 0
[pid=9308/tid=9318] ppid=9306 vsize=340832 CPUtime=604.31 cores=0,2,4,6
/proc/9308/task/9318/stat : 9318 (python) S 9306 9308 8588 0 -1 4202560 24 9753 0 0 0 0 60323 108 21 0 7 0 658157170 349011968 7282 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 2 18446604452100827904 0 0 -1 2 0 0 0
[pid=9308/tid=9321] ppid=9306 vsize=340832 CPUtime=604.31 cores=0,2,4,6
/proc/9308/task/9321/stat : 9321 (python) S 9306 9308 8588 0 -1 4202560 22 9753 0 0 0 0 60323 108 20 0 7 0 658157171 349011968 7282 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 2 18446604452100827904 0 0 -1 0 0 0 0
[pid=9308/tid=9324] ppid=9306 vsize=340832 CPUtime=604.31 cores=0,2,4,6
/proc/9308/task/9324/stat : 9324 (python) S 9306 9308 8588 0 -1 4202560 27 9753 0 0 0 0 60323 108 22 0 7 0 658157172 349011968 7282 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 2 18446604452100827904 0 0 -1 6 0 0 0
[pid=9308/tid=9331] ppid=9306 vsize=340832 CPUtime=604.31 cores=0,2,4,6
/proc/9308/task/9331/stat : 9331 (python) S 9306 9308 8588 0 -1 4202560 28 9753 0 0 0 0 60323 108 20 0 7 0 658167107 349011968 7282 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 2 18446604452100827904 0 0 -1 6 0 0 0
[pid=9308/tid=9335] ppid=9306 vsize=340832 CPUtime=604.31 cores=0,2,4,6
/proc/9308/task/9335/stat : 9335 (python) S 9306 9308 8588 0 -1 4202560 28 9753 0 0 0 0 60323 108 20 0 7 0 658167203 349011968 7282 33554432000 4194304 5683644 140734548234512 18446744073709551615 253652683185 0 0 16781312 2 18446604452100827904 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 1005.06
Current children cumulated vsize (KiB) 643620

Child status: 0

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 9308 and gives
#  childrusage.ru_utime.tv_sec=1204
#  childrusage.ru_utime.tv_usec=243927
#  childrusage.ru_stime.tv_sec=2
#  childrusage.ru_stime.tv_usec=273654
# CPU time returned by wait4() is 1206.52
# while last known CPU time is 1005.06
#
# Using CPU time of the last sample as value...


# WARNING:
# The CPU time of some children was not reported to their father
# (probably because of a missing or aborted wait()).
# This 'lost CPU time' is added to the watched process CPU time.
#  lost CPU time (s): 200.91
#  lost CPU user time (s): 200.55
#  lost CPU system time (s): 0.36

Real time (s): 303.223
CPU time (s): 1205.97
CPU user time (s): 1203.76
CPU system time (s): 2.21
CPU usage (%): 397.717
Max. virtual memory (cumulated for all children) (KiB): 4255248

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1204.24
system time used= 2.27365
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 36262
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3966
involuntary context switches= 8013

runsolver used 1.49077 second user time and 2.92156 second system time

The end

Launcher Data 

Begin job on node106 at 2011-05-15 15:19:27
IDJOB=3378443
IDBENCH=82852
IDSOLVER=1859
FILE ID=node106/3378443-1305465567
RUNJOBID= node106-1305463093-8606
PBS_JOBID= 13324980
Free space on /tmp= 73356 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/medium/unif-k5-r21.3-v100-c2130-S2051088205-036.UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3378443-1305465567/watcher-3378443-1305465567 -o /tmp/evaluation-result-3378443-1305465567/solver-3378443-1305465567 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378443-1305465567.cnf -seed 1000761321 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 901aadbc76f4961d582bc9856b3c63b3
RANDOM SEED=1000761321

node106.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.20
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      21447600 kB
Buffers:        300244 kB
Cached:        6291720 kB
SwapCached:      27536 kB
Active:        2904252 kB
Inactive:      8337044 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      21447600 kB
SwapTotal:    67111528 kB
SwapFree:     56449232 kB
Dirty:           31512 kB
Writeback:           0 kB
AnonPages:     4630400 kB
Mapped:          26808 kB
Slab:           131208 kB
PageTables:      64052 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 126047948 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73324 MiB
End job on node106 at 2011-05-15 15:24:31