Trace number 3317099

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-random? (MO) 3147.88 792.018

General information on the benchmark

Namerandom/large/
unif-k7-r85-v200-c17000-S1348818483-020-UNKNOWN.cnf
MD5SUM112b17aaf1cede32f350e11f6333fc88
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark339.4
Satisfiable
(Un)Satisfiability was proved
Number of variables200
Number of clauses17000
Sum of the clauses size119000
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 517000

Solver Data 

0.35/0.65	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.35/0.65	c INFO: solving HOME/instance-3317099-1304267310.cnf
0.35/0.65	c INFO: computing task features
0.43/0.77	c INFO: computed variable-clause graph statistics
0.43/0.77	c INFO: computed clause balance statistics
1.03/1.36	c INFO: computed variable balance statistics
1.23/1.59	c INFO: computed small-clause counts
1.23/1.59	c INFO: computed Horn-clause counts
1.33/1.61	c INFO: computed variable graph statistics
1.33/1.61	c INFO: collected features for HOME/instance-3317099-1304267310.cnf in 0.95 s
1.33/1.64	c INFO: running gnovelty+2@0 for 97 with 4799 remaining (b = 0.35)
1.33/1.67	c INFO: running gnovelty+2@0 for 97 with 4799 remaining (b = 0.12)
1.33/1.69	c INFO: running adaptg2wsat2009++@0 for 97 with 4798 remaining (b = 0.08)
1.43/1.72	c INFO: running adaptg2wsat2009++@0 for 97 with 4798 remaining (b = 0.06)
397.31/100.88	c INFO: running adaptg2wsat2009++@0 for 97 with 4699 remaining (b = 0.05)
397.68/100.91	c INFO: running gnovelty+2@0 for 97 with 4699 remaining (b = 0.04)
401.26/101.83	c INFO: running FH@0 for 97 with 4698 remaining (b = 0.03)
401.26/101.87	c INFO: running adaptg2wsat2009++@0 for 97 with 4698 remaining (b = 0.03)
796.76/201.04	c INFO: running adaptg2wsat2009++@0 for 97 with 4599 remaining (b = 0.02)
797.10/201.10	c INFO: running FH@0 for 97 with 4599 remaining (b = 0.02)
800.70/202.02	c INFO: running hybridGM3@0 for 194 with 4598 remaining (b = 0.04)
800.70/202.07	c INFO: running gnovelty+2@0 for 97 with 4598 remaining (b = 0.02)
1195.85/301.21	c INFO: running hybridGM3@0 for 194 with 4499 remaining (b = 0.03)
1195.85/301.27	c INFO: running hybridGM3@0 for 194 with 4499 remaining (b = 0.03)
1196.20/301.32	c INFO: running TNM@0 for 1552 with 4499 remaining (b = 0.20)
1606.47/404.38	c INFO: running hybridGM3@98 for 97 with 4396 remaining (b = 0.01)
1977.74/497.58	c INFO: running hybridGM3@99 for 97 with 4303 remaining (b = 0.01)
1978.13/497.64	c INFO: running hybridGM3@99 for 97 with 4303 remaining (b = 0.01)
1985.70/499.55	c INFO: running adaptg2wsat2009++@0 for 97 with 4301 remaining (b = 0.01)
2380.00/598.76	c INFO: running hybridGM3@294 for 97 with 4201 remaining (b = 0.01)
2380.37/598.82	c INFO: running hybridGM3@295 for 97 with 4201 remaining (b = 0.01)
2383.96/599.74	c INFO: running FH@0 for 97 with 4200 remaining (b = 0.01)
2758.32/694.00	c INFO: running adaptg2wsat2009++@0 for 97 with 4106 remaining (b = 0.01)
2766.28/695.95	c INFO: running adaptg2wsat2009++@98 for 97 with 4104 remaining (b = 0.01)
2782.18/699.93	c INFO: running hybridGM3@395 for 97 with 4100 remaining (b = 0.01)
3144.64/791.18	c INFO: running gnovelty+2@0 for 97 with 4009 remaining (b = 0.01)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3317099-1304267310/watcher-3317099-1304267310 -o /tmp/evaluation-result-3317099-1304267310/solver-3317099-1304267310 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3317099-1304267310.cnf -seed 979871631 -budget 4800 -cores 4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.49 0.11 0.04 2/177 22281
/proc/meminfo: memFree=30138352/32951124 swapFree=67111524/67111528
[pid=22281] ppid=22279 vsize=15184 CPUtime=0 cores=1,3,5,7
/proc/22281/stat : 22281 (python) R 22279 22281 22195 0 -1 4202496 709 0 0 0 0 0 0 0 25 0 1 0 270922299 15548416 600 33554432000 4194304 5683644 140735173406864 18446744073709551615 268660356062 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/22281/statm: 3796 601 319 364 0 283 0

[startup+0.080725 s]
/proc/loadavg: 0.49 0.11 0.04 2/177 22281
/proc/meminfo: memFree=30138352/32951124 swapFree=67111524/67111528
[pid=22281] ppid=22279 vsize=38904 CPUtime=0.04 cores=1,3,5,7
/proc/22281/stat : 22281 (python) S 22279 22281 22195 0 -1 4202496 1630 0 0 0 3 1 0 0 25 0 1 0 270922299 39837696 1458 33554432000 4194304 5683644 140735173406864 18446744073709551615 268660660224 0 2147483391 16781312 2 18446744071699199166 0 0 17 7 0 0 0
/proc/22281/statm: 9726 1458 455 364 0 1026 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 38904

[startup+0.100908 s]
/proc/loadavg: 0.49 0.11 0.04 2/177 22281
/proc/meminfo: memFree=30138352/32951124 swapFree=67111524/67111528
[pid=22281] ppid=22279 vsize=84524 CPUtime=0.06 cores=1,3,5,7
/proc/22281/stat : 22281 (python) S 22279 22281 22195 0 -1 4202496 2399 0 0 0 4 2 0 0 18 0 1 0 270922299 86552576 2127 33554432000 4194304 5683644 140735173406864 18446744073709551615 268660688668 0 2147483391 16781312 2 18446744071699199166 0 0 17 7 0 0 0
/proc/22281/statm: 21118 2127 707 364 0 1335 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 84524

[startup+0.301025 s]
/proc/loadavg: 0.49 0.11 0.04 2/177 22281
/proc/meminfo: memFree=30138352/32951124 swapFree=67111524/67111528
[pid=22281] ppid=22279 vsize=135584 CPUtime=0.17 cores=1,3,5,7
/proc/22281/stat : 22281 (python) S 22279 22281 22195 0 -1 4202496 5909 933 1 0 13 4 0 0 17 0 1 0 270922299 138838016 4132 33554432000 4194304 5683644 140735173406864 18446744073709551615 268660660224 0 2147483391 16781312 2 18446744071699199166 0 0 17 7 0 0 3
/proc/22281/statm: 33896 4132 1031 364 0 3082 0
Current children cumulated CPU time (s) 0.17
Current children cumulated vsize (KiB) 135584

[startup+0.705676 s]
/proc/loadavg: 0.49 0.11 0.04 2/177 22281
/proc/meminfo: memFree=30138352/32951124 swapFree=67111524/67111528
[pid=22281] ppid=22279 vsize=280040 CPUtime=0.43 cores=1,3,5,7
/proc/22281/stat : 22281 (python) R 22279 22281 22195 0 -1 4202496 9762 933 1 0 33 10 0 0 16 0 1 0 270922299 286760960 7330 33554432000 4194304 5683644 140735173406864 18446744073709551615 268660720042 0 0 16781312 2 0 0 0 17 1 0 0 3
/proc/22281/statm: 70010 7330 1761 364 0 5934 0
Current children cumulated CPU time (s) 0.43
Current children cumulated vsize (KiB) 280040

[startup+1.50051 s]
/proc/loadavg: 0.49 0.11 0.04 3/178 22286
/proc/meminfo: memFree=30112264/32951124 swapFree=67111524/67111528
[pid=22281] ppid=22279 vsize=282588 CPUtime=1.23 cores=1,3,5,7
/proc/22281/stat : 22281 (python) R 22279 22281 22195 0 -1 4202496 11328 933 1 0 112 11 0 0 17 0 1 0 270922299 289370112 8060 33554432000 4194304 5683644 140735173406864 18446744073709551615 47909266180126 0 0 16781312 2 0 0 0 17 1 0 0 3
/proc/22281/statm: 70647 8060 1827 364 0 6571 0
Current children cumulated CPU time (s) 1.23
Current children cumulated vsize (KiB) 282588

[startup+3.10119 s]
/proc/loadavg: 0.49 0.11 0.04 6/190 22298
/proc/meminfo: memFree=30091708/32951124 swapFree=67111524/67111528
[pid=22281] ppid=22279 vsize=320512 CPUtime=1.45 cores=1,3,5,7
/proc/22281/stat : 22281 (python) S 22279 22281 22195 0 -1 4202496 14166 933 1 0 134 11 0 0 25 0 5 0 270922299 328204288 7327 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672488907 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 3
/proc/22281/statm: 80128 7327 1883 364 0 16037 0
[pid=22287] ppid=22281 vsize=279232 CPUtime=0 cores=1,3,5,7
/proc/22287/stat : 22287 (python) S 22281 22281 22195 0 -1 4202560 727 0 0 0 0 0 0 0 18 0 1 0 270922464 285933568 5715 33554432000 4194304 5683644 140735173406864 18446744073709551615 268660682255 0 0 16781312 514 4594547110346457352 0 0 17 7 0 0 0
/proc/22287/statm: 69808 5715 354 364 0 5726 0
[pid=22289] ppid=22287 vsize=1018616 CPUtime=1.45 cores=1,3,5,7
/proc/22289/stat : 22289 (gnovelty+2) R 22287 22289 22289 0 -1 4202496 763 0 0 0 145 0 0 0 25 0 1 0 270922464 1043062784 500 33554432000 134512640 135663704 4288135600 18446744073709551615 134525286 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/22289/statm: 254654 500 113 282 0 254370 0
[pid=22290] ppid=22281 vsize=289624 CPUtime=0 cores=1,3,5,7
/proc/22290/stat : 22290 (python) S 22281 22281 22195 0 -1 4202560 744 0 0 0 0 0 0 0 18 0 1 0 270922466 296574976 5763 33554432000 4194304 5683644 140735173406864 18446744073709551615 268660682310 0 0 16781312 514 7133855745677689096 0 0 17 5 0 0 0
/proc/22290/statm: 72406 5763 348 364 0 8321 0
[pid=22292] ppid=22290 vsize=1018616 CPUtime=1.42 cores=1,3,5,7
/proc/22292/stat : 22292 (gnovelty+2) R 22290 22292 22292 0 -1 4202496 762 0 0 0 142 0 0 0 25 0 1 0 270922467 1043062784 500 33554432000 134512640 135663704 4291324720 18446744073709551615 134524918 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/22292/statm: 254654 500 113 282 0 254370 0
[pid=22293] ppid=22281 vsize=300012 CPUtime=0 cores=1,3,5,7
/proc/22293/stat : 22293 (python) S 22281 22281 22195 0 -1 4202560 738 0 0 0 0 0 0 0 18 0 1 0 270922469 307212288 5784 33554432000 4194304 5683644 140735173406864 18446744073709551615 268660682310 0 0 16781312 514 4500037488869375999 0 0 17 7 0 0 0
/proc/22293/statm: 75003 5784 348 364 0 10915 0
[pid=22295] ppid=22293 vsize=418500 CPUtime=1.38 cores=1,3,5,7
/proc/22295/stat : 22295 (adaptg2wsat2009) R 22293 22295 22295 0 -1 4202496 1074 0 0 0 138 0 0 0 25 0 1 0 270922469 428544000 809 33554432000 134512640 135107411 4290033072 18446744073709551615 134523302 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/22295/statm: 104625 809 49 146 0 103501 0
[pid=22296] ppid=22281 vsize=310268 CPUtime=0 cores=1,3,5,7
/proc/22296/stat : 22296 (python) S 22281 22281 22195 0 -1 4202560 747 0 0 0 0 0 0 0 18 0 1 0 270922472 317714432 5788 33554432000 4194304 5683644 140735173406864 18446744073709551615 268660682310 0 0 16781312 514 4493260648951578623 0 0 17 5 0 0 0
/proc/22296/statm: 77567 5788 348 364 0 13476 0
[pid=22298] ppid=22296 vsize=418500 CPUtime=1.36 cores=1,3,5,7
/proc/22298/stat : 22298 (adaptg2wsat2009) R 22296 22298 22298 0 -1 4202496 1072 0 0 0 136 0 0 0 25 0 1 0 270922472 428544000 808 33554432000 134512640 135107411 4287391856 18446744073709551615 134523424 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/22298/statm: 104625 808 49 146 0 103501 0
[pid=22281/tid=22288] ppid=22279 vsize=320512 CPUtime=0 cores=1,3,5,7
/proc/22281/task/22288/stat : 22288 (python) S 22279 22281 22195 0 -1 4202560 24 933 0 0 0 0 0 0 20 0 5 0 270922464 328204288 7327 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 2 18446604471130863744 0 0 -1 7 0 0 0
[pid=22281/tid=22291] ppid=22279 vsize=320512 CPUtime=0 cores=1,3,5,7
/proc/22281/task/22291/stat : 22291 (python) S 22279 22281 22195 0 -1 4202560 21 933 0 0 0 0 0 0 22 0 5 0 270922466 328204288 7327 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 2 18446604471130863744 0 0 -1 3 0 0 0
[pid=22281/tid=22294] ppid=22279 vsize=320512 CPUtime=0 cores=1,3,5,7
/proc/22281/task/22294/stat : 22294 (python) S 22279 22281 22195 0 -1 4202560 22 933 0 0 0 0 0 0 24 0 5 0 270922469 328204288 7327 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 2 18446604471130863744 0 0 -1 7 0 0 0
[pid=22281/tid=22297] ppid=22279 vsize=320512 CPUtime=0 cores=1,3,5,7
/proc/22281/task/22297/stat : 22297 (python) S 22279 22281 22195 0 -1 4202560 28 933 0 0 0 0 0 0 25 0 5 0 270922472 328204288 7327 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 2 18446604471130863744 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 7.06
Current children cumulated vsize (KiB) 4373880

[startup+6.30063 s]
/proc/loadavg: 1.09 0.24 0.08 9/202 22310
/proc/meminfo: memFree=29967020/32951124 swapFree=67111524/67111528
[pid=22281] ppid=22279 vsize=320512 CPUtime=1.45 cores=1,3,5,7
/proc/22281/stat : 22281 (python) S 22279 22281 22195 0 -1 4202496 14166 933 1 0 134 11 0 0 25 0 5 0 270922299 328204288 7327 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672488907 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 3
/proc/22281/statm: 80128 7327 1883 364 0 16037 0
[pid=22287] ppid=22281 vsize=279232 CPUtime=0 cores=1,3,5,7
/proc/22287/stat : 22287 (python) S 22281 22281 22195 0 -1 4202560 881 0 0 0 0 0 0 0 18 0 1 0 270922464 285933568 5738 33554432000 4194304 5683644 140735173406864 18446744073709551615 268660682255 0 0 16781312 514 4594547110346457352 0 0 17 7 0 0 0
/proc/22287/statm: 69808 5738 377 364 0 5726 0
[pid=22289] ppid=22287 vsize=1018616 CPUtime=4.65 cores=1,3,5,7
/proc/22289/stat : 22289 (gnovelty+2) R 22287 22289 22289 0 -1 4202496 763 0 0 0 465 0 0 0 25 0 1 0 270922464 1043062784 500 33554432000 134512640 135663704 4288135600 18446744073709551615 134524503 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/22289/statm: 254654 500 113 282 0 254370 0
[pid=22290] ppid=22281 vsize=289624 CPUtime=0 cores=1,3,5,7
/proc/22290/stat : 22290 (python) S 22281 22281 22195 0 -1 4202560 902 0 0 0 0 0 0 0 18 0 1 0 270922466 296574976 5786 33554432000 4194304 5683644 140735173406864 18446744073709551615 268660682310 0 0 16781312 514 7133855745677689096 0 0 17 5 0 0 0
/proc/22290/statm: 72406 5786 371 364 0 8321 0
[pid=22292] ppid=22290 vsize=1018616 CPUtime=4.62 cores=1,3,5,7
/proc/22292/stat : 22292 (gnovelty+2) R 22290 22292 22292 0 -1 4202496 762 0 0 0 462 0 0 0 25 0 1 0 270922467 1043062784 500 33554432000 134512640 135663704 4291324720 18446744073709551615 134524553 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/22292/statm: 254654 500 113 282 0 254370 0
[pid=22293] ppid=22281 vsize=300012 CPUtime=0 cores=1,3,5,7
/proc/22293/stat : 22293 (python) S 22281 22281 22195 0 -1 4202560 899 0 0 0 0 0 0 0 18 0 1 0 270922469 307212288 5807 33554432000 4194304 5683644 140735173406864 18446744073709551615 268660682310 0 0 16781312 514 4500037488869343496 0 0 17 7 0 0 0
/proc/22293/statm: 75003 5807 371 364 0 10915 0
[pid=22295] ppid=22293 vsize=418636 CPUtime=4.55 cores=1,3,5,7
/proc/22295/stat : 22295 (adaptg2wsat2009) R 22293 22295 22295 0 -1 4202496 1221 0 0 0 455 0 0 0 25 0 1 0 270922469 428683264 956 33554432000 134512640 135107411 4290033072 18446744073709551615 134547592 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/22295/statm: 104659 956 59 146 0 103535 0
[pid=22296] ppid=22281 vsize=310268 CPUtime=0 cores=1,3,5,7
/proc/22296/stat : 22296 (python) S 22281 22281 22195 0 -1 4202560 927 0 0 0 0 0 0 0 18 0 1 0 270922472 317714432 5811 33554432000 4194304 5683644 140735173406864 18446744073709551615 268660682310 0 0 16781312 514 4493260648951546120 0 0 17 5 0 0 0
/proc/22296/statm: 77567 5811 371 364 0 13476 0
[pid=22298] ppid=22296 vsize=418636 CPUtime=4.55 cores=1,3,5,7
/proc/22298/stat : 22298 (adaptg2wsat2009) R 22296 22298 22298 0 -1 4202496 1220 0 0 0 455 0 0 0 25 0 1 0 270922472 428683264 956 33554432000 134512640 135107411 4287391856 18446744073709551615 134531613 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/22298/statm: 104659 956 59 146 0 103535 0
[pid=22281/tid=22288] ppid=22279 vsize=320512 CPUtime=0 cores=1,3,5,7
/proc/22281/task/22288/stat : 22288 (python) S 22279 22281 22195 0 -1 4202560 24 933 0 0 0 0 0 0 20 0 5 0 270922464 328204288 7327 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 2 18446604471130863744 0 0 -1 7 0 0 0
[pid=22281/tid=22291] ppid=22279 vsize=320512 CPUtime=0 cores=1,3,5,7
/proc/22281/task/22291/stat : 22291 (python) S 22279 22281 22195 0 -1 4202560 21 933 0 0 0 0 0 0 22 0 5 0 270922466 328204288 7327 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 2 18446604471130863744 0 0 -1 3 0 0 0
[pid=22281/tid=22294] ppid=22279 vsize=320512 CPUtime=0 cores=1,3,5,7
/proc/22281/task/22294/stat : 22294 (python) S 22279 22281 22195 0 -1 4202560 22 933 0 0 0 0 0 0 24 0 5 0 270922469 328204288 7327 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 2 18446604471130863744 0 0 -1 7 0 0 0
[pid=22281/tid=22297] ppid=22279 vsize=320512 CPUtime=0 cores=1,3,5,7
/proc/22281/task/22297/stat : 22297 (python) S 22279 22281 22195 0 -1 4202560 28 933 0 0 0 0 0 0 25 0 5 0 270922472 328204288 7327 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 2 18446604471130863744 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 19.82
Current children cumulated vsize (KiB) 4374152

[startup+12.7005 s]
/proc/loadavg: 1.65 0.37 0.12 9/202 22310
/proc/meminfo: memFree=29963916/32951124 swapFree=67111524/67111528
[pid=22281] ppid=22279 vsize=320512 CPUtime=1.45 cores=1,3,5,7
/proc/22281/stat : 22281 (python) S 22279 22281 22195 0 -1 4202496 14166 933 1 0 134 11 0 0 25 0 5 0 270922299 328204288 7327 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672488907 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 3
/proc/22281/statm: 80128 7327 1883 364 0 16037 0
[pid=22287] ppid=22281 vsize=279232 CPUtime=0 cores=1,3,5,7
/proc/22287/stat : 22287 (python) S 22281 22281 22195 0 -1 4202560 884 0 0 0 0 0 0 0 18 0 1 0 270922464 285933568 5738 33554432000 4194304 5683644 140735173406864 18446744073709551615 268660682255 0 0 16781312 514 14500500084598210824 0 0 17 7 0 0 0
/proc/22287/statm: 69808 5738 377 364 0 5726 0
[pid=22289] ppid=22287 vsize=1018616 CPUtime=11.05 cores=1,3,5,7
/proc/22289/stat : 22289 (gnovelty+2) R 22287 22289 22289 0 -1 4202496 763 0 0 0 1105 0 0 0 25 0 1 0 270922464 1043062784 500 33554432000 134512640 135663704 4288135600 18446744073709551615 134524536 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/22289/statm: 254654 500 113 282 0 254370 0
[pid=22290] ppid=22281 vsize=289624 CPUtime=0.01 cores=1,3,5,7
/proc/22290/stat : 22290 (python) S 22281 22281 22195 0 -1 4202560 904 0 0 0 1 0 0 0 18 0 1 0 270922466 296574976 5786 33554432000 4194304 5683644 140735173406864 18446744073709551615 268660682310 0 0 16781312 514 14500500084598210824 0 0 17 5 0 0 0
/proc/22290/statm: 72406 5786 371 364 0 8321 0
[pid=22292] ppid=22290 vsize=1018616 CPUtime=11.02 cores=1,3,5,7
/proc/22292/stat : 22292 (gnovelty+2) R 22290 22292 22292 0 -1 4202496 762 0 0 0 1102 0 0 0 25 0 1 0 270922467 1043062784 500 33554432000 134512640 135663704 4291324720 18446744073709551615 134524868 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/22292/statm: 254654 500 113 282 0 254370 0
[pid=22293] ppid=22281 vsize=300012 CPUtime=0 cores=1,3,5,7
/proc/22293/stat : 22293 (python) S 22281 22281 22195 0 -1 4202560 904 0 0 0 0 0 0 0 18 0 1 0 270922469 307212288 5807 33554432000 4194304 5683644 140735173406864 18446744073709551615 268660682310 0 0 16781312 514 14500500084598210824 0 0 17 7 0 0 0
/proc/22293/statm: 75003 5807 371 364 0 10915 0
[pid=22295] ppid=22293 vsize=418636 CPUtime=10.9 cores=1,3,5,7
/proc/22295/stat : 22295 (adaptg2wsat2009) R 22293 22295 22295 0 -1 4202496 1221 0 0 0 1090 0 0 0 25 0 1 0 270922469 428683264 956 33554432000 134512640 135107411 4290033072 18446744073709551615 134531401 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/22295/statm: 104659 956 59 146 0 103535 0
[pid=22296] ppid=22281 vsize=310268 CPUtime=0 cores=1,3,5,7
/proc/22296/stat : 22296 (python) S 22281 22281 22195 0 -1 4202560 930 0 0 0 0 0 0 0 18 0 1 0 270922472 317714432 5811 33554432000 4194304 5683644 140735173406864 18446744073709551615 268660682310 0 0 16781312 514 14500500084598210824 0 0 17 5 0 0 0
/proc/22296/statm: 77567 5811 371 364 0 13476 0
[pid=22298] ppid=22296 vsize=418636 CPUtime=10.93 cores=1,3,5,7
/proc/22298/stat : 22298 (adaptg2wsat2009) R 22296 22298 22298 0 -1 4202496 1220 0 0 0 1093 0 0 0 25 0 1 0 270922472 428683264 956 33554432000 134512640 135107411 4287391856 18446744073709551615 134531779 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/22298/statm: 104659 956 59 146 0 103535 0
[pid=22281/tid=22288] ppid=22279 vsize=320512 CPUtime=0 cores=1,3,5,7
/proc/22281/task/22288/stat : 22288 (python) S 22279 22281 22195 0 -1 4202560 24 933 0 0 0 0 0 0 20 0 5 0 270922464 328204288 7327 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 2 18446604471130863744 0 0 -1 7 0 0 0
[pid=22281/tid=22291] ppid=22279 vsize=320512 CPUtime=0 cores=1,3,5,7
/proc/22281/task/22291/stat : 22291 (python) S 22279 22281 22195 0 -1 4202560 21 933 0 0 0 0 0 0 22 0 5 0 270922466 328204288 7327 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 2 18446604471130863744 0 0 -1 3 0 0 0
[pid=22281/tid=22294] ppid=22279 vsize=320512 CPUtime=0 cores=1,3,5,7
/proc/22281/task/22294/stat : 22294 (python) S 22279 22281 22195 0 -1 4202560 22 933 0 0 0 0 0 0 24 0 5 0 270922469 328204288 7327 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 2 18446604471130863744 0 0 -1 7 0 0 0
[pid=22281/tid=22297] ppid=22279 vsize=320512 CPUtime=0 cores=1,3,5,7
/proc/22281/task/22297/stat : 22297 (python) S 22279 22281 22195 0 -1 4202560 28 933 0 0 0 0 0 0 25 0 5 0 270922472 328204288 7327 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 2 18446604471130863744 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 45.36
Current children cumulated vsize (KiB) 4374152

[startup+25.5012 s]
/proc/loadavg: 3.13 0.76 0.25 9/202 22311
/proc/meminfo: memFree=29964212/32951124 swapFree=67111524/67111528
[pid=22281] ppid=22279 vsize=320512 CPUtime=1.45 cores=1,3,5,7

################
# More data... #
################

[pid=22293/tid=22314] ppid=22281 vsize=300012 CPUtime=0 cores=1,3,5,7
/proc/22293/task/22314/stat : 22314 (python) S 22281 22281 22195 0 -1 4202560 47 0 0 0 0 0 0 0 18 0 2 0 270932384 307212288 5837 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 514 18446604471140309056 0 0 -1 7 0 0 0
[pid=22296] ppid=22281 vsize=310268 CPUtime=0.27 cores=1,3,5,7
/proc/22296/stat : 22296 (python) S 22281 22281 22195 0 -1 4202560 1141 0 0 0 12 15 0 0 18 0 2 0 270922472 317714432 5841 33554432000 4194304 5683644 140735173406864 18446744073709551615 268660682310 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/22296/statm: 77567 5841 400 364 0 13476 0
[pid=22298] ppid=22296 vsize=418636 CPUtime=192.24 cores=1,3,5,7
/proc/22298/stat : 22298 (adaptg2wsat2009) R 22296 22298 22298 0 -1 4202496 1220 0 0 0 19224 0 0 0 25 0 1 0 270922472 428683264 956 33554432000 134512640 135107411 4287391856 18446744073709551615 134531696 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/22298/statm: 104659 956 59 146 0 103535 0
[pid=22296/tid=22315] ppid=22281 vsize=310268 CPUtime=0 cores=1,3,5,7
/proc/22296/task/22315/stat : 22315 (python) S 22281 22281 22195 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 270932387 317714432 5841 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 514 18446604471138731200 0 0 -1 7 0 0 0
[pid=22316] ppid=22281 vsize=320524 CPUtime=0.13 cores=1,3,5,7
/proc/22316/stat : 22316 (python) S 22281 22281 22195 0 -1 4202560 1129 0 0 0 7 6 0 0 19 0 2 0 270932387 328216576 5845 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672488907 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/22316/statm: 80131 5845 400 364 0 16037 0
[pid=22318] ppid=22316 vsize=418636 CPUtime=100.11 cores=1,3,5,7
/proc/22318/stat : 22318 (adaptg2wsat2009) T 22316 22318 22318 0 -1 4202496 1221 0 0 0 10011 0 0 0 25 0 1 0 270932388 428683264 955 33554432000 134512640 135107411 4290852016 18446744073709551615 134532264 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/22318/statm: 104659 955 59 146 0 103535 0
[pid=22316/tid=22340] ppid=22281 vsize=320524 CPUtime=0 cores=1,3,5,7
/proc/22316/task/22340/stat : 22340 (python) S 22281 22281 22195 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 270942400 328216576 5845 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 514 18446604469115665280 0 0 -1 7 0 0 0
[pid=22319] ppid=22281 vsize=330780 CPUtime=0.13 cores=1,3,5,7
/proc/22319/stat : 22319 (python) S 22281 22281 22195 0 -1 4202560 1128 0 0 0 7 6 0 0 19 0 2 0 270932391 338718720 5868 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672488907 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/22319/statm: 82695 5868 400 364 0 18598 0
[pid=22321] ppid=22319 vsize=1018616 CPUtime=99.87 cores=1,3,5,7
/proc/22321/stat : 22321 (gnovelty+2) T 22319 22321 22321 0 -1 4202496 764 0 0 0 9987 0 0 0 25 0 1 0 270932391 1043062784 500 33554432000 134512640 135663704 4291404496 18446744073709551615 134524897 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/22321/statm: 254654 500 113 282 0 254370 0
[pid=22319/tid=22344] ppid=22281 vsize=330780 CPUtime=0 cores=1,3,5,7
/proc/22319/task/22344/stat : 22344 (python) S 22281 22281 22195 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 270942406 338718720 5868 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 514 18446604471130864640 0 0 -1 5 0 0 0
[pid=22323] ppid=22281 vsize=341196 CPUtime=0.13 cores=1,3,5,7
/proc/22323/stat : 22323 (python) S 22281 22281 22195 0 -1 4202560 1174 0 0 0 6 7 0 0 19 0 2 0 270932482 349384704 5899 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672488907 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/22323/statm: 85299 5899 406 364 0 21199 0
[pid=22326] ppid=22323 vsize=248648 CPUtime=100 cores=1,3,5,7
/proc/22326/stat : 22326 (FH) T 22323 22326 22326 0 -1 4202496 1124 0 0 0 10000 0 0 0 25 0 1 0 270932483 254615552 857 33554432000 134512640 134936988 4286811552 18446744073709551615 134552486 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/22326/statm: 62162 857 41 104 0 61666 0
[pid=22323/tid=22348] ppid=22281 vsize=341196 CPUtime=0 cores=1,3,5,7
/proc/22323/task/22348/stat : 22348 (python) S 22281 22281 22195 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 270942497 349384704 5899 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 514 18446604454222987648 0 0 -1 3 0 0 0
[pid=22327] ppid=22281 vsize=351632 CPUtime=0.12 cores=1,3,5,7
/proc/22327/stat : 22327 (python) S 22281 22281 22195 0 -1 4202560 1169 0 0 0 6 6 0 0 19 0 2 0 270932486 360071168 5948 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672488907 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/22327/statm: 87908 5948 406 364 0 23805 0
[pid=22329] ppid=22327 vsize=418636 CPUtime=98.71 cores=1,3,5,7
/proc/22329/stat : 22329 (adaptg2wsat2009) T 22327 22329 22329 0 -1 4202496 1226 0 0 0 9871 0 0 0 25 0 1 0 270932487 428683264 956 33554432000 134512640 135107411 4289090016 18446744073709551615 134531246 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/22329/statm: 104659 956 59 146 0 103535 0
[pid=22327/tid=22349] ppid=22281 vsize=351632 CPUtime=0 cores=1,3,5,7
/proc/22327/task/22349/stat : 22349 (python) S 22281 22281 22195 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 270942500 360071168 5948 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 514 18446604454222990336 0 0 -1 1 0 0 0
[pid=22341] ppid=22281 vsize=362084 CPUtime=0.13 cores=1,3,5,7
/proc/22341/stat : 22341 (python) S 22281 22281 22195 0 -1 4202560 1151 0 0 0 6 7 0 0 19 0 2 0 270942404 370774016 5995 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672488907 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/22341/statm: 90521 5995 400 364 0 26415 0
[pid=22343] ppid=22341 vsize=418636 CPUtime=99.98 cores=1,3,5,7
/proc/22343/stat : 22343 (adaptg2wsat2009) T 22341 22343 22343 0 -1 4202496 1227 0 0 0 9998 0 0 0 25 0 1 0 270942404 428683264 956 33554432000 134512640 135107411 4286854720 18446744073709551615 134531736 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/22343/statm: 104659 956 59 146 0 103535 0
[pid=22341/tid=22368] ppid=22281 vsize=362084 CPUtime=0 cores=1,3,5,7
/proc/22341/task/22368/stat : 22368 (python) S 22281 22281 22195 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 270952417 370774016 5995 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 514 18446604460957132864 0 0 -1 5 0 0 0
[pid=22345] ppid=22281 vsize=372340 CPUtime=0.13 cores=1,3,5,7
/proc/22345/stat : 22345 (python) S 22281 22281 22195 0 -1 4202560 1166 0 0 0 6 7 0 0 19 0 2 0 270942410 381276160 5999 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672488907 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/22345/statm: 93085 5999 400 364 0 28976 0
[pid=22347] ppid=22345 vsize=248652 CPUtime=99.69 cores=1,3,5,7
/proc/22347/stat : 22347 (FH) T 22345 22347 22347 0 -1 4202496 1125 0 0 0 9969 0 0 0 25 0 1 0 270942410 254619648 857 33554432000 134512640 134936988 4288544176 18446744073709551615 134554078 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/22347/statm: 62163 857 41 104 0 61667 0
[pid=22345/tid=22372] ppid=22281 vsize=372340 CPUtime=0 cores=1,3,5,7
/proc/22345/task/22372/stat : 22372 (python) S 22281 22281 22195 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 270952422 381276160 5999 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 514 18446604471162311104 0 0 -1 5 0 0 0
[pid=22350] ppid=22281 vsize=382596 CPUtime=0.58 cores=1,3,5,7
/proc/22350/stat : 22350 (python) S 22281 22281 22195 0 -1 4202560 1184 0 0 0 29 29 0 0 18 0 2 0 270942502 391778304 6017 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672488907 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/22350/statm: 95649 6017 400 364 0 31537 0
[pid=22352] ppid=22350 vsize=268428 CPUtime=390.69 cores=1,3,5,7
/proc/22352/stat : 22352 (hybridGM3) T 22350 22352 22352 0 -1 4202496 5167 0 0 0 39069 0 0 0 25 0 1 0 270942502 274870272 4897 33554432000 134512640 135204385 4294748560 18446744073709551615 134537792 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/22352/statm: 67107 4897 63 169 0 66937 0
[pid=22350/tid=22383] ppid=22281 vsize=382596 CPUtime=0 cores=1,3,5,7
/proc/22350/task/22383/stat : 22383 (python) S 22281 22281 22195 0 -1 4202560 48 0 0 0 0 0 0 0 18 0 2 0 270962733 391778304 6017 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 514 18446604471138732992 0 0 -1 7 0 0 0
[pid=22353] ppid=22281 vsize=392852 CPUtime=0.13 cores=1,3,5,7
/proc/22353/stat : 22353 (python) S 22281 22281 22195 0 -1 4202560 1162 0 0 0 7 6 0 0 19 0 2 0 270942506 402280448 6021 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672488907 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/22353/statm: 98213 6021 400 364 0 34098 0
[pid=22355] ppid=22353 vsize=1018616 CPUtime=99.15 cores=1,3,5,7
/proc/22355/stat : 22355 (gnovelty+2) T 22353 22355 22355 0 -1 4202496 779 0 0 0 9915 0 0 0 25 0 1 0 270942507 1043062784 501 33554432000 134512640 135663704 4292103856 18446744073709551615 134524918 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/22355/statm: 254654 501 113 282 0 254370 0
[pid=22353/tid=22371] ppid=22281 vsize=392852 CPUtime=0 cores=1,3,5,7
/proc/22353/task/22371/stat : 22371 (python) S 22281 22281 22195 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 270952422 402280448 6021 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 514 18446604464717339328 0 0 -1 1 0 0 0
[pid=22369] ppid=22281 vsize=403108 CPUtime=0.73 cores=1,3,5,7
/proc/22369/stat : 22369 (python) S 22281 22281 22195 0 -1 4202560 1185 0 0 0 36 37 0 0 18 0 2 0 270952421 412782592 6040 33554432000 4194304 5683644 140735173406864 18446744073709551615 268660682310 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/22369/statm: 100777 6040 400 364 0 36659 0
[pid=22373] ppid=22369 vsize=268428 CPUtime=483.82 cores=1,3,5,7
/proc/22373/stat : 22373 (hybridGM3) R 22369 22373 22373 0 -1 4202496 5164 0 0 0 48382 0 0 0 25 0 1 0 270952422 274870272 4897 33554432000 134512640 135204385 4292405616 18446744073709551615 134545486 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/22373/statm: 67107 4897 63 169 0 66937 0
[pid=22369/tid=22430] ppid=22281 vsize=403108 CPUtime=0 cores=1,3,5,7
/proc/22369/task/22430/stat : 22430 (python) S 22281 22281 22195 0 -1 4202560 46 0 0 0 0 0 0 0 18 0 2 0 270972053 412782592 6040 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 514 18446604464717338432 0 0 -1 3 0 0 0
[pid=22374] ppid=22281 vsize=413364 CPUtime=0.44 cores=1,3,5,7
/proc/22374/stat : 22374 (python) S 22281 22281 22195 0 -1 4202560 1203 0 0 0 24 20 0 0 18 0 2 0 270952427 423284736 6044 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672488907 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/22374/statm: 103341 6044 400 364 0 39220 0
[pid=22376] ppid=22374 vsize=268424 CPUtime=296.99 cores=1,3,5,7
/proc/22376/stat : 22376 (hybridGM3) T 22374 22376 22376 0 -1 4202496 5171 0 0 0 29699 0 0 0 25 0 1 0 270952427 274866176 4896 33554432000 134512640 135204385 4292067072 18446744073709551615 134538314 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/22376/statm: 67106 4896 63 169 0 66936 0
[pid=22374/tid=22431] ppid=22281 vsize=413364 CPUtime=0 cores=1,3,5,7
/proc/22374/task/22431/stat : 22431 (python) S 22281 22281 22195 0 -1 4202560 45 0 0 0 0 0 0 0 18 0 2 0 270972059 423284736 6044 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 514 18446604471110373760 0 0 -1 1 0 0 0
[pid=22377] ppid=22281 vsize=423800 CPUtime=0.73 cores=1,3,5,7
/proc/22377/stat : 22377 (python) S 22281 22281 22195 0 -1 4202560 1073 0 0 0 39 34 0 0 18 0 1 0 270952432 433971200 6040 33554432000 4194304 5683644 140735173406864 18446744073709551615 268660682310 0 0 16781312 514 14500500084598210824 0 0 17 5 0 0 0
/proc/22377/statm: 105950 6040 378 364 0 41826 0
[pid=22379] ppid=22377 vsize=457124 CPUtime=483.57 cores=1,3,5,7
/proc/22379/stat : 22379 (TNM) R 22377 22379 22379 0 -1 4202496 1112 0 0 0 48356 1 0 0 25 0 1 0 270952432 468094976 837 33554432000 134512640 135008748 4292836544 18446744073709551615 134527658 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/22379/statm: 114281 837 46 122 0 113181 0
[pid=22432] ppid=22281 vsize=434056 CPUtime=0.15 cores=1,3,5,7
/proc/22432/stat : 22432 (python) S 22281 22281 22195 0 -1 4202560 1208 0 0 0 8 7 0 0 19 0 2 0 270972255 444473344 6089 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672488907 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/22432/statm: 108514 6089 407 364 0 44387 0
[pid=22434] ppid=22432 vsize=418636 CPUtime=99.84 cores=1,3,5,7
/proc/22434/stat : 22434 (adaptg2wsat2009) T 22432 22434 22434 0 -1 4202496 1230 0 0 0 9984 0 0 0 25 0 1 0 270972255 428683264 957 33554432000 134512640 135107411 4293050736 18446744073709551615 134532135 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/22434/statm: 104659 957 59 146 0 103535 0
[pid=22432/tid=22454] ppid=22281 vsize=434056 CPUtime=0 cores=1,3,5,7
/proc/22432/task/22454/stat : 22454 (python) S 22281 22281 22195 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 270982269 444473344 6089 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 514 18446604471110375552 0 0 -1 1 0 0 0
[pid=22455] ppid=22281 vsize=444312 CPUtime=0.15 cores=1,3,5,7
/proc/22455/stat : 22455 (python) S 22281 22281 22195 0 -1 4202560 1183 0 0 0 8 7 0 0 19 0 2 0 270982274 454975488 6094 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672488907 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/22455/statm: 111078 6094 402 364 0 46948 0
[pid=22457] ppid=22455 vsize=248648 CPUtime=99.99 cores=1,3,5,7
/proc/22457/stat : 22457 (FH) T 22455 22457 22457 0 -1 4202496 1133 0 0 0 9999 0 0 0 25 0 1 0 270982274 254615552 857 33554432000 134512640 134936988 4288249328 18446744073709551615 134544965 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/22457/statm: 62162 857 41 104 0 61666 0
[pid=22455/tid=22477] ppid=22281 vsize=444312 CPUtime=0 cores=1,3,5,7
/proc/22455/task/22477/stat : 22477 (python) S 22281 22281 22195 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 270992288 454975488 6094 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 514 18446604471137240960 0 0 -1 3 0 0 0
[pid=22568] ppid=22281 vsize=454568 CPUtime=0 cores=1,3,5,7
/proc/22568/stat : 22568 (python) S 22281 22281 22195 0 -1 4202560 846 0 0 0 0 0 0 0 19 0 1 0 271001418 465477632 6054 33554432000 4194304 5683644 140735173406864 18446744073709551615 268660682310 0 0 16781312 514 4346249347248160767 0 0 17 1 0 0 0
/proc/22568/statm: 113642 6054 350 364 0 49509 0
[pid=22570] ppid=22568 vsize=1018616 CPUtime=0.72 cores=1,3,5,7
/proc/22570/stat : 22570 (gnovelty+2) R 22568 22570 22570 0 -1 4202496 774 0 0 0 72 0 0 0 22 0 1 0 271001418 1043062784 500 33554432000 134512640 135663704 4292288576 18446744073709551615 134524518 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/22570/statm: 254654 500 113 282 0 254370 0
[pid=22281/tid=22288] ppid=22279 vsize=464812 CPUtime=0 cores=1,3,5,7
/proc/22281/task/22288/stat : 22288 (python) S 22279 22281 22195 0 -1 4202560 24 933 0 0 0 0 0 0 20 0 19 0 270922464 475967488 7620 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 2 18446604471130863744 0 0 -1 7 0 0 0
[pid=22281/tid=22291] ppid=22279 vsize=464812 CPUtime=0 cores=1,3,5,7
/proc/22281/task/22291/stat : 22291 (python) S 22279 22281 22195 0 -1 4202560 21 933 0 0 0 0 0 0 22 0 19 0 270922466 475967488 7620 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 2 18446604471130863744 0 0 -1 3 0 0 0
[pid=22281/tid=22294] ppid=22279 vsize=464812 CPUtime=0 cores=1,3,5,7
/proc/22281/task/22294/stat : 22294 (python) S 22279 22281 22195 0 -1 4202560 25 933 0 0 0 0 0 0 18 0 19 0 270922469 475967488 7620 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 2 18446604471130863744 0 0 -1 1 0 0 0
[pid=22281/tid=22297] ppid=22279 vsize=464812 CPUtime=0 cores=1,3,5,7
/proc/22281/task/22297/stat : 22297 (python) S 22279 22281 22195 0 -1 4202560 32 933 0 0 0 0 0 0 18 0 19 0 270922472 475967488 7620 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 2 18446604471130863744 0 0 -1 5 0 0 0
[pid=22281/tid=22317] ppid=22279 vsize=464812 CPUtime=0 cores=1,3,5,7
/proc/22281/task/22317/stat : 22317 (python) S 22279 22281 22195 0 -1 4202560 28 933 0 0 0 0 0 0 20 0 19 0 270932387 475967488 7620 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 2 18446604471130863744 0 0 -1 7 0 0 0
[pid=22281/tid=22320] ppid=22279 vsize=464812 CPUtime=0 cores=1,3,5,7
/proc/22281/task/22320/stat : 22320 (python) S 22279 22281 22195 0 -1 4202560 30 933 0 0 0 0 0 0 20 0 19 0 270932391 475967488 7620 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 2 18446604471130863744 0 0 -1 5 0 0 0
[pid=22281/tid=22324] ppid=22279 vsize=464812 CPUtime=0 cores=1,3,5,7
/proc/22281/task/22324/stat : 22324 (python) S 22279 22281 22195 0 -1 4202560 22 933 0 0 0 0 0 0 20 0 19 0 270932483 475967488 7620 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 2 18446604471130863744 0 0 -1 3 0 0 0
[pid=22281/tid=22328] ppid=22279 vsize=464812 CPUtime=0 cores=1,3,5,7
/proc/22281/task/22328/stat : 22328 (python) S 22279 22281 22195 0 -1 4202560 29 933 0 0 0 0 0 0 20 0 19 0 270932487 475967488 7620 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 2 18446604471130863744 0 0 -1 5 0 0 0
[pid=22281/tid=22342] ppid=22279 vsize=464812 CPUtime=0 cores=1,3,5,7
/proc/22281/task/22342/stat : 22342 (python) S 22279 22281 22195 0 -1 4202560 30 933 0 0 0 0 0 0 20 0 19 0 270942404 475967488 7620 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 2 18446604471130863744 0 0 -1 5 0 0 0
[pid=22281/tid=22346] ppid=22279 vsize=464812 CPUtime=0 cores=1,3,5,7
/proc/22281/task/22346/stat : 22346 (python) S 22279 22281 22195 0 -1 4202560 28 933 0 0 0 0 0 0 21 0 19 0 270942410 475967488 7620 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 2 18446604471130863744 0 0 -1 5 0 0 0
[pid=22281/tid=22351] ppid=22279 vsize=464812 CPUtime=0 cores=1,3,5,7
/proc/22281/task/22351/stat : 22351 (python) S 22279 22281 22195 0 -1 4202560 34 933 0 0 0 0 0 0 18 0 19 0 270942502 475967488 7620 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 2 18446604471130863744 0 0 -1 5 0 0 0
[pid=22281/tid=22354] ppid=22279 vsize=464812 CPUtime=0 cores=1,3,5,7
/proc/22281/task/22354/stat : 22354 (python) S 22279 22281 22195 0 -1 4202560 28 933 0 0 0 0 0 0 20 0 19 0 270942507 475967488 7620 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 2 18446604471130863744 0 0 -1 1 0 0 0
[pid=22281/tid=22370] ppid=22279 vsize=464812 CPUtime=0 cores=1,3,5,7
/proc/22281/task/22370/stat : 22370 (python) S 22279 22281 22195 0 -1 4202560 38 933 0 0 0 0 0 0 18 0 19 0 270952421 475967488 7620 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 2 18446604471130863744 0 0 -1 3 0 0 0
[pid=22281/tid=22375] ppid=22279 vsize=464812 CPUtime=0 cores=1,3,5,7
/proc/22281/task/22375/stat : 22375 (python) S 22279 22281 22195 0 -1 4202560 25 933 0 0 0 0 0 0 18 0 19 0 270952427 475967488 7620 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 2 18446604471130863744 0 0 -1 5 0 0 0
[pid=22281/tid=22378] ppid=22279 vsize=464812 CPUtime=0 cores=1,3,5,7
/proc/22281/task/22378/stat : 22378 (python) S 22279 22281 22195 0 -1 4202560 25 933 0 0 0 0 0 0 23 0 19 0 270952432 475967488 7620 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 2 18446604471130863744 0 0 -1 5 0 0 0
[pid=22281/tid=22433] ppid=22279 vsize=464812 CPUtime=0 cores=1,3,5,7
/proc/22281/task/22433/stat : 22433 (python) S 22279 22281 22195 0 -1 4202560 24 933 0 0 0 0 0 0 20 0 19 0 270972255 475967488 7620 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 2 18446604471130863744 0 0 -1 5 0 0 0
[pid=22281/tid=22456] ppid=22279 vsize=464812 CPUtime=0 cores=1,3,5,7
/proc/22281/task/22456/stat : 22456 (python) S 22279 22281 22195 0 -1 4202560 25 933 0 0 0 0 0 0 20 0 19 0 270982274 475967488 7620 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 2 18446604471130863744 0 0 -1 1 0 0 0
[pid=22281/tid=22569] ppid=22279 vsize=464812 CPUtime=0 cores=1,3,5,7
/proc/22281/task/22569/stat : 22569 (python) S 22279 22281 22195 0 -1 4202560 24 933 0 0 0 0 0 0 20 0 19 0 271001418 475967488 7620 33554432000 4194304 5683644 140735173406864 18446744073709551615 268672485809 0 0 16781312 2 18446604471130863744 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3147.88
Current children cumulated vsize (KiB) 16694652

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 22281 and gives
#  childrusage.ru_utime.tv_sec=2
#  childrusage.ru_utime.tv_usec=275654
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=164974
# CPU time returned by wait4() is 2.44063
# while last known CPU time is 3147.88
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 792.018
CPU time (s): 3147.88
CPU user time (s): 3145.53
CPU system time (s): 2.35
CPU usage (%): 397.451
Max. virtual memory (cumulated for all children) (KiB): 16694652

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2.27565
system time used= 0.164974
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 29227
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2787
involuntary context switches= 71

runsolver used 4.48332 second user time and 9.67353 second system time

The end

Launcher Data 

Begin job on node107 at 2011-05-01 18:28:30
IDJOB=3317099
IDBENCH=82564
IDSOLVER=1705
FILE ID=node107/3317099-1304267310
RUNJOBID= node107-1304267294-22214
PBS_JOBID= 13173971
Free space on /tmp= 72624 MiB

SOLVER NAME= borg-sat 11.03.02-random
BENCH NAME= SAT11/random/large/unif-k7-r85-v200-c17000-S1348818483-020-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3317099-1304267310/watcher-3317099-1304267310 -o /tmp/evaluation-result-3317099-1304267310/solver-3317099-1304267310 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3317099-1304267310.cnf -seed 979871631 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 112b17aaf1cede32f350e11f6333fc88
RANDOM SEED=979871631

node107.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5359.47
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      30138508 kB
Buffers:        416724 kB
Cached:        1927268 kB
SwapCached:          4 kB
Active:         338640 kB
Inactive:      2114892 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      30138508 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:           67096 kB
Writeback:           0 kB
AnonPages:      109420 kB
Mapped:          19920 kB
Slab:           294660 kB
PageTables:       5368 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   256048 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72592 MiB
End job on node107 at 2011-05-01 18:41:44