Trace number 3378098

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)? (MO) 2392.61 602.831

General information on the benchmark

Namerandom/large/
unif-k7-r85-v200-c17000-S862393334-098-UNKNOWN.cnf
MD5SUM860d0404ebc2282cbe692f8828f81b2c
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark454.473
Satisfiable
(Un)Satisfiability was proved
Number of variables200
Number of clauses17000
Sum of the clauses size119000
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 517000

Solver Data 

0.36/0.67	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.36/0.67	c INFO: solving HOME/instance-3378098-1305462636.cnf
0.36/0.67	c INFO: computing task features
0.44/0.79	c INFO: computed variable-clause graph statistics
0.44/0.79	c INFO: computed clause balance statistics
1.04/1.39	c INFO: computed variable balance statistics
1.34/1.63	c INFO: computed small-clause counts
1.34/1.63	c INFO: computed Horn-clause counts
1.34/1.65	c INFO: computed variable graph statistics
1.34/1.65	c INFO: collected features for HOME/instance-3378098-1305462636.cnf in 0.98 s
1.34/1.66	c INFO: running gnovelty+2@0 for 97 with 1199 remaining (b = 0.36)
1.34/1.67	c INFO: running gnovelty+2@0 for 97 with 1199 remaining (b = 0.12)
1.34/1.68	c INFO: running adaptg2wsat2009++@0 for 97 with 1198 remaining (b = 0.08)
1.34/1.69	c INFO: running adaptg2wsat2009++@0 for 97 with 1198 remaining (b = 0.07)
393.06/99.96	c INFO: running adaptg2wsat2009++@0 for 97 with 1100 remaining (b = 0.05)
397.04/100.99	c INFO: running gnovelty+2@0 for 97 with 1099 remaining (b = 0.04)
401.03/101.94	c INFO: running FH@0 for 97 with 1098 remaining (b = 0.03)
401.03/102.00	c INFO: running adaptg2wsat2009++@0 for 97 with 1098 remaining (b = 0.03)
792.69/200.23	c INFO: running adaptg2wsat2009++@0 for 97 with 1000 remaining (b = 0.02)
796.67/201.26	c INFO: running FH@0 for 97 with 999 remaining (b = 0.02)
796.67/201.29	c INFO: running hybridGM3@0 for 194 with 999 remaining (b = 0.04)
798.80/202.21	c INFO: running gnovelty+2@0 for 97 with 998 remaining (b = 0.02)
1191.59/300.52	c INFO: running hybridGM3@0 for 194 with 900 remaining (b = 0.03)
1195.56/301.56	c INFO: running adaptg2wsat2009++@0 for 97 with 899 remaining (b = 0.02)
1199.60/302.52	c INFO: running hybridGM3@0 for 194 with 898 remaining (b = 0.03)
1594.92/401.97	c INFO: running FH@0 for 97 with 798 remaining (b = 0.01)
1598.48/402.89	c INFO: running hybridGM3@99 for 97 with 797 remaining (b = 0.01)
1973.48/497.11	c INFO: running hybridGM3@99 for 97 with 703 remaining (b = 0.01)
1977.41/498.16	c INFO: running hybridGM3@295 for 97 with 702 remaining (b = 0.01)
1993.72/502.26	c INFO: running adaptg2wsat2009++@0 for 97 with 698 remaining (b = 0.01)
1997.28/503.17	c INFO: running hybridGM3@100 for 97 with 697 remaining (b = 0.01)
2371.48/597.44	c INFO: running adaptg2wsat2009++@98 for 97 with 603 remaining (b = 0.01)
2371.48/597.47	c INFO: running hybridGM3@396 for 97 with 603 remaining (b = 0.01)
2379.88/599.56	c INFO: running TNM@0 for 601 with 601 remaining (b = 0.07)
2391.32/602.49	c INFO: running hybridGM3@0 for 194 with 598 remaining (b = 0.02)
2392.61/602.76	c INFO: running TNM@0 for 597 with 597 remaining (b = 0.06)
2392.61/602.80	c INFO: running FH@198 for 194 with 597 remaining (b = 0.02)
2392.61/602.82	c INFO: running gnovelty+2@0 for 97 with 597 remaining (b = 0.01)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3378098-1305462636/watcher-3378098-1305462636 -o /tmp/evaluation-result-3378098-1305462636/solver-3378098-1305462636 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378098-1305462636.cnf -seed 446795767 -budget 4800 -cores 4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 5.90 3.77 3.60 5/351 28249
/proc/meminfo: memFree=17238496/32951124 swapFree=67082152/67111528
[pid=28249] ppid=28247 vsize=15772 CPUtime=0 cores=1,3,5,7
/proc/28249/stat : 28249 (python) R 28247 28249 28149 0 -1 4202496 902 0 0 0 0 0 0 0 25 0 1 0 657861587 16150528 783 33554432000 4194304 5683644 140734154132272 18446744073709551615 228244344677 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/28249/statm: 3943 783 346 364 0 430 0

[startup+0.0565539 s]
/proc/loadavg: 5.90 3.77 3.60 5/351 28249
/proc/meminfo: memFree=17238496/32951124 swapFree=67082152/67111528
[pid=28249] ppid=28247 vsize=29788 CPUtime=0.02 cores=1,3,5,7
/proc/28249/stat : 28249 (python) R 28247 28249 28149 0 -1 4202496 1402 0 0 0 2 0 0 0 25 0 1 0 657861587 30502912 1246 33554432000 4194304 5683644 140734154132272 18446744073709551615 5378009 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/28249/statm: 7447 1246 443 364 0 817 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 29788

[startup+0.100552 s]
/proc/loadavg: 5.90 3.77 3.60 5/351 28249
/proc/meminfo: memFree=17238496/32951124 swapFree=67082152/67111528
[pid=28249] ppid=28247 vsize=87704 CPUtime=0.05 cores=1,3,5,7
/proc/28249/stat : 28249 (python) S 28247 28249 28149 0 -1 4202496 2637 0 0 0 4 1 0 0 17 0 1 0 657861587 89808896 2354 33554432000 4194304 5683644 140734154132272 18446744073709551615 228244346880 0 2147483391 16781312 2 18446744071699199166 0 0 17 5 0 0 0
/proc/28249/statm: 21926 2354 717 364 0 1622 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 87704

[startup+0.300526 s]
/proc/loadavg: 5.90 3.77 3.60 5/351 28249
/proc/meminfo: memFree=17238496/32951124 swapFree=67082152/67111528
[pid=28249] ppid=28247 vsize=135748 CPUtime=0.18 cores=1,3,5,7
/proc/28249/stat : 28249 (python) S 28247 28249 28149 0 -1 4202496 5966 932 1 0 13 5 0 0 17 0 1 0 657861587 139005952 4151 33554432000 4194304 5683644 140734154132272 18446744073709551615 228244344837 0 2147483391 16781312 2 18446744071699199166 0 0 17 5 0 0 0
/proc/28249/statm: 33937 4151 1032 364 0 3123 0
Current children cumulated CPU time (s) 0.18
Current children cumulated vsize (KiB) 135748

[startup+0.700466 s]
/proc/loadavg: 5.90 3.77 3.60 5/351 28249
/proc/meminfo: memFree=17238496/32951124 swapFree=67082152/67111528
[pid=28249] ppid=28247 vsize=279368 CPUtime=0.44 cores=1,3,5,7
/proc/28249/stat : 28249 (python) R 28247 28249 28149 0 -1 4202496 9425 932 1 0 32 12 0 0 15 0 1 0 657861587 286072832 7105 33554432000 4194304 5683644 140734154132272 18446744073709551615 5007621 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/28249/statm: 69842 7105 1761 364 0 5766 0
Current children cumulated CPU time (s) 0.44
Current children cumulated vsize (KiB) 279368

[startup+1.50031 s]
/proc/loadavg: 5.90 3.77 3.60 6/352 28254
/proc/meminfo: memFree=17211084/32951124 swapFree=67082152/67111528
[pid=28249] ppid=28247 vsize=282584 CPUtime=1.24 cores=1,3,5,7
/proc/28249/stat : 28249 (python) R 28247 28249 28149 0 -1 4202496 11331 932 1 0 112 12 0 0 16 0 1 0 657861587 289366016 8059 33554432000 4194304 5683644 140734154132272 18446744073709551615 228244007137 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/28249/statm: 70646 8059 1827 364 0 6570 0
Current children cumulated CPU time (s) 1.24
Current children cumulated vsize (KiB) 282584

[startup+3.10098 s]
/proc/loadavg: 6.07 3.84 3.62 9/364 28266
/proc/meminfo: memFree=17190776/32951124 swapFree=67082152/67111528
[pid=28249] ppid=28247 vsize=320376 CPUtime=1.42 cores=1,3,5,7
/proc/28249/stat : 28249 (python) S 28247 28249 28149 0 -1 4202496 14088 932 1 0 130 12 0 0 24 0 5 0 657861587 328065024 7301 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256175563 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/28249/statm: 80094 7301 1875 364 0 16003 0
[pid=28255] ppid=28249 vsize=279228 CPUtime=0 cores=1,3,5,7
/proc/28255/stat : 28255 (python) S 28249 28249 28149 0 -1 4202560 721 0 0 0 0 0 0 0 18 0 1 0 657861754 285929472 5714 33554432000 4194304 5683644 140734154132272 18446744073709551615 228244368911 0 0 16781312 514 85858118528893192 0 0 17 7 0 0 0
/proc/28255/statm: 69807 5714 354 364 0 5725 0
[pid=28257] ppid=28255 vsize=1018616 CPUtime=1.42 cores=1,3,5,7
/proc/28257/stat : 28257 (gnovelty+2) R 28255 28257 28257 0 -1 4202496 764 0 0 0 142 0 0 0 23 0 1 0 657861755 1043062784 501 33554432000 134512640 135663704 4290536096 18446744073709551615 134525265 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/28257/statm: 254654 501 113 282 0 254370 0
[pid=28258] ppid=28249 vsize=289620 CPUtime=0 cores=1,3,5,7
/proc/28258/stat : 28258 (python) S 28249 28249 28149 0 -1 4202560 732 0 0 0 0 0 0 0 18 0 1 0 657861755 296570880 5762 33554432000 4194304 5683644 140734154132272 18446744073709551615 228244368966 0 0 16781312 514 10117342339618799876 0 0 17 7 0 0 0
/proc/28258/statm: 72405 5762 348 364 0 8320 0
[pid=28260] ppid=28258 vsize=1018616 CPUtime=1.41 cores=1,3,5,7
/proc/28260/stat : 28260 (gnovelty+2) R 28258 28260 28260 0 -1 4202496 762 0 0 0 141 0 0 0 25 0 1 0 657861755 1043062784 501 33554432000 134512640 135663704 4293681728 18446744073709551615 134525276 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/28260/statm: 254654 501 113 282 0 254370 0
[pid=28261] ppid=28249 vsize=299876 CPUtime=0 cores=1,3,5,7
/proc/28261/stat : 28261 (python) S 28249 28249 28149 0 -1 4202560 734 0 0 0 0 0 0 0 18 0 1 0 657861756 307073024 5766 33554432000 4194304 5683644 140734154132272 18446744073709551615 228244368966 0 0 16781312 514 85861966819622911 0 0 17 5 0 0 0
/proc/28261/statm: 74969 5766 348 364 0 10881 0
[pid=28263] ppid=28261 vsize=418500 CPUtime=1.39 cores=1,3,5,7
/proc/28263/stat : 28263 (adaptg2wsat2009) R 28261 28263 28263 0 -1 4202496 1073 0 0 0 139 0 0 0 25 0 1 0 657861756 428544000 808 33554432000 134512640 135107411 4290845168 18446744073709551615 134523424 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/28263/statm: 104625 808 49 146 0 103501 0
[pid=28264] ppid=28249 vsize=310132 CPUtime=0 cores=1,3,5,7
/proc/28264/stat : 28264 (python) S 28249 28249 28149 0 -1 4202560 737 0 0 0 0 0 0 0 18 0 1 0 657861757 317575168 5770 33554432000 4194304 5683644 140734154132272 18446744073709551615 228244368966 0 0 16781312 514 16781529594569162751 0 0 17 5 0 0 0
/proc/28264/statm: 77533 5770 348 364 0 13442 0
[pid=28266] ppid=28264 vsize=418500 CPUtime=1.39 cores=1,3,5,7
/proc/28266/stat : 28266 (adaptg2wsat2009) R 28264 28266 28266 0 -1 4202496 1073 0 0 0 139 0 0 0 25 0 1 0 657861757 428544000 808 33554432000 134512640 135107411 4289207232 18446744073709551615 134523496 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/28266/statm: 104625 808 49 146 0 103501 0
[pid=28249/tid=28256] ppid=28247 vsize=320376 CPUtime=0 cores=1,3,5,7
/proc/28249/task/28256/stat : 28256 (python) S 28247 28249 28149 0 -1 4202560 24 932 0 0 0 0 0 0 19 0 5 0 657861754 328065024 7301 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 2 18446604465817584640 0 0 -1 7 0 0 0
[pid=28249/tid=28259] ppid=28247 vsize=320376 CPUtime=0 cores=1,3,5,7
/proc/28249/task/28259/stat : 28259 (python) S 28247 28249 28149 0 -1 4202560 22 932 0 0 0 0 0 0 20 0 5 0 657861755 328065024 7301 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 2 18446604465817584640 0 0 -1 7 0 0 0
[pid=28249/tid=28262] ppid=28247 vsize=320376 CPUtime=0 cores=1,3,5,7
/proc/28249/task/28262/stat : 28262 (python) S 28247 28249 28149 0 -1 4202560 26 932 0 0 0 0 0 0 22 0 5 0 657861756 328065024 7301 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 2 18446604465817584640 0 0 -1 5 0 0 0
[pid=28249/tid=28265] ppid=28247 vsize=320376 CPUtime=0 cores=1,3,5,7
/proc/28249/task/28265/stat : 28265 (python) S 28247 28249 28149 0 -1 4202560 29 932 0 0 0 0 0 0 24 0 5 0 657861757 328065024 7301 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 2 18446604465817584640 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 7.03
Current children cumulated vsize (KiB) 4373464

[startup+6.30142 s]
/proc/loadavg: 6.07 3.84 3.62 9/364 28266
/proc/meminfo: memFree=17188344/32951124 swapFree=67082152/67111528
[pid=28249] ppid=28247 vsize=320376 CPUtime=1.42 cores=1,3,5,7
/proc/28249/stat : 28249 (python) S 28247 28249 28149 0 -1 4202496 14088 932 1 0 130 12 0 0 24 0 5 0 657861587 328065024 7301 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256175563 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/28249/statm: 80094 7301 1875 364 0 16003 0
[pid=28255] ppid=28249 vsize=279228 CPUtime=0.01 cores=1,3,5,7
/proc/28255/stat : 28255 (python) S 28249 28249 28149 0 -1 4202560 878 0 0 0 1 0 0 0 18 0 1 0 657861754 285929472 5737 33554432000 4194304 5683644 140734154132272 18446744073709551615 228244368911 0 0 16781312 514 85858118528893192 0 0 17 7 0 0 0
/proc/28255/statm: 69807 5737 377 364 0 5725 0
[pid=28257] ppid=28255 vsize=1018616 CPUtime=4.62 cores=1,3,5,7
/proc/28257/stat : 28257 (gnovelty+2) R 28255 28257 28257 0 -1 4202496 764 0 0 0 461 1 0 0 25 0 1 0 657861755 1043062784 501 33554432000 134512640 135663704 4290536096 18446744073709551615 134524894 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/28257/statm: 254654 501 113 282 0 254370 0
[pid=28258] ppid=28249 vsize=289620 CPUtime=0 cores=1,3,5,7
/proc/28258/stat : 28258 (python) S 28249 28249 28149 0 -1 4202560 884 0 0 0 0 0 0 0 18 0 1 0 657861755 296570880 5785 33554432000 4194304 5683644 140734154132272 18446744073709551615 228244368966 0 0 16781312 514 10117342339618799876 0 0 17 7 0 0 0
/proc/28258/statm: 72405 5785 371 364 0 8320 0
[pid=28260] ppid=28258 vsize=1018616 CPUtime=4.61 cores=1,3,5,7
/proc/28260/stat : 28260 (gnovelty+2) R 28258 28260 28260 0 -1 4202496 762 0 0 0 461 0 0 0 25 0 1 0 657861755 1043062784 501 33554432000 134512640 135663704 4293681728 18446744073709551615 134524868 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/28260/statm: 254654 501 113 282 0 254370 0
[pid=28261] ppid=28249 vsize=299876 CPUtime=0 cores=1,3,5,7
/proc/28261/stat : 28261 (python) S 28249 28249 28149 0 -1 4202560 887 0 0 0 0 0 0 0 18 0 1 0 657861756 307073024 5789 33554432000 4194304 5683644 140734154132272 18446744073709551615 228244368966 0 0 16781312 514 85861966819590408 0 0 17 5 0 0 0
/proc/28261/statm: 74969 5789 371 364 0 10881 0
[pid=28263] ppid=28261 vsize=418636 CPUtime=4.52 cores=1,3,5,7
/proc/28263/stat : 28263 (adaptg2wsat2009) R 28261 28263 28263 0 -1 4202496 1220 0 0 0 452 0 0 0 25 0 1 0 657861756 428683264 955 33554432000 134512640 135107411 4290845168 18446744073709551615 134531561 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/28263/statm: 104659 955 59 146 0 103535 0
[pid=28264] ppid=28249 vsize=310132 CPUtime=0.01 cores=1,3,5,7
/proc/28264/stat : 28264 (python) S 28249 28249 28149 0 -1 4202560 895 0 0 0 0 1 0 0 18 0 1 0 657861757 317575168 5793 33554432000 4194304 5683644 140734154132272 18446744073709551615 228244368966 0 0 16781312 514 16781529594569162751 0 0 17 5 0 0 0
/proc/28264/statm: 77533 5793 371 364 0 13442 0
[pid=28266] ppid=28264 vsize=418636 CPUtime=4.56 cores=1,3,5,7
/proc/28266/stat : 28266 (adaptg2wsat2009) R 28264 28266 28266 0 -1 4202496 1220 0 0 0 456 0 0 0 25 0 1 0 657861757 428683264 955 33554432000 134512640 135107411 4289207232 18446744073709551615 134532277 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/28266/statm: 104659 955 59 146 0 103535 0
[pid=28249/tid=28256] ppid=28247 vsize=320376 CPUtime=0 cores=1,3,5,7
/proc/28249/task/28256/stat : 28256 (python) S 28247 28249 28149 0 -1 4202560 24 932 0 0 0 0 0 0 19 0 5 0 657861754 328065024 7301 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 2 18446604465817584640 0 0 -1 7 0 0 0
[pid=28249/tid=28259] ppid=28247 vsize=320376 CPUtime=0 cores=1,3,5,7
/proc/28249/task/28259/stat : 28259 (python) S 28247 28249 28149 0 -1 4202560 22 932 0 0 0 0 0 0 20 0 5 0 657861755 328065024 7301 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 2 18446604465817584640 0 0 -1 7 0 0 0
[pid=28249/tid=28262] ppid=28247 vsize=320376 CPUtime=0 cores=1,3,5,7
/proc/28249/task/28262/stat : 28262 (python) S 28247 28249 28149 0 -1 4202560 26 932 0 0 0 0 0 0 22 0 5 0 657861756 328065024 7301 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 2 18446604465817584640 0 0 -1 5 0 0 0
[pid=28249/tid=28265] ppid=28247 vsize=320376 CPUtime=0 cores=1,3,5,7
/proc/28249/task/28265/stat : 28265 (python) S 28247 28249 28149 0 -1 4202560 29 932 0 0 0 0 0 0 24 0 5 0 657861757 328065024 7301 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 2 18446604465817584640 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 19.75
Current children cumulated vsize (KiB) 4373736

[startup+12.7013 s]
/proc/loadavg: 6.45 3.99 3.67 9/364 28266
/proc/meminfo: memFree=17186464/32951124 swapFree=67082152/67111528
[pid=28249] ppid=28247 vsize=320376 CPUtime=1.42 cores=1,3,5,7
/proc/28249/stat : 28249 (python) S 28247 28249 28149 0 -1 4202496 14088 932 1 0 130 12 0 0 24 0 5 0 657861587 328065024 7301 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256175563 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/28249/statm: 80094 7301 1875 364 0 16003 0
[pid=28255] ppid=28249 vsize=279228 CPUtime=0.02 cores=1,3,5,7
/proc/28255/stat : 28255 (python) S 28249 28249 28149 0 -1 4202560 881 0 0 0 1 1 0 0 18 0 1 0 657861754 285929472 5737 33554432000 4194304 5683644 140734154132272 18446744073709551615 228244368911 0 0 16781312 514 180601381931942152 0 0 17 7 0 0 0
/proc/28255/statm: 69807 5737 377 364 0 5725 0
[pid=28257] ppid=28255 vsize=1018616 CPUtime=11.02 cores=1,3,5,7
/proc/28257/stat : 28257 (gnovelty+2) R 28255 28257 28257 0 -1 4202496 764 0 0 0 1101 1 0 0 25 0 1 0 657861755 1043062784 501 33554432000 134512640 135663704 4290536096 18446744073709551615 134524894 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/28257/statm: 254654 501 113 282 0 254370 0
[pid=28258] ppid=28249 vsize=289620 CPUtime=0.02 cores=1,3,5,7
/proc/28258/stat : 28258 (python) S 28249 28249 28149 0 -1 4202560 889 0 0 0 1 1 0 0 18 0 1 0 657861755 296570880 5785 33554432000 4194304 5683644 140734154132272 18446744073709551615 228244368966 0 0 16781312 514 180601381931942148 0 0 17 7 0 0 0
/proc/28258/statm: 72405 5785 371 364 0 8320 0
[pid=28260] ppid=28258 vsize=1018616 CPUtime=11.01 cores=1,3,5,7
/proc/28260/stat : 28260 (gnovelty+2) R 28258 28260 28260 0 -1 4202496 762 0 0 0 1101 0 0 0 25 0 1 0 657861755 1043062784 501 33554432000 134512640 135663704 4293681728 18446744073709551615 134545607 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/28260/statm: 254654 501 113 282 0 254370 0
[pid=28261] ppid=28249 vsize=299876 CPUtime=0.02 cores=1,3,5,7
/proc/28261/stat : 28261 (python) S 28249 28249 28149 0 -1 4202560 891 0 0 0 1 1 0 0 18 0 1 0 657861756 307073024 5789 33554432000 4194304 5683644 140734154132272 18446744073709551615 228244368966 0 0 16781312 514 180601381931942152 0 0 17 5 0 0 0
/proc/28261/statm: 74969 5789 371 364 0 10881 0
[pid=28263] ppid=28261 vsize=418636 CPUtime=10.82 cores=1,3,5,7
/proc/28263/stat : 28263 (adaptg2wsat2009) R 28261 28263 28263 0 -1 4202496 1220 0 0 0 1082 0 0 0 25 0 1 0 657861756 428683264 955 33554432000 134512640 135107411 4290845168 18446744073709551615 134531781 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/28263/statm: 104659 955 59 146 0 103535 0
[pid=28264] ppid=28249 vsize=310132 CPUtime=0.02 cores=1,3,5,7
/proc/28264/stat : 28264 (python) S 28249 28249 28149 0 -1 4202560 899 0 0 0 1 1 0 0 18 0 1 0 657861757 317575168 5793 33554432000 4194304 5683644 140734154132272 18446744073709551615 228244368966 0 0 16781312 514 180601381931942151 0 0 17 5 0 0 0
/proc/28264/statm: 77533 5793 371 364 0 13442 0
[pid=28266] ppid=28264 vsize=418636 CPUtime=10.93 cores=1,3,5,7
/proc/28266/stat : 28266 (adaptg2wsat2009) R 28264 28266 28266 0 -1 4202496 1220 0 0 0 1093 0 0 0 25 0 1 0 657861757 428683264 955 33554432000 134512640 135107411 4289207232 18446744073709551615 134531416 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/28266/statm: 104659 955 59 146 0 103535 0
[pid=28249/tid=28256] ppid=28247 vsize=320376 CPUtime=0 cores=1,3,5,7
/proc/28249/task/28256/stat : 28256 (python) S 28247 28249 28149 0 -1 4202560 24 932 0 0 0 0 0 0 19 0 5 0 657861754 328065024 7301 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 2 18446604465817584640 0 0 -1 7 0 0 0
[pid=28249/tid=28259] ppid=28247 vsize=320376 CPUtime=0 cores=1,3,5,7
/proc/28249/task/28259/stat : 28259 (python) S 28247 28249 28149 0 -1 4202560 22 932 0 0 0 0 0 0 20 0 5 0 657861755 328065024 7301 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 2 18446604465817584640 0 0 -1 7 0 0 0
[pid=28249/tid=28262] ppid=28247 vsize=320376 CPUtime=0 cores=1,3,5,7
/proc/28249/task/28262/stat : 28262 (python) S 28247 28249 28149 0 -1 4202560 26 932 0 0 0 0 0 0 22 0 5 0 657861756 328065024 7301 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 2 18446604465817584640 0 0 -1 5 0 0 0
[pid=28249/tid=28265] ppid=28247 vsize=320376 CPUtime=0 cores=1,3,5,7
/proc/28249/task/28265/stat : 28265 (python) S 28247 28249 28149 0 -1 4202560 29 932 0 0 0 0 0 0 24 0 5 0 657861757 328065024 7301 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 2 18446604465817584640 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 45.28
Current children cumulated vsize (KiB) 4373736

[startup+25.5011 s]
/proc/loadavg: 6.46 4.07 3.70 9/364 28311
/proc/meminfo: memFree=17101200/32951124 swapFree=67082152/67111528
[pid=28249] ppid=28247 vsize=320376 CPUtime=1.42 cores=1,3,5,7

################
# More data... #
################

[pid=28261/tid=28410] ppid=28249 vsize=299876 CPUtime=0 cores=1,3,5,7
/proc/28261/task/28410/stat : 28410 (python) S 28249 28249 28149 0 -1 4202560 49 0 0 0 0 0 0 0 18 0 2 0 657871781 307073024 5819 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 514 18446604465817586432 0 0 -1 1 0 0 0
[pid=28264] ppid=28249 vsize=310132 CPUtime=0.27 cores=1,3,5,7
/proc/28264/stat : 28264 (python) S 28249 28249 28149 0 -1 4202560 1091 0 0 0 12 15 0 0 18 0 2 0 657861757 317575168 5823 33554432000 4194304 5683644 140734154132272 18446744073709551615 228244368966 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/28264/statm: 77533 5823 400 364 0 13442 0
[pid=28266] ppid=28264 vsize=418636 CPUtime=105.07 cores=1,3,5,7
/proc/28266/stat : 28266 (adaptg2wsat2009) R 28264 28266 28266 0 -1 4202496 1220 0 0 0 10507 0 0 0 25 0 1 0 657861757 428683264 955 33554432000 134512640 135107411 4289207232 18446744073709551615 134531604 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/28266/statm: 104659 955 59 146 0 103535 0
[pid=28264/tid=28414] ppid=28249 vsize=310132 CPUtime=0 cores=1,3,5,7
/proc/28264/task/28414/stat : 28414 (python) S 28249 28249 28149 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 657871786 317575168 5823 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 514 18446604469704849472 0 0 -1 5 0 0 0
[pid=28403] ppid=28249 vsize=320520 CPUtime=0.26 cores=1,3,5,7
/proc/28403/stat : 28403 (python) S 28249 28249 28149 0 -1 4202560 1091 0 0 0 13 13 0 0 19 0 2 0 657871584 328212480 5844 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256175563 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/28403/statm: 80130 5844 400 364 0 16036 0
[pid=28405] ppid=28403 vsize=418636 CPUtime=100.22 cores=1,3,5,7
/proc/28405/stat : 28405 (adaptg2wsat2009) T 28403 28405 28405 0 -1 4202496 1222 0 0 0 10022 0 0 0 25 0 1 0 657871584 428683264 955 33554432000 134512640 135107411 4292795072 18446744073709551615 134532277 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/28405/statm: 104659 955 59 146 0 103535 0
[pid=28403/tid=28464] ppid=28249 vsize=320520 CPUtime=0 cores=1,3,5,7
/proc/28403/task/28464/stat : 28464 (python) S 28249 28249 28149 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 657881610 328212480 5844 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 514 18446604461711012096 0 0 -1 1 0 0 0
[pid=28407] ppid=28249 vsize=331316 CPUtime=0.27 cores=1,3,5,7
/proc/28407/stat : 28407 (python) S 28249 28249 28149 0 -1 4202560 1077 0 0 0 16 11 0 0 19 0 2 0 657871687 339267584 5983 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256175563 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/28407/statm: 82829 5983 399 364 0 18732 0
[pid=28409] ppid=28407 vsize=1018616 CPUtime=100.25 cores=1,3,5,7
/proc/28409/stat : 28409 (gnovelty+2) T 28407 28409 28409 0 -1 4202496 763 0 0 0 10025 0 0 0 25 0 1 0 657871687 1043062784 500 33554432000 134512640 135663704 4290473328 18446744073709551615 134524882 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/28409/statm: 254654 500 113 282 0 254370 0
[pid=28407/tid=28471] ppid=28249 vsize=331316 CPUtime=0 cores=1,3,5,7
/proc/28407/task/28471/stat : 28471 (python) S 28249 28249 28149 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 657881716 339267584 5983 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 514 18446604461711012992 0 0 -1 5 0 0 0
[pid=28411] ppid=28249 vsize=341572 CPUtime=0.27 cores=1,3,5,7
/proc/28411/stat : 28411 (python) S 28249 28249 28149 0 -1 4202560 1094 0 0 0 15 12 0 0 19 0 2 0 657871782 349769728 5987 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256175563 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/28411/statm: 85393 5987 399 364 0 21293 0
[pid=28413] ppid=28411 vsize=248648 CPUtime=99.22 cores=1,3,5,7
/proc/28413/stat : 28413 (FH) T 28411 28413 28413 0 -1 4202496 1120 0 0 0 9922 0 0 0 25 0 1 0 657871782 254615552 857 33554432000 134512640 134936988 4288212816 18446744073709551615 134552408 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/28413/statm: 62162 857 41 104 0 61666 0
[pid=28411/tid=28476] ppid=28249 vsize=341572 CPUtime=0 cores=1,3,5,7
/proc/28411/task/28476/stat : 28476 (python) S 28249 28249 28149 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 657881808 349769728 5987 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 514 18446604471115966656 0 0 -1 1 0 0 0
[pid=28415] ppid=28249 vsize=351828 CPUtime=0.25 cores=1,3,5,7
/proc/28415/stat : 28415 (python) S 28249 28249 28149 0 -1 4202560 1123 0 0 0 12 13 0 0 19 0 2 0 657871787 360271872 5997 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256175563 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/28415/statm: 87957 5997 405 364 0 23854 0
[pid=28417] ppid=28415 vsize=418636 CPUtime=97.72 cores=1,3,5,7
/proc/28417/stat : 28417 (adaptg2wsat2009) T 28415 28417 28417 0 -1 4202496 1226 0 0 0 9772 0 0 0 25 0 1 0 657871788 428683264 955 33554432000 134512640 135107411 4291594608 18446744073709551615 134532128 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/28417/statm: 104659 955 59 146 0 103535 0
[pid=28415/tid=28468] ppid=28249 vsize=351828 CPUtime=0 cores=1,3,5,7
/proc/28415/task/28468/stat : 28468 (python) S 28249 28249 28149 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 657881713 360271872 5997 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 514 18446604470223229056 0 0 -1 7 0 0 0
[pid=28465] ppid=28249 vsize=362084 CPUtime=0.27 cores=1,3,5,7
/proc/28465/stat : 28465 (python) S 28249 28249 28149 0 -1 4202560 1136 0 0 0 14 13 0 0 19 0 2 0 657881611 370774016 6001 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256175563 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/28465/statm: 90521 6001 406 364 0 26415 0
[pid=28467] ppid=28465 vsize=418636 CPUtime=99.95 cores=1,3,5,7
/proc/28467/stat : 28467 (adaptg2wsat2009) T 28465 28467 28467 0 -1 4202496 1227 0 0 0 9995 0 0 0 25 0 1 0 657881611 428683264 957 33554432000 134512640 135107411 4290577296 18446744073709551615 134532036 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/28467/statm: 104659 957 59 146 0 103535 0
[pid=28465/tid=28499] ppid=28249 vsize=362084 CPUtime=0 cores=1,3,5,7
/proc/28465/task/28499/stat : 28499 (python) S 28249 28249 28149 0 -1 4202560 51 0 0 0 0 0 0 0 19 0 2 0 657891638 370774016 6001 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 514 18446604471095813440 0 0 -1 1 0 0 0
[pid=28469] ppid=28249 vsize=372340 CPUtime=0.27 cores=1,3,5,7
/proc/28469/stat : 28469 (python) S 28249 28249 28149 0 -1 4202560 1112 0 0 0 13 14 0 0 19 0 2 0 657881714 381276160 5999 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256175563 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/28469/statm: 93085 5999 400 364 0 28976 0
[pid=28472] ppid=28469 vsize=248648 CPUtime=99.98 cores=1,3,5,7
/proc/28472/stat : 28472 (FH) T 28469 28472 28472 0 -1 4202496 1122 0 0 0 9998 0 0 0 25 0 1 0 657881716 254615552 856 33554432000 134512640 134936988 4290625328 18446744073709551615 134544944 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/28472/statm: 62162 856 41 104 0 61666 0
[pid=28469/tid=28503] ppid=28249 vsize=372340 CPUtime=0 cores=1,3,5,7
/proc/28469/task/28503/stat : 28503 (python) S 28249 28249 28149 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 657891742 381276160 5999 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 514 18446604457268712704 0 0 -1 3 0 0 0
[pid=28473] ppid=28249 vsize=382596 CPUtime=1.14 cores=1,3,5,7
/proc/28473/stat : 28473 (python) S 28249 28249 28149 0 -1 4202560 1176 0 0 0 61 53 0 0 18 0 2 0 657881717 391778304 6003 33554432000 4194304 5683644 140734154132272 18446744073709551615 228244368966 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/28473/statm: 95649 6003 400 364 0 31537 0
[pid=28475] ppid=28473 vsize=268420 CPUtime=396.03 cores=1,3,5,7
/proc/28475/stat : 28475 (hybridGM3) R 28473 28475 28475 0 -1 4202496 5165 0 0 0 39603 0 0 0 25 0 1 0 657881717 274862080 4896 33554432000 134512640 135204385 4287873840 18446744073709551615 134538016 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/28475/statm: 67105 4896 63 169 0 66935 0
[pid=28473/tid=28525] ppid=28249 vsize=382596 CPUtime=0 cores=1,3,5,7
/proc/28473/task/28525/stat : 28525 (python) S 28249 28249 28149 0 -1 4202560 51 0 0 0 0 0 0 0 18 0 2 0 657901876 391778304 6003 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 514 18446604457995578304 0 0 -1 5 0 0 0
[pid=28477] ppid=28249 vsize=392852 CPUtime=0.27 cores=1,3,5,7
/proc/28477/stat : 28477 (python) S 28249 28249 28149 0 -1 4202560 1123 0 0 0 14 13 0 0 19 0 2 0 657881809 402280448 6007 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256175563 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/28477/statm: 98213 6007 400 364 0 34098 0
[pid=28479] ppid=28477 vsize=1018616 CPUtime=99.76 cores=1,3,5,7
/proc/28479/stat : 28479 (gnovelty+2) T 28477 28479 28479 0 -1 4202496 767 0 0 0 9976 0 0 0 25 0 1 0 657881809 1043062784 501 33554432000 134512640 135663704 4286585152 18446744073709551615 134524857 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/28479/statm: 254654 501 113 282 0 254370 0
[pid=28477/tid=28507] ppid=28249 vsize=392852 CPUtime=0 cores=1,3,5,7
/proc/28477/task/28507/stat : 28507 (python) S 28249 28249 28149 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 657891838 402280448 6007 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 514 18446604467864254656 0 0 -1 1 0 0 0
[pid=28500] ppid=28249 vsize=403108 CPUtime=0.86 cores=1,3,5,7
/proc/28500/stat : 28500 (python) S 28249 28249 28149 0 -1 4202560 1170 0 0 0 43 43 0 0 18 0 2 0 657891640 412782592 6011 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256175563 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/28500/statm: 100777 6011 400 364 0 36659 0
[pid=28502] ppid=28500 vsize=268420 CPUtime=296.85 cores=1,3,5,7
/proc/28502/stat : 28502 (hybridGM3) T 28500 28502 28502 0 -1 4202496 5166 0 0 0 29685 0 0 0 25 0 1 0 657891640 274862080 4895 33554432000 134512640 135204385 4293750736 18446744073709551615 134538349 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/28502/statm: 67105 4895 63 169 0 66935 0
[pid=28500/tid=28541] ppid=28249 vsize=403108 CPUtime=0 cores=1,3,5,7
/proc/28500/task/28541/stat : 28541 (python) S 28249 28249 28149 0 -1 4202560 47 0 0 0 0 0 0 0 18 0 2 0 657911298 412782592 6011 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 514 18446604467864255552 0 0 -1 5 0 0 0
[pid=28504] ppid=28249 vsize=413364 CPUtime=0.28 cores=1,3,5,7
/proc/28504/stat : 28504 (python) S 28249 28249 28149 0 -1 4202560 1141 0 0 0 14 14 0 0 19 0 2 0 657891744 423284736 6015 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256175563 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/28504/statm: 103341 6015 400 364 0 39220 0
[pid=28506] ppid=28504 vsize=418636 CPUtime=100.35 cores=1,3,5,7
/proc/28506/stat : 28506 (adaptg2wsat2009) T 28504 28506 28506 0 -1 4202496 1228 0 0 0 10035 0 0 0 25 0 1 0 657891744 428683264 957 33554432000 134512640 135107411 4288286720 18446744073709551615 134531935 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/28506/statm: 104659 957 59 146 0 103535 0
[pid=28504/tid=28521] ppid=28249 vsize=413364 CPUtime=0 cores=1,3,5,7
/proc/28504/task/28521/stat : 28521 (python) S 28249 28249 28149 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 657901783 423284736 6015 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 514 18446604458852389312 0 0 -1 1 0 0 0
[pid=28508] ppid=28249 vsize=423620 CPUtime=0.88 cores=1,3,5,7
/proc/28508/stat : 28508 (python) S 28249 28249 28149 0 -1 4202560 1226 0 0 0 45 43 0 0 18 0 2 0 657891840 433786880 6039 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256175563 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/28508/statm: 105905 6039 407 364 0 41781 0
[pid=28510] ppid=28508 vsize=268420 CPUtime=291.91 cores=1,3,5,7
/proc/28510/stat : 28510 (hybridGM3) T 28508 28510 28510 0 -1 4202496 5171 0 0 0 29190 1 0 0 25 0 1 0 657891840 274862080 4895 33554432000 134512640 135204385 4289961776 18446744073709551615 134538541 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/28510/statm: 67105 4895 63 169 0 66935 0
[pid=28508/tid=28543] ppid=28249 vsize=423620 CPUtime=0 cores=1,3,5,7
/proc/28508/task/28543/stat : 28543 (python) S 28249 28249 28149 0 -1 4202560 46 0 0 0 0 0 0 0 18 0 2 0 657911904 433786880 6039 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 514 18446604458852391104 0 0 -1 5 0 0 0
[pid=28522] ppid=28249 vsize=433876 CPUtime=0.28 cores=1,3,5,7
/proc/28522/stat : 28522 (python) S 28249 28249 28149 0 -1 4202560 1186 0 0 0 14 14 0 0 19 0 2 0 657901784 444289024 6044 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256175563 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/28522/statm: 108469 6044 408 364 0 44342 0
[pid=28524] ppid=28522 vsize=248652 CPUtime=100.23 cores=1,3,5,7
/proc/28524/stat : 28524 (FH) T 28522 28524 28524 0 -1 4202496 1134 0 0 0 10023 0 0 0 25 0 1 0 657901785 254619648 856 33554432000 134512640 134936988 4290915840 18446744073709551615 134545264 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/28524/statm: 62163 856 41 104 0 61667 0
[pid=28522/tid=28542] ppid=28249 vsize=433876 CPUtime=0 cores=1,3,5,7
/proc/28522/task/28542/stat : 28542 (python) S 28249 28249 28149 0 -1 4202560 52 0 0 0 0 0 0 0 19 0 2 0 657911812 444289024 6044 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 514 18446604469206954368 0 0 -1 5 0 0 0
[pid=28559] ppid=28249 vsize=444304 CPUtime=0 cores=1,3,5,7
/proc/28559/stat : 28559 (python) S 28249 28249 28149 0 -1 4202560 854 0 0 0 0 0 0 0 18 0 1 0 657921543 454967296 6024 33554432000 4194304 5683644 140734154132272 18446744073709551615 228244368966 0 0 16781312 514 14643094102632005895 0 0 17 5 0 0 0
/proc/28559/statm: 111076 6024 355 364 0 46946 0
[pid=28561] ppid=28559 vsize=457124 CPUtime=3.13 cores=1,3,5,7
/proc/28561/stat : 28561 (TNM) R 28559 28561 28561 0 -1 4202496 1112 0 0 0 313 0 0 0 25 0 1 0 657921544 468094976 837 33554432000 134512640 135008748 4289379888 18446744073709551615 134529612 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/28561/statm: 114281 837 46 122 0 113181 0
[pid=28562] ppid=28249 vsize=454560 CPUtime=0 cores=1,3,5,7
/proc/28562/stat : 28562 (python) S 28249 28249 28149 0 -1 4202560 835 0 0 0 0 0 0 0 19 0 1 0 657921837 465469440 6023 33554432000 4194304 5683644 140734154132272 18446744073709551615 228244368966 0 0 16781312 514 15969494902601940991 0 0 17 5 0 0 0
/proc/28562/statm: 113640 6023 350 364 0 49507 0
[pid=28564] ppid=28562 vsize=268420 CPUtime=0.19 cores=1,3,5,7
/proc/28564/stat : 28564 (hybridGM3) R 28562 28564 28564 0 -1 4202496 5171 0 0 0 19 0 0 0 21 0 1 0 657921837 274862080 4896 33554432000 134512640 135204385 4294271760 18446744073709551615 134538541 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/28564/statm: 67105 4896 63 169 0 66935 0
[pid=28249/tid=28256] ppid=28247 vsize=464804 CPUtime=0 cores=1,3,5,7
/proc/28249/task/28256/stat : 28256 (python) S 28247 28249 28149 0 -1 4202560 24 932 0 0 0 0 0 0 19 0 19 0 657861754 475959296 7589 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 2 18446604465817584640 0 0 -1 7 0 0 0
[pid=28249/tid=28259] ppid=28247 vsize=464804 CPUtime=0 cores=1,3,5,7
/proc/28249/task/28259/stat : 28259 (python) S 28247 28249 28149 0 -1 4202560 22 932 0 0 0 0 0 0 20 0 19 0 657861755 475959296 7589 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 2 18446604465817584640 0 0 -1 7 0 0 0
[pid=28249/tid=28262] ppid=28247 vsize=464804 CPUtime=0 cores=1,3,5,7
/proc/28249/task/28262/stat : 28262 (python) S 28247 28249 28149 0 -1 4202560 32 932 0 0 0 0 0 0 18 0 19 0 657861756 475959296 7589 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 2 18446604465817584640 0 0 -1 5 0 0 0
[pid=28249/tid=28265] ppid=28247 vsize=464804 CPUtime=0 cores=1,3,5,7
/proc/28249/task/28265/stat : 28265 (python) S 28247 28249 28149 0 -1 4202560 34 932 0 0 0 0 0 0 18 0 19 0 657861757 475959296 7589 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 2 18446604465817584640 0 0 -1 5 0 0 0
[pid=28249/tid=28404] ppid=28247 vsize=464804 CPUtime=0 cores=1,3,5,7
/proc/28249/task/28404/stat : 28404 (python) S 28247 28249 28149 0 -1 4202560 29 932 0 0 0 0 0 0 20 0 19 0 657871584 475959296 7589 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 2 18446604465817584640 0 0 -1 5 0 0 0
[pid=28249/tid=28408] ppid=28247 vsize=464804 CPUtime=0 cores=1,3,5,7
/proc/28249/task/28408/stat : 28408 (python) S 28247 28249 28149 0 -1 4202560 27 932 0 0 0 0 0 0 20 0 19 0 657871687 475959296 7589 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 2 18446604465817584640 0 0 -1 5 0 0 0
[pid=28249/tid=28412] ppid=28247 vsize=464804 CPUtime=0 cores=1,3,5,7
/proc/28249/task/28412/stat : 28412 (python) S 28247 28249 28149 0 -1 4202560 27 932 0 0 0 0 0 0 20 0 19 0 657871782 475959296 7589 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 2 18446604465817584640 0 0 -1 5 0 0 0
[pid=28249/tid=28416] ppid=28247 vsize=464804 CPUtime=0 cores=1,3,5,7
/proc/28249/task/28416/stat : 28416 (python) S 28247 28249 28149 0 -1 4202560 28 932 0 0 0 0 0 0 21 0 19 0 657871788 475959296 7589 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 2 18446604465817584640 0 0 -1 5 0 0 0
[pid=28249/tid=28466] ppid=28247 vsize=464804 CPUtime=0 cores=1,3,5,7
/proc/28249/task/28466/stat : 28466 (python) S 28247 28249 28149 0 -1 4202560 21 932 0 0 0 0 0 0 20 0 19 0 657881611 475959296 7589 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 2 18446604465817584640 0 0 -1 5 0 0 0
[pid=28249/tid=28470] ppid=28247 vsize=464804 CPUtime=0 cores=1,3,5,7
/proc/28249/task/28470/stat : 28470 (python) S 28247 28249 28149 0 -1 4202560 21 932 0 0 0 0 0 0 20 0 19 0 657881714 475959296 7589 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 2 18446604465817584640 0 0 -1 3 0 0 0
[pid=28249/tid=28474] ppid=28247 vsize=464804 CPUtime=0 cores=1,3,5,7
/proc/28249/task/28474/stat : 28474 (python) S 28247 28249 28149 0 -1 4202560 30 932 0 0 0 0 0 0 18 0 19 0 657881717 475959296 7589 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 2 18446604465817584640 0 0 -1 5 0 0 0
[pid=28249/tid=28478] ppid=28247 vsize=464804 CPUtime=0 cores=1,3,5,7
/proc/28249/task/28478/stat : 28478 (python) S 28247 28249 28149 0 -1 4202560 25 932 0 0 0 0 0 0 20 0 19 0 657881809 475959296 7589 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 2 18446604465817584640 0 0 -1 5 0 0 0
[pid=28249/tid=28501] ppid=28247 vsize=464804 CPUtime=0 cores=1,3,5,7
/proc/28249/task/28501/stat : 28501 (python) S 28247 28249 28149 0 -1 4202560 34 932 0 0 0 0 0 0 18 0 19 0 657891640 475959296 7589 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 2 18446604465817584640 0 0 -1 5 0 0 0
[pid=28249/tid=28505] ppid=28247 vsize=464804 CPUtime=0 cores=1,3,5,7
/proc/28249/task/28505/stat : 28505 (python) S 28247 28249 28149 0 -1 4202560 28 932 0 0 0 0 0 0 20 0 19 0 657891744 475959296 7589 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 2 18446604465817584640 0 0 -1 5 0 0 0
[pid=28249/tid=28509] ppid=28247 vsize=464804 CPUtime=0 cores=1,3,5,7
/proc/28249/task/28509/stat : 28509 (python) S 28247 28249 28149 0 -1 4202560 28 932 0 0 0 0 0 0 18 0 19 0 657891840 475959296 7589 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 2 18446604465817584640 0 0 -1 5 0 0 0
[pid=28249/tid=28523] ppid=28247 vsize=464804 CPUtime=0 cores=1,3,5,7
/proc/28249/task/28523/stat : 28523 (python) S 28247 28249 28149 0 -1 4202560 27 932 0 0 0 0 0 0 20 0 19 0 657901785 475959296 7589 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 2 18446604465817584640 0 0 -1 5 0 0 0
[pid=28249/tid=28560] ppid=28247 vsize=464804 CPUtime=0 cores=1,3,5,7
/proc/28249/task/28560/stat : 28560 (python) S 28247 28249 28149 0 -1 4202560 29 932 0 0 0 0 0 0 20 0 19 0 657921544 475959296 7589 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 2 18446604465817584640 0 0 -1 5 0 0 0
[pid=28249/tid=28563] ppid=28247 vsize=464804 CPUtime=0 cores=1,3,5,7
/proc/28249/task/28563/stat : 28563 (python) S 28247 28249 28149 0 -1 4202560 27 932 0 0 0 0 0 0 20 0 19 0 657921837 475959296 7589 33554432000 4194304 5683644 140734154132272 18446744073709551615 228256172465 0 0 16781312 2 18446604465817584640 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 2392.61
Current children cumulated vsize (KiB) 15944876

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 28249 and gives
#  childrusage.ru_utime.tv_sec=509
#  childrusage.ru_utime.tv_usec=726509
#  childrusage.ru_stime.tv_sec=3
#  childrusage.ru_stime.tv_usec=511466
# CPU time returned by wait4() is 513.238
# while last known CPU time is 2392.61
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 602.831
CPU time (s): 2392.61
CPU user time (s): 2389.18
CPU system time (s): 3.43
CPU usage (%): 396.895
Max. virtual memory (cumulated for all children) (KiB): 15944876

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 509.727
system time used= 3.51147
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 62641
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 6248
involuntary context switches= 3508

runsolver used 4.16237 second user time and 8.49471 second system time

The end

Launcher Data 

Begin job on node136 at 2011-05-15 14:30:36
IDJOB=3378098
IDBENCH=82507
IDSOLVER=1859
FILE ID=node136/3378098-1305462636
RUNJOBID= node136-1305462567-28168
PBS_JOBID= 13324968
Free space on /tmp= 73028 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/large/unif-k7-r85-v200-c17000-S862393334-098-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3378098-1305462636/watcher-3378098-1305462636 -o /tmp/evaluation-result-3378098-1305462636/solver-3378098-1305462636 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378098-1305462636.cnf -seed 446795767 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 860d0404ebc2282cbe692f8828f81b2c
RANDOM SEED=446795767

node136.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      17238776 kB
Buffers:        312728 kB
Cached:        5885716 kB
SwapCached:       2740 kB
Active:       10963332 kB
Inactive:      4401484 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      17238776 kB
SwapTotal:    67111528 kB
SwapFree:     67082152 kB
Dirty:           32220 kB
Writeback:           0 kB
AnonPages:     9164456 kB
Mapped:          34148 kB
Slab:           239836 kB
PageTables:      42536 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 72988928 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72996 MiB
End job on node136 at 2011-05-15 14:40:41