Trace number 3378163

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)? (MO) 4316.84 1091.24

General information on the benchmark

Namerandom/large/
unif-k3-r4.2-v35000-c147000-S950666065-093-UNKNOWN.cnf
MD5SUMe27af9a147b2d7c0283d090128805575
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark919.622
Satisfiable
(Un)Satisfiability was proved
Number of variables34999
Number of clauses147000
Sum of the clauses size441000
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3147000
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.37/0.64	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.37/0.64	c INFO: solving HOME/instance-3378163-1305463282.cnf
0.37/0.64	c INFO: computing task features
0.87/1.15	c INFO: computed variable-clause graph statistics
0.87/1.17	c INFO: computed clause balance statistics
3.56/3.82	c INFO: computed variable balance statistics
5.76/6.04	c INFO: computed small-clause counts
5.76/6.05	c INFO: computed Horn-clause counts
7.36/7.61	c INFO: computed variable graph statistics
7.36/7.61	c INFO: collected features for HOME/instance-3378163-1305463282.cnf in 6.96 s
7.36/7.62	c INFO: running hybridGM3@0 for 97 with 1193 remaining (b = 0.34)
7.36/7.63	c INFO: running hybridGM3@0 for 97 with 1193 remaining (b = 0.21)
7.36/7.64	c INFO: running hybridGM3@0 for 97 with 1193 remaining (b = 0.14)
7.36/7.65	c INFO: running hybridGM3@0 for 194 with 1192 remaining (b = 0.18)
407.46/108.04	c INFO: running hybridGM3@0 for 97 with 1092 remaining (b = 0.08)
407.46/108.08	c INFO: running hybridGM3@0 for 97 with 1092 remaining (b = 0.08)
407.86/108.10	c INFO: running hybridGM3@0 for 97 with 1092 remaining (b = 0.08)
787.46/203.31	c INFO: running hybridGM3@100 for 97 with 997 remaining (b = 0.08)
787.46/203.35	c INFO: running hybridGM3@99 for 97 with 997 remaining (b = 0.08)
787.46/203.37	c INFO: running TNM@0 for 97 with 997 remaining (b = 0.10)
791.44/204.36	c INFO: running hybridGM3@94 for 97 with 996 remaining (b = 0.07)
1187.40/303.67	c INFO: running hybridGM3@99 for 97 with 896 remaining (b = 0.07)
1190.95/304.59	c INFO: running hybridGM3@0 for 97 with 896 remaining (b = 0.07)
1191.35/304.65	c INFO: running hybridGM3@0 for 194 with 895 remaining (b = 0.10)
1193.39/305.65	c INFO: running hybridGM3@0 for 194 with 894 remaining (b = 0.07)
1590.61/405.08	c INFO: running hybridGM3@99 for 97 with 795 remaining (b = 0.03)
1598.18/406.98	c INFO: running hybridGM3@0 for 194 with 793 remaining (b = 0.05)
1973.59/501.35	c INFO: running hybridGM3@101 for 97 with 699 remaining (b = 0.02)
1977.56/502.35	c INFO: running hybridGM3@100 for 97 with 698 remaining (b = 0.02)
1993.47/506.38	c INFO: running TNM@0 for 97 with 694 remaining (b = 0.02)
2356.58/597.65	c INFO: running hybridGM3@0 for 194 with 602 remaining (b = 0.04)
2376.45/602.68	c INFO: running hybridGM3@297 for 97 with 597 remaining (b = 0.02)
2380.84/603.72	c INFO: running TNM@194 for 388 with 596 remaining (b = 0.06)
2388.79/605.73	c INFO: running TNM@0 for 97 with 594 remaining (b = 0.02)
2755.42/697.98	c INFO: running hybridGM3@0 for 194 with 502 remaining (b = 0.03)
2787.58/706.08	c INFO: running hybridGM3@0 for 194 with 494 remaining (b = 0.03)
3139.04/794.41	c INFO: running hybridGM3@98 for 97 with 406 remaining (b = 0.01)
3537.78/894.75	c INFO: running hybridGM3@100 for 97 with 305 remaining (b = 0.01)
3541.74/895.73	c INFO: running hybridGM3@293 for 97 with 304 remaining (b = 0.01)
3570.40/902.90	c INFO: running TNM@0 for 97 with 297 remaining (b = 0.01)
3919.85/991.03	c INFO: running TNM@0 for 97 with 209 remaining (b = 0.01)
3939.63/996.07	c INFO: running hybridGM3@0 for 204 with 204 remaining (b = 0.02)
3959.07/1000.94	c INFO: running TNM@0 for 97 with 199 remaining (b = 0.01)
3968.21/1003.24	c INFO: running TNM@0 for 97 with 197 remaining (b = 0.01)
4313.60/1090.38	c INFO: running TNM@0 for 110 with 110 remaining (b = 0.01)
4316.84/1091.23	c INFO: running TNM@0 for 109 with 109 remaining (b = 0.01)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3378163-1305463282/watcher-3378163-1305463282 -o /tmp/evaluation-result-3378163-1305463282/solver-3378163-1305463282 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378163-1305463282.cnf -seed 2015938851 -budget 4800 -cores 4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.83 3.01 3.53 3/356 14485
/proc/meminfo: memFree=17079756/32951124 swapFree=67056804/67111528
[pid=14485] ppid=14483 vsize=15780 CPUtime=0 cores=1,3,5,7
/proc/14485/stat : 14485 (python) R 14483 14485 14447 0 -1 4202496 879 0 0 0 0 0 0 0 17 0 1 0 657929174 16158720 761 33554432000 4194304 5683644 140734449211568 18446744073709551615 270612771525 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/14485/statm: 3945 763 343 364 0 432 0

[startup+0.0735359 s]
/proc/loadavg: 1.83 3.01 3.53 3/356 14485
/proc/meminfo: memFree=17079756/32951124 swapFree=67056804/67111528
[pid=14485] ppid=14483 vsize=54540 CPUtime=0.05 cores=1,3,5,7
/proc/14485/stat : 14485 (python) S 14483 14485 14447 0 -1 4202496 2049 0 0 0 4 1 0 0 17 0 1 0 657929174 55848960 1808 33554432000 4194304 5683644 140734449211568 18446744073709551615 270613108736 0 2147483391 16781312 2 18446744071699199166 0 0 17 1 0 0 0
/proc/14485/statm: 13635 1808 492 364 0 1285 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 54540

[startup+0.100511 s]
/proc/loadavg: 1.83 3.01 3.53 3/356 14485
/proc/meminfo: memFree=17079756/32951124 swapFree=67056804/67111528
[pid=14485] ppid=14483 vsize=101744 CPUtime=0.06 cores=1,3,5,7
/proc/14485/stat : 14485 (python) S 14483 14485 14447 0 -1 4202496 2930 0 0 0 5 1 0 0 17 0 1 0 657929174 104185856 2626 33554432000 4194304 5683644 140734449211568 18446744073709551615 270613108736 0 2147483391 16781312 2 18446744071699199166 0 0 17 1 0 0 0
/proc/14485/statm: 25437 2627 837 364 0 1737 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 101744

[startup+0.300497 s]
/proc/loadavg: 1.83 3.01 3.53 3/356 14485
/proc/meminfo: memFree=17079756/32951124 swapFree=67056804/67111528
[pid=14485] ppid=14483 vsize=135752 CPUtime=0.2 cores=1,3,5,7
/proc/14485/stat : 14485 (python) R 14483 14485 14447 0 -1 4202496 5966 935 1 0 14 6 0 0 17 0 1 0 657929174 139010048 4151 33554432000 4194304 5683644 140734449211568 18446744073709551615 270613106693 0 2147483391 16781312 2 0 0 0 17 1 0 0 0
/proc/14485/statm: 33938 4151 1032 364 0 3124 0
Current children cumulated CPU time (s) 0.2
Current children cumulated vsize (KiB) 135752

[startup+0.70044 s]
/proc/loadavg: 1.83 3.01 3.53 3/356 14485
/proc/meminfo: memFree=17079756/32951124 swapFree=67056804/67111528
[pid=14485] ppid=14483 vsize=284340 CPUtime=0.45 cores=1,3,5,7
/proc/14485/stat : 14485 (python) R 14483 14485 14447 0 -1 4202496 10709 935 1 0 33 12 0 0 16 0 1 0 657929174 291164160 8389 33554432000 4194304 5683644 140734449211568 18446744073709551615 270612771525 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/14485/statm: 71085 8392 1761 364 0 7009 0
Current children cumulated CPU time (s) 0.45
Current children cumulated vsize (KiB) 284340

[startup+1.50026 s]
/proc/loadavg: 1.83 3.01 3.53 3/357 14494
/proc/meminfo: memFree=17002736/32951124 swapFree=67056804/67111528
[pid=14485] ppid=14483 vsize=312976 CPUtime=1.25 cores=1,3,5,7
/proc/14485/stat : 14485 (python) R 14483 14485 14447 0 -1 4202496 21969 935 1 0 112 13 0 0 16 0 1 0 657929174 320487424 15628 33554432000 4194304 5683644 140734449211568 18446744073709551615 270612769177 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/14485/statm: 78244 15628 1826 364 0 14168 0
Current children cumulated CPU time (s) 1.25
Current children cumulated vsize (KiB) 312976

[startup+3.10098 s]
/proc/loadavg: 1.83 3.01 3.53 3/357 14494
/proc/meminfo: memFree=16992212/32951124 swapFree=67056804/67111528
[pid=14485] ppid=14483 vsize=312976 CPUtime=2.86 cores=1,3,5,7
/proc/14485/stat : 14485 (python) R 14483 14485 14447 0 -1 4202496 21969 935 1 0 273 13 0 0 19 0 1 0 657929174 320487424 15628 33554432000 4194304 5683644 140734449211568 18446744073709551615 270612769922 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/14485/statm: 78244 15628 1826 364 0 14168 0
Current children cumulated CPU time (s) 2.86
Current children cumulated vsize (KiB) 312976

[startup+6.30041 s]
/proc/loadavg: 1.84 2.99 3.53 6/369 14507
/proc/meminfo: memFree=16891684/32951124 swapFree=67056804/67111528
[pid=14485] ppid=14483 vsize=312976 CPUtime=6.06 cores=1,3,5,7
/proc/14485/stat : 14485 (python) R 14483 14485 14447 0 -1 4202496 21976 935 1 0 593 13 0 0 25 0 1 0 657929174 320487424 15635 33554432000 4194304 5683644 140734449211568 18446744073709551615 46930513766528 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/14485/statm: 78244 15635 1833 364 0 14168 0
Current children cumulated CPU time (s) 6.06
Current children cumulated vsize (KiB) 312976

[startup+12.7013 s]
/proc/loadavg: 2.34 3.07 3.55 9/381 14520
/proc/meminfo: memFree=16752068/32951124 swapFree=67056804/67111528
[pid=14485] ppid=14483 vsize=416816 CPUtime=7.41 cores=1,3,5,7
/proc/14485/stat : 14485 (python) S 14483 14485 14447 0 -1 4202496 24700 935 1 0 727 14 0 0 19 0 5 0 657929174 426819584 15189 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624937419 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/14485/statm: 104204 15189 1881 364 0 40113 0
[pid=14508] ppid=14485 vsize=310268 CPUtime=0.01 cores=1,3,5,7
/proc/14508/stat : 14508 (python) S 14485 14485 14447 0 -1 4202560 894 0 0 0 0 1 0 0 16 0 1 0 657929936 317714432 13495 33554432000 4194304 5683644 140734449211568 18446744073709551615 270613130767 0 0 16781312 514 4485984630754738440 0 0 17 5 0 0 0
/proc/14508/statm: 77567 13495 376 364 0 13485 0
[pid=14512] ppid=14508 vsize=268428 CPUtime=4.99 cores=1,3,5,7
/proc/14512/stat : 14512 (hybridGM3) R 14508 14512 14512 0 -1 4202496 8094 0 0 0 498 1 0 0 25 0 1 0 657929938 274870272 7825 33554432000 134512640 135204385 4292650816 18446744073709551615 134536744 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/14512/statm: 67107 7825 64 169 0 66937 0
[pid=14510] ppid=14485 vsize=320524 CPUtime=0.01 cores=1,3,5,7
/proc/14510/stat : 14510 (python) S 14485 14485 14447 0 -1 4202560 920 0 0 0 0 1 0 0 16 0 1 0 657929937 328216576 13493 33554432000 4194304 5683644 140734449211568 18446744073709551615 270613130822 0 0 16781312 514 5297398648408080647 0 0 17 1 0 0 0
/proc/14510/statm: 80131 13493 370 364 0 16046 0
[pid=14515] ppid=14510 vsize=268428 CPUtime=5.04 cores=1,3,5,7
/proc/14515/stat : 14515 (hybridGM3) R 14510 14515 14515 0 -1 4202496 8095 0 0 0 503 1 0 0 25 0 1 0 657929939 274870272 7831 33554432000 134512640 135204385 4293695664 18446744073709551615 134537249 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/14515/statm: 67107 7831 64 169 0 66937 0
[pid=14513] ppid=14485 vsize=330780 CPUtime=0.01 cores=1,3,5,7
/proc/14513/stat : 14513 (python) S 14485 14485 14447 0 -1 4202560 913 0 0 0 0 1 0 0 18 0 1 0 657929938 338718720 13497 33554432000 4194304 5683644 140734449211568 18446744073709551615 270613130822 0 0 16781312 514 2693533837119422726 0 0 17 5 0 0 0
/proc/14513/statm: 82695 13497 370 364 0 18607 0
[pid=14517] ppid=14513 vsize=268424 CPUtime=5.01 cores=1,3,5,7
/proc/14517/stat : 14517 (hybridGM3) R 14513 14517 14517 0 -1 4202496 8085 0 0 0 500 1 0 0 25 0 1 0 657929940 274866176 7827 33554432000 134512640 135204385 4292387984 18446744073709551615 134536806 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/14517/statm: 67106 7827 64 169 0 66936 0
[pid=14516] ppid=14485 vsize=406572 CPUtime=0.02 cores=1,3,5,7
/proc/14516/stat : 14516 (python) S 14485 14485 14447 0 -1 4202560 783 0 0 0 1 1 0 0 16 0 1 0 657929940 416329728 13675 33554432000 4194304 5683644 140734449211568 18446744073709551615 270613130822 0 0 16781312 514 15261479783058342149 0 0 17 1 0 0 0
/proc/14516/statm: 101643 13675 371 364 0 37552 0
[pid=14520] ppid=14516 vsize=268428 CPUtime=4.97 cores=1,3,5,7
/proc/14520/stat : 14520 (hybridGM3) R 14516 14520 14520 0 -1 4202496 8075 0 0 0 497 0 0 0 25 0 1 0 657929940 274870272 7822 33554432000 134512640 135204385 4292728704 18446744073709551615 134537167 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/14520/statm: 67107 7822 64 169 0 66937 0
[pid=14485/tid=14509] ppid=14483 vsize=416816 CPUtime=0 cores=1,3,5,7
/proc/14485/task/14509/stat : 14509 (python) S 14483 14485 14447 0 -1 4202560 38 935 0 0 0 0 0 0 18 0 5 0 657929937 426819584 15189 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 2 18446604464099614784 0 0 -1 5 0 0 0
[pid=14485/tid=14511] ppid=14483 vsize=416816 CPUtime=0 cores=1,3,5,7
/proc/14485/task/14511/stat : 14511 (python) S 14483 14485 14447 0 -1 4202560 36 935 0 0 0 0 0 0 25 0 5 0 657929938 426819584 15189 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 2 18446604464099614784 0 0 -1 5 0 0 0
[pid=14485/tid=14514] ppid=14483 vsize=416816 CPUtime=0 cores=1,3,5,7
/proc/14485/task/14514/stat : 14514 (python) S 14483 14485 14447 0 -1 4202560 21 935 0 0 0 0 0 0 25 0 5 0 657929939 426819584 15189 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 2 18446604464099614784 0 0 -1 5 0 0 0
[pid=14485/tid=14519] ppid=14483 vsize=416816 CPUtime=0 cores=1,3,5,7
/proc/14485/task/14519/stat : 14519 (python) S 14483 14485 14447 0 -1 4202560 60 935 0 0 0 0 0 0 20 0 5 0 657929940 426819584 15189 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 2 18446604464099614784 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 27.47
Current children cumulated vsize (KiB) 2858668

[startup+25.501 s]
/proc/loadavg: 3.59 3.31 3.62 9/381 14520
/proc/meminfo: memFree=16752428/32951124 swapFree=67056804/67111528
[pid=14485] ppid=14483 vsize=416816 CPUtime=7.41 cores=1,3,5,7
/proc/14485/stat : 14485 (python) S 14483 14485 14447 0 -1 4202496 24700 935 1 0 727 14 0 0 19 0 5 0 657929174 426819584 15189 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624937419 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/14485/statm: 104204 15189 1881 364 0 40113 0
[pid=14508] ppid=14485 vsize=310268 CPUtime=0.04 cores=1,3,5,7
/proc/14508/stat : 14508 (python) S 14485 14485 14447 0 -1 4202560 902 0 0 0 2 2 0 0 15 0 1 0 657929936 317714432 13495 33554432000 4194304 5683644 140734449211568 18446744073709551615 270613130767 0 0 16781312 514 15677223816923152648 0 0 17 5 0 0 0
/proc/14508/statm: 77567 13495 376 364 0 13485 0
[pid=14512] ppid=14508 vsize=268428 CPUtime=17.62 cores=1,3,5,7
/proc/14512/stat : 14512 (hybridGM3) R 14508 14512 14512 0 -1 4202496 8094 0 0 0 1761 1 0 0 25 0 1 0 657929938 274870272 7825 33554432000 134512640 135204385 4292650816 18446744073709551615 134537403 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/14512/statm: 67107 7825 64 169 0 66937 0
[pid=14510] ppid=14485 vsize=320524 CPUtime=0.04 cores=1,3,5,7
/proc/14510/stat : 14510 (python) S 14485 14485 14447 0 -1 4202560 926 0 0 0 2 2 0 0 15 0 1 0 657929937 328216576 13493 33554432000 4194304 5683644 140734449211568 18446744073709551615 270613130822 0 0 16781312 514 15677223816923152647 0 0 17 1 0 0 0
/proc/14510/statm: 80131 13493 370 364 0 16046 0
[pid=14515] ppid=14510 vsize=268428 CPUtime=17.84 cores=1,3,5,7
/proc/14515/stat : 14515 (hybridGM3) R 14510 14515 14515 0 -1 4202496 8095 0 0 0 1783 1 0 0 25 0 1 0 657929939 274870272 7831 33554432000 134512640 135204385 4293695664 18446744073709551615 134537510 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/14515/statm: 67107 7831 64 169 0 66937 0
[pid=14513] ppid=14485 vsize=330780 CPUtime=0.05 cores=1,3,5,7
/proc/14513/stat : 14513 (python) S 14485 14485 14447 0 -1 4202560 916 0 0 0 2 3 0 0 18 0 1 0 657929938 338718720 13497 33554432000 4194304 5683644 140734449211568 18446744073709551615 270613130822 0 0 16781312 514 15677223816923152646 0 0 17 5 0 0 0
/proc/14513/statm: 82695 13497 370 364 0 18607 0
[pid=14517] ppid=14513 vsize=268424 CPUtime=17.74 cores=1,3,5,7
/proc/14517/stat : 14517 (hybridGM3) R 14513 14517 14517 0 -1 4202496 8085 0 0 0 1773 1 0 0 25 0 1 0 657929940 274866176 7827 33554432000 134512640 135204385 4292387984 18446744073709551615 134537435 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/14517/statm: 67106 7827 64 169 0 66936 0
[pid=14516] ppid=14485 vsize=406572 CPUtime=0.05 cores=1,3,5,7
/proc/14516/stat : 14516 (python) S 14485 14485 14447 0 -1 4202560 788 0 0 0 3 2 0 0 15 0 1 0 657929940 416329728 13675 33554432000 4194304 5683644 140734449211568 18446744073709551615 270613130822 0 0 16781312 514 15677223816923152645 0 0 17 1 0 0 0
/proc/14516/statm: 101643 13675 371 364 0 37552 0
[pid=14520] ppid=14516 vsize=268428 CPUtime=17.71 cores=1,3,5,7
/proc/14520/stat : 14520 (hybridGM3) R 14516 14520 14520 0 -1 4202496 8075 0 0 0 1771 0 0 0 25 0 1 0 657929940 274870272 7822 33554432000 134512640 135204385 4292728704 18446744073709551615 134546089 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/14520/statm: 67107 7822 64 169 0 66937 0
[pid=14485/tid=14509] ppid=14483 vsize=416816 CPUtime=0 cores=1,3,5,7
/proc/14485/task/14509/stat : 14509 (python) S 14483 14485 14447 0 -1 4202560 38 935 0 0 0 0 0 0 18 0 5 0 657929937 426819584 15189 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 2 18446604464099614784 0 0 -1 5 0 0 0
[pid=14485/tid=14511] ppid=14483 vsize=416816 CPUtime=0 cores=1,3,5,7
/proc/14485/task/14511/stat : 14511 (python) S 14483 14485 14447 0 -1 4202560 36 935 0 0 0 0 0 0 25 0 5 0 657929938 426819584 15189 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 2 18446604464099614784 0 0 -1 5 0 0 0
[pid=14485/tid=14514] ppid=14483 vsize=416816 CPUtime=0 cores=1,3,5,7
/proc/14485/task/14514/stat : 14514 (python) S 14483 14485 14447 0 -1 4202560 21 935 0 0 0 0 0 0 25 0 5 0 657929939 426819584 15189 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 2 18446604464099614784 0 0 -1 5 0 0 0
[pid=14485/tid=14519] ppid=14483 vsize=416816 CPUtime=0 cores=1,3,5,7
/proc/14485/task/14519/stat : 14519 (python) S 14483 14485 14447 0 -1 4202560 60 935 0 0 0 0 0 0 20 0 5 0 657929940 426819584 15189 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 2 18446604464099614784 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 78.5
Current children cumulated vsize (KiB) 2858668

[startup+51.1015 s]
/proc/loadavg: 5.55 3.79 3.77 6/369 14607
/proc/meminfo: memFree=16865780/32951124 swapFree=67056804/67111528
[pid=14485] ppid=14483 vsize=416816 CPUtime=7.41 cores=1,3,5,7
/proc/14485/stat : 14485 (python) S 14483 14485 14447 0 -1 4202496 24700 935 1 0 727 14 0 0 19 0 5 0 657929174 426819584 15189 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624937419 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/14485/statm: 104204 15189 1881 364 0 40113 0
[pid=14508] ppid=14485 vsize=310268 CPUtime=0.11 cores=1,3,5,7
/proc/14508/stat : 14508 (python) S 14485 14485 14447 0 -1 4202560 911 0 0 0 6 5 0 0 15 0 1 0 657929936 317714432 13495 33554432000 4194304 5683644 140734449211568 18446744073709551615 270613130767 0 0 16781312 514 15677223816923152648 0 0 17 5 0 0 0
/proc/14508/statm: 77567 13495 376 364 0 13485 0
[pid=14512] ppid=14508 vsize=268428 CPUtime=42.91 cores=1,3,5,7
/proc/14512/stat : 14512 (hybridGM3) R 14508 14512 14512 0 -1 4202496 8094 0 0 0 4290 1 0 0 25 0 1 0 657929938 274870272 7825 33554432000 134512640 135204385 4292650816 18446744073709551615 134536777 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/14512/statm: 67107 7825 64 169 0 66937 0
[pid=14510] ppid=14485 vsize=320524 CPUtime=0.12 cores=1,3,5,7
/proc/14510/stat : 14510 (python) S 14485 14485 14447 0 -1 4202560 936 0 0 0 6 6 0 0 15 0 1 0 657929937 328216576 13493 33554432000 4194304 5683644 140734449211568 18446744073709551615 270613130822 0 0 16781312 514 15677223816923152647 0 0 17 1 0 0 0
/proc/14510/statm: 80131 13493 370 364 0 16046 0
[pid=14515] ppid=14510 vsize=268428 CPUtime=43.42 cores=1,3,5,7
/proc/14515/stat : 14515 (hybridGM3) R 14510 14515 14515 0 -1 4202496 8095 0 0 0 4341 1 0 0 25 0 1 0 657929939 274870272 7831 33554432000 134512640 135204385 4293695664 18446744073709551615 134537292 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/14515/statm: 67107 7831 64 169 0 66937 0
[pid=14513] ppid=14485 vsize=330780 CPUtime=0.11 cores=1,3,5,7
/proc/14513/stat : 14513 (python) S 14485 14485 14447 0 -1 4202560 921 0 0 0 6 5 0 0 18 0 1 0 657929938 338718720 13497 33554432000 4194304 5683644 140734449211568 18446744073709551615 270613130822 0 0 16781312 514 15677223816923152646 0 0 17 5 0 0 0
/proc/14513/statm: 82695 13497 370 364 0 18607 0
[pid=14517] ppid=14513 vsize=268424 CPUtime=43.21 cores=1,3,5,7
/proc/14517/stat : 14517 (hybridGM3) R 14513 14517 14517 0 -1 4202496 8085 0 0 0 4320 1 0 0 25 0 1 0 657929940 274866176 7827 33554432000 134512640 135204385 4292387984 18446744073709551615 134537154 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/14517/statm: 67106 7827 64 169 0 66936 0
[pid=14516] ppid=14485 vsize=406572 CPUtime=0.11 cores=1,3,5,7
/proc/14516/stat : 14516 (python) S 14485 14485 14447 0 -1 4202560 788 0 0 0 6 5 0 0 15 0 1 0 657929940 416329728 13675 33554432000 4194304 5683644 140734449211568 18446744073709551615 270613130822 0 0 16781312 514 15677223816923152645 0 0 17 1 0 0 0
/proc/14516/statm: 101643 13675 371 364 0 37552 0

################
# More data... #
################

[pid=14643/tid=14666] ppid=14485 vsize=416828 CPUtime=0 cores=1,3,5,7
/proc/14643/task/14666/stat : 14666 (python) S 14485 14485 14447 0 -1 4202560 77 0 0 0 0 0 0 0 18 0 2 0 657959540 426831872 13668 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 514 18446604462129762368 0 0 -1 3 0 0 0
[pid=14667] ppid=14485 vsize=427084 CPUtime=0.6 cores=1,3,5,7
/proc/14667/stat : 14667 (python) S 14485 14485 14447 0 -1 4202560 990 0 0 0 33 27 0 0 15 0 2 0 657959633 437334016 13667 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624937419 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/14667/statm: 106771 13667 399 364 0 42674 0
[pid=14669] ppid=14667 vsize=268424 CPUtime=199.13 cores=1,3,5,7
/proc/14669/stat : 14669 (hybridGM3) T 14667 14669 14669 0 -1 4202496 8080 0 0 0 19912 1 0 0 25 0 1 0 657959634 274866176 7825 33554432000 134512640 135204385 4294050320 18446744073709551615 134536663 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/14669/statm: 67106 7825 64 169 0 66936 0
[pid=14667/tid=14690] ppid=14485 vsize=427084 CPUtime=0 cores=1,3,5,7
/proc/14667/task/14690/stat : 14690 (python) S 14485 14485 14447 0 -1 4202560 74 0 0 0 0 0 0 0 15 0 2 0 657969679 437334016 13667 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 514 18446604470561596672 0 0 -1 3 0 0 0
[pid=14670] ppid=14485 vsize=437340 CPUtime=1.19 cores=1,3,5,7
/proc/14670/stat : 14670 (python) S 14485 14485 14447 0 -1 4202560 1004 0 0 0 66 53 0 0 16 0 2 0 657959640 447836160 13811 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624937419 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/14670/statm: 109335 13811 399 364 0 45235 0
[pid=14672] ppid=14670 vsize=268424 CPUtime=389.53 cores=1,3,5,7
/proc/14672/stat : 14672 (hybridGM3) T 14670 14672 14672 0 -1 4202496 8088 0 0 0 38952 1 0 0 25 0 1 0 657959640 274866176 7828 33554432000 134512640 135204385 4292423072 18446744073709551615 134537403 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/14672/statm: 67106 7828 64 169 0 66936 0
[pid=14670/tid=14702] ppid=14485 vsize=437340 CPUtime=0 cores=1,3,5,7
/proc/14670/task/14702/stat : 14702 (python) S 14485 14485 14447 0 -1 4202560 70 0 0 0 0 0 0 0 15 0 2 0 657979309 447836160 13811 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 514 18446604470561598464 0 0 -1 3 0 0 0
[pid=14673] ppid=14485 vsize=447596 CPUtime=0.88 cores=1,3,5,7
/proc/14673/stat : 14673 (python) S 14485 14485 14447 0 -1 4202560 1043 0 0 0 55 33 0 0 15 0 2 0 657959739 458338304 13702 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624937419 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/14673/statm: 111899 13702 405 364 0 47796 0
[pid=14675] ppid=14673 vsize=268424 CPUtime=291.38 cores=1,3,5,7
/proc/14675/stat : 14675 (hybridGM3) T 14673 14675 14675 0 -1 4202496 8090 0 0 0 29136 2 0 0 25 0 1 0 657959739 274866176 7826 33554432000 134512640 135204385 4286797264 18446744073709551615 134537383 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/14675/statm: 67106 7826 64 169 0 66936 0
[pid=14673/tid=14703] ppid=14485 vsize=447596 CPUtime=0 cores=1,3,5,7
/proc/14673/task/14703/stat : 14703 (python) S 14485 14485 14447 0 -1 4202560 75 0 0 0 0 0 0 0 15 0 2 0 657979408 458338304 13702 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 514 18446604463387537792 0 0 -1 3 0 0 0
[pid=14691] ppid=14485 vsize=457852 CPUtime=0.6 cores=1,3,5,7
/proc/14691/stat : 14691 (python) S 14485 14485 14447 0 -1 4202560 1046 0 0 0 33 27 0 0 17 0 2 0 657969873 468840448 13732 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624937419 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/14691/statm: 114463 13732 405 364 0 50357 0
[pid=14693] ppid=14691 vsize=268424 CPUtime=195.6 cores=1,3,5,7
/proc/14693/stat : 14693 (hybridGM3) T 14691 14693 14693 0 -1 4202496 8087 0 0 0 19559 1 0 0 25 0 1 0 657969873 274866176 7823 33554432000 134512640 135204385 4293924848 18446744073709551615 134537331 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/14693/statm: 67106 7823 64 169 0 66936 0
[pid=14691/tid=14716] ppid=14485 vsize=457852 CPUtime=0 cores=1,3,5,7
/proc/14691/task/14716/stat : 14716 (python) S 14485 14485 14447 0 -1 4202560 79 0 0 0 0 0 0 0 17 0 2 0 657989545 468840448 13732 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 514 18446604469192407232 0 0 -1 3 0 0 0
[pid=14704] ppid=14485 vsize=468108 CPUtime=0.32 cores=1,3,5,7
/proc/14704/stat : 14704 (python) S 14485 14485 14447 0 -1 4202560 1032 0 0 0 17 15 0 0 19 0 2 0 657979812 479342592 13758 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624937419 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/14704/statm: 117027 13758 399 364 0 52918 0
[pid=14706] ppid=14704 vsize=465044 CPUtime=99.32 cores=1,3,5,7
/proc/14706/stat : 14706 (TNM) T 14704 14706 14706 0 -1 4202496 6013 0 0 0 9931 1 0 0 25 0 1 0 657979812 476205056 5750 33554432000 134512640 135008748 4287033840 18446744073709551615 134529480 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/14706/statm: 116261 5750 46 122 0 115161 0
[pid=14704/tid=14717] ppid=14485 vsize=468108 CPUtime=0 cores=1,3,5,7
/proc/14704/task/14717/stat : 14717 (python) S 14485 14485 14447 0 -1 4202560 76 0 0 0 0 0 0 0 19 0 2 0 657989746 479342592 13758 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 514 18446604463387538688 0 0 -1 7 0 0 0
[pid=14712] ppid=14485 vsize=478364 CPUtime=1.23 cores=1,3,5,7
/proc/14712/stat : 14712 (python) S 14485 14485 14447 0 -1 4202560 1042 0 0 0 63 60 0 0 16 0 2 0 657988939 489844736 13790 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624937419 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/14712/statm: 119591 13790 399 364 0 55479 0
[pid=14714] ppid=14712 vsize=268424 CPUtime=390.94 cores=1,3,5,7
/proc/14714/stat : 14714 (hybridGM3) T 14712 14714 14714 0 -1 4202496 8094 0 0 0 39092 2 0 0 25 0 1 0 657988940 274866176 7828 33554432000 134512640 135204385 4289174192 18446744073709551615 134536929 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/14714/statm: 67106 7828 64 169 0 66936 0
[pid=14712/tid=14801] ppid=14485 vsize=478364 CPUtime=0 cores=1,3,5,7
/proc/14712/task/14801/stat : 14801 (python) S 14485 14485 14447 0 -1 4202560 77 0 0 0 0 0 0 0 15 0 2 0 658008614 489844736 13790 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 514 18446604467742495680 0 0 -1 3 0 0 0
[pid=14718] ppid=14485 vsize=488620 CPUtime=0.31 cores=1,3,5,7
/proc/14718/stat : 14718 (python) S 14485 14485 14447 0 -1 4202560 1051 0 0 0 17 14 0 0 19 0 2 0 657989747 500346880 13770 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624937419 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/14718/statm: 122155 13770 399 364 0 58040 0
[pid=14720] ppid=14718 vsize=465044 CPUtime=100.32 cores=1,3,5,7
/proc/14720/stat : 14720 (TNM) T 14718 14720 14720 0 -1 4202496 6018 0 0 0 10031 1 0 0 25 0 1 0 657989747 476205056 5749 33554432000 134512640 135008748 4293275120 18446744073709551615 134529300 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/14720/statm: 116261 5749 46 122 0 115161 0
[pid=14718/tid=14778] ppid=14485 vsize=488620 CPUtime=0 cores=1,3,5,7
/proc/14718/task/14778/stat : 14778 (python) S 14485 14485 14447 0 -1 4202560 83 0 0 0 0 0 0 0 19 0 2 0 657999781 500346880 13770 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 514 18446604467742496576 0 0 -1 3 0 0 0
[pid=14775] ppid=14485 vsize=498876 CPUtime=0.94 cores=1,3,5,7
/proc/14775/stat : 14775 (python) S 14485 14485 14447 0 -1 4202560 1060 0 0 0 46 48 0 0 15 0 2 0 657998972 510849024 13807 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624937419 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/14775/statm: 124719 13807 399 364 0 60601 0
[pid=14777] ppid=14775 vsize=268424 CPUtime=296.79 cores=1,3,5,7
/proc/14777/stat : 14777 (hybridGM3) T 14775 14777 14777 0 -1 4202496 8094 0 0 0 29678 1 0 0 25 0 1 0 657998973 274866176 7827 33554432000 134512640 135204385 4287210912 18446744073709551615 134659343 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/14777/statm: 67106 7827 64 169 0 66936 0
[pid=14775/tid=14820] ppid=14485 vsize=498876 CPUtime=0 cores=1,3,5,7
/proc/14775/task/14820/stat : 14820 (python) S 14485 14485 14447 0 -1 4202560 83 0 0 0 0 0 0 0 15 0 2 0 658018648 510849024 13807 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 514 18446604455590392896 0 0 -1 3 0 0 0
[pid=14779] ppid=14485 vsize=509132 CPUtime=0.69 cores=1,3,5,7
/proc/14779/stat : 14779 (python) S 14485 14485 14447 0 -1 4202560 10746 0 0 0 34 35 0 0 16 0 2 0 657999782 521351168 13821 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624937419 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/14779/statm: 127283 13821 399 364 0 63162 0
[pid=14781] ppid=14779 vsize=268428 CPUtime=196.81 cores=1,3,5,7
/proc/14781/stat : 14781 (hybridGM3) T 14779 14781 14781 0 -1 4202496 8100 0 0 0 19679 2 0 0 25 0 1 0 657999782 274870272 7832 33554432000 134512640 135204385 4291220880 18446744073709551615 134537272 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/14781/statm: 67107 7832 64 169 0 66937 0
[pid=14779/tid=14821] ppid=14485 vsize=509132 CPUtime=0 cores=1,3,5,7
/proc/14779/task/14821/stat : 14821 (python) S 14485 14485 14447 0 -1 4202560 85 0 0 0 0 0 0 0 16 0 2 0 658019463 521351168 13821 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 514 18446604455590393792 0 0 -1 3 0 0 0
[pid=14822] ppid=14485 vsize=519388 CPUtime=0.32 cores=1,3,5,7
/proc/14822/stat : 14822 (python) S 14485 14485 14447 0 -1 4202560 1128 0 0 0 17 15 0 0 19 0 2 0 658019464 531853312 13844 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624937419 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/14822/statm: 129847 13844 406 364 0 65723 0
[pid=14824] ppid=14822 vsize=465044 CPUtime=99.03 cores=1,3,5,7
/proc/14824/stat : 14824 (TNM) T 14822 14824 14824 0 -1 4202496 6021 0 0 0 9902 1 0 0 25 0 1 0 658019465 476205056 5749 33554432000 134512640 135008748 4288787216 18446744073709551615 134527773 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/14824/statm: 116261 5749 46 122 0 115161 0
[pid=14822/tid=14849] ppid=14485 vsize=519388 CPUtime=0 cores=1,3,5,7
/proc/14822/task/14849/stat : 14849 (python) S 14485 14485 14447 0 -1 4202560 85 0 0 0 0 0 0 0 19 0 2 0 658029497 531853312 13844 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 514 18446604460606469248 0 0 -1 3 0 0 0
[pid=14840] ppid=14485 vsize=529644 CPUtime=0.34 cores=1,3,5,7
/proc/14840/stat : 14840 (python) S 14485 14485 14447 0 -1 4202560 1109 0 0 0 16 18 0 0 19 0 2 0 658028277 542355456 13884 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624937419 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/14840/statm: 132411 13884 406 364 0 68284 0
[pid=14842] ppid=14840 vsize=465044 CPUtime=98.91 cores=1,3,5,7
/proc/14842/stat : 14842 (TNM) T 14840 14842 14842 0 -1 4202496 6021 0 0 0 9891 0 0 0 25 0 1 0 658028277 476205056 5749 33554432000 134512640 135008748 4294684080 18446744073709551615 134529448 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/14842/statm: 116261 5749 46 122 0 115161 0
[pid=14840/tid=14860] ppid=14485 vsize=529644 CPUtime=0 cores=1,3,5,7
/proc/14840/task/14860/stat : 14860 (python) S 14485 14485 14447 0 -1 4202560 81 0 0 0 0 0 0 0 19 0 2 0 658038210 542355456 13884 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 514 18446604460606470144 0 0 -1 1 0 0 0
[pid=14843] ppid=14485 vsize=539900 CPUtime=0.31 cores=1,3,5,7
/proc/14843/stat : 14843 (python) S 14485 14485 14447 0 -1 4202560 930 0 0 0 17 14 0 0 16 0 1 0 658028781 552857600 13862 33554432000 4194304 5683644 140734449211568 18446744073709551615 270613130822 0 0 16781312 514 15677223816923152644 0 0 17 3 0 0 0
/proc/14843/statm: 134975 13862 378 364 0 70845 0
[pid=14845] ppid=14843 vsize=268424 CPUtime=92.05 cores=1,3,5,7
/proc/14845/stat : 14845 (hybridGM3) R 14843 14845 14845 0 -1 4202496 8107 0 0 0 9204 1 0 0 25 0 1 0 658028782 274866176 7826 33554432000 134512640 135204385 4293047600 18446744073709551615 134537383 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/14845/statm: 67106 7826 64 169 0 66936 0
[pid=14846] ppid=14485 vsize=550156 CPUtime=0.29 cores=1,3,5,7
/proc/14846/stat : 14846 (python) S 14485 14485 14447 0 -1 4202560 910 0 0 0 15 14 0 0 18 0 1 0 658029268 563359744 13870 33554432000 4194304 5683644 140734449211568 18446744073709551615 270613130822 0 0 16781312 514 15677223816923152648 0 0 17 3 0 0 0
/proc/14846/statm: 137539 13870 372 364 0 73406 0
[pid=14848] ppid=14846 vsize=465044 CPUtime=89.31 cores=1,3,5,7
/proc/14848/stat : 14848 (TNM) R 14846 14848 14848 0 -1 4202496 6023 0 0 0 8930 1 0 0 25 0 1 0 658029268 476205056 5749 33554432000 134512640 135008748 4293236704 18446744073709551615 134524656 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/14848/statm: 116261 5749 46 122 0 115161 0
[pid=14850] ppid=14485 vsize=560412 CPUtime=0.27 cores=1,3,5,7
/proc/14850/stat : 14850 (python) S 14485 14485 14447 0 -1 4202560 914 0 0 0 14 13 0 0 18 0 1 0 658029498 573861888 13875 33554432000 4194304 5683644 140734449211568 18446744073709551615 270613130822 0 0 16781312 514 15677223816923152645 0 0 17 3 0 0 0
/proc/14850/statm: 140103 13875 372 364 0 75967 0
[pid=14852] ppid=14850 vsize=465044 CPUtime=87.87 cores=1,3,5,7
/proc/14852/stat : 14852 (TNM) R 14850 14852 14852 0 -1 4202496 6025 0 0 0 8786 1 0 0 25 0 1 0 658029499 476205056 5750 33554432000 134512640 135008748 4293988352 18446744073709551615 134529475 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/14852/statm: 116261 5750 46 122 0 115161 0
[pid=14861] ppid=14485 vsize=570668 CPUtime=0 cores=1,3,5,7
/proc/14861/stat : 14861 (python) S 14485 14485 14447 0 -1 4202560 818 0 0 0 0 0 0 0 19 0 1 0 658038212 584364032 13866 33554432000 4194304 5683644 140734449211568 18446744073709551615 270613130822 0 0 16781312 514 17098368988397699071 0 0 17 3 0 0 0
/proc/14861/statm: 142667 13866 345 364 0 78528 0
[pid=14863] ppid=14861 vsize=465044 CPUtime=0.72 cores=1,3,5,7
/proc/14863/stat : 14863 (TNM) R 14861 14863 14863 0 -1 4202496 6021 0 0 0 71 1 0 0 22 0 1 0 658038212 476205056 5750 33554432000 134512640 135008748 4288676400 18446744073709551615 134528760 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/14863/statm: 116261 5750 46 122 0 115161 0
[pid=14485/tid=14509] ppid=14483 vsize=580912 CPUtime=0 cores=1,3,5,7
/proc/14485/task/14509/stat : 14509 (python) S 14483 14485 14447 0 -1 4202560 44 935 0 0 0 0 0 0 15 0 21 0 657929937 594853888 15439 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 2 18446604464099614784 0 0 -1 5 0 0 0
[pid=14485/tid=14511] ppid=14483 vsize=580912 CPUtime=0 cores=1,3,5,7
/proc/14485/task/14511/stat : 14511 (python) S 14483 14485 14447 0 -1 4202560 42 935 0 0 0 0 0 0 18 0 21 0 657929938 594853888 15439 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 2 18446604464099614784 0 0 -1 1 0 0 0
[pid=14485/tid=14514] ppid=14483 vsize=580912 CPUtime=0 cores=1,3,5,7
/proc/14485/task/14514/stat : 14514 (python) S 14483 14485 14447 0 -1 4202560 30 935 0 0 0 0 0 0 18 0 21 0 657929939 594853888 15439 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 2 18446604464099614784 0 0 -1 5 0 0 0
[pid=14485/tid=14519] ppid=14483 vsize=580912 CPUtime=0 cores=1,3,5,7
/proc/14485/task/14519/stat : 14519 (python) S 14483 14485 14447 0 -1 4202560 64 935 0 0 0 0 0 0 18 0 21 0 657929940 594853888 15439 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 2 18446604464099614784 0 0 -1 1 0 0 0
[pid=14485/tid=14644] ppid=14483 vsize=580912 CPUtime=0 cores=1,3,5,7
/proc/14485/task/14644/stat : 14644 (python) S 14483 14485 14447 0 -1 4202560 50 935 0 0 0 0 0 0 18 0 21 0 657949512 594853888 15439 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 2 18446604464099614784 0 0 -1 3 0 0 0
[pid=14485/tid=14668] ppid=14483 vsize=580912 CPUtime=0 cores=1,3,5,7
/proc/14485/task/14668/stat : 14668 (python) S 14483 14485 14447 0 -1 4202560 45 935 0 0 0 0 0 0 18 0 21 0 657959634 594853888 15439 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 2 18446604464099614784 0 0 -1 3 0 0 0
[pid=14485/tid=14671] ppid=14483 vsize=580912 CPUtime=0 cores=1,3,5,7
/proc/14485/task/14671/stat : 14671 (python) S 14483 14485 14447 0 -1 4202560 46 935 0 0 0 0 0 0 18 0 21 0 657959640 594853888 15439 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 2 18446604464099614784 0 0 -1 3 0 0 0
[pid=14485/tid=14674] ppid=14483 vsize=580912 CPUtime=0 cores=1,3,5,7
/proc/14485/task/14674/stat : 14674 (python) S 14483 14485 14447 0 -1 4202560 37 935 0 0 0 0 0 0 18 0 21 0 657959739 594853888 15439 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 2 18446604464099614784 0 0 -1 3 0 0 0
[pid=14485/tid=14692] ppid=14483 vsize=580912 CPUtime=0 cores=1,3,5,7
/proc/14485/task/14692/stat : 14692 (python) S 14483 14485 14447 0 -1 4202560 29 935 0 0 0 0 0 0 20 0 21 0 657969873 594853888 15439 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 2 18446604464099614784 0 0 -1 3 0 0 0
[pid=14485/tid=14705] ppid=14483 vsize=580912 CPUtime=0 cores=1,3,5,7
/proc/14485/task/14705/stat : 14705 (python) S 14483 14485 14447 0 -1 4202560 35 935 0 0 0 0 0 0 20 0 21 0 657979812 594853888 15439 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 2 18446604464099614784 0 0 -1 3 0 0 0
[pid=14485/tid=14713] ppid=14483 vsize=580912 CPUtime=0 cores=1,3,5,7
/proc/14485/task/14713/stat : 14713 (python) S 14483 14485 14447 0 -1 4202560 38 935 0 0 0 0 0 0 18 0 21 0 657988940 594853888 15439 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 2 18446604464099614784 0 0 -1 3 0 0 0
[pid=14485/tid=14719] ppid=14483 vsize=580912 CPUtime=0 cores=1,3,5,7
/proc/14485/task/14719/stat : 14719 (python) S 14483 14485 14447 0 -1 4202560 39 935 0 0 0 0 0 0 20 0 21 0 657989747 594853888 15439 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 2 18446604464099614784 0 0 -1 3 0 0 0
[pid=14485/tid=14776] ppid=14483 vsize=580912 CPUtime=0 cores=1,3,5,7
/proc/14485/task/14776/stat : 14776 (python) S 14483 14485 14447 0 -1 4202560 36 935 0 0 0 0 0 0 18 0 21 0 657998973 594853888 15439 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 2 18446604464099614784 0 0 -1 3 0 0 0
[pid=14485/tid=14780] ppid=14483 vsize=580912 CPUtime=0 cores=1,3,5,7
/proc/14485/task/14780/stat : 14780 (python) S 14483 14485 14447 0 -1 4202560 36 935 0 0 0 0 0 0 20 0 21 0 657999782 594853888 15439 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 2 18446604464099614784 0 0 -1 3 0 0 0
[pid=14485/tid=14823] ppid=14483 vsize=580912 CPUtime=0 cores=1,3,5,7
/proc/14485/task/14823/stat : 14823 (python) S 14483 14485 14447 0 -1 4202560 34 935 0 0 0 0 0 0 20 0 21 0 658019465 594853888 15439 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 2 18446604464099614784 0 0 -1 3 0 0 0
[pid=14485/tid=14841] ppid=14483 vsize=580912 CPUtime=0 cores=1,3,5,7
/proc/14485/task/14841/stat : 14841 (python) S 14483 14485 14447 0 -1 4202560 32 935 0 0 0 0 0 0 20 0 21 0 658028277 594853888 15439 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 2 18446604464099614784 0 0 -1 3 0 0 0
[pid=14485/tid=14844] ppid=14483 vsize=580912 CPUtime=0 cores=1,3,5,7
/proc/14485/task/14844/stat : 14844 (python) S 14483 14485 14447 0 -1 4202560 31 935 0 0 0 0 0 0 20 0 21 0 658028782 594853888 15439 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 2 18446604464099614784 0 0 -1 3 0 0 0
[pid=14485/tid=14847] ppid=14483 vsize=580912 CPUtime=0 cores=1,3,5,7
/proc/14485/task/14847/stat : 14847 (python) S 14483 14485 14447 0 -1 4202560 33 935 0 0 0 0 0 0 20 0 21 0 658029268 594853888 15439 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 2 18446604464099614784 0 0 -1 3 0 0 0
[pid=14485/tid=14851] ppid=14483 vsize=580912 CPUtime=0 cores=1,3,5,7
/proc/14485/task/14851/stat : 14851 (python) S 14483 14485 14447 0 -1 4202560 30 935 0 0 0 0 0 0 20 0 21 0 658029499 594853888 15439 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 2 18446604464099614784 0 0 -1 3 0 0 0
[pid=14485/tid=14862] ppid=14483 vsize=580912 CPUtime=0 cores=1,3,5,7
/proc/14485/task/14862/stat : 14862 (python) S 14483 14485 14447 0 -1 4202560 34 935 0 0 0 0 0 0 20 0 21 0 658038212 594853888 15439 33554432000 4194304 5683644 140734449211568 18446744073709551615 270624934321 0 0 16781312 2 18446604464099614784 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4316.84
Current children cumulated vsize (KiB) 16800724

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 14485 and gives
#  childrusage.ru_utime.tv_sec=196
#  childrusage.ru_utime.tv_usec=357149
#  childrusage.ru_stime.tv_sec=6
#  childrusage.ru_stime.tv_usec=496012
# CPU time returned by wait4() is 202.853
# while last known CPU time is 4316.84
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1091.24
CPU time (s): 4316.84
CPU user time (s): 4310.31
CPU system time (s): 6.53
CPU usage (%): 395.588
Max. virtual memory (cumulated for all children) (KiB): 16800724

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 196.357
system time used= 6.49601
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 86329
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 13636
involuntary context switches= 1858

runsolver used 7.44587 second user time and 16.6985 second system time

The end

Launcher Data 

Begin job on node108 at 2011-05-15 14:41:22
IDJOB=3378163
IDBENCH=82572
IDSOLVER=1859
FILE ID=node108/3378163-1305463282
RUNJOBID= node108-1305463282-14466
PBS_JOBID= 13324989
Free space on /tmp= 73520 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/large/unif-k3-r4.2-v35000-c147000-S950666065-093-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3378163-1305463282/watcher-3378163-1305463282 -o /tmp/evaluation-result-3378163-1305463282/solver-3378163-1305463282 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378163-1305463282.cnf -seed 2015938851 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= e27af9a147b2d7c0283d090128805575
RANDOM SEED=2015938851

node108.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.64
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      17080036 kB
Buffers:        307508 kB
Cached:        5114596 kB
SwapCached:       8700 kB
Active:       11279456 kB
Inactive:      4271424 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      17080036 kB
SwapTotal:    67111528 kB
SwapFree:     67056804 kB
Dirty:           67676 kB
Writeback:           0 kB
AnonPages:    10126348 kB
Mapped:          30956 kB
Slab:           213012 kB
PageTables:      40056 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 67470120 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73480 MiB
End job on node108 at 2011-05-15 14:59:36