Trace number 3378289

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)? 4728.68 1197.1

General information on the benchmark

Namerandom/large/
unif-k3-r4.2-v50000-c210000-S1170024351-015-UNKNOWN.cnf
MD5SUM0f7796213ec78265bd44aa83c0fa283d
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark153.855
Satisfiable
(Un)Satisfiability was proved
Number of variables50000
Number of clauses210000
Sum of the clauses size630000
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3210000
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.30/0.67	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.30/0.67	c INFO: solving HOME/instance-3378289-1305464517.cnf
0.30/0.67	c INFO: computing task features
1.08/1.44	c INFO: computed variable-clause graph statistics
1.08/1.47	c INFO: computed clause balance statistics
4.78/5.13	c INFO: computed variable balance statistics
7.67/8.00	c INFO: computed small-clause counts
7.67/8.02	c INFO: computed Horn-clause counts
10.77/11.10	c INFO: computed variable graph statistics
10.77/11.10	c INFO: collected features for HOME/instance-3378289-1305464517.cnf in 10.39 s
10.77/11.11	c INFO: running hybridGM3@0 for 97 with 1189 remaining (b = 0.35)
10.77/11.12	c INFO: running hybridGM3@0 for 97 with 1189 remaining (b = 0.21)
10.77/11.13	c INFO: running hybridGM3@0 for 97 with 1189 remaining (b = 0.14)
10.77/11.14	c INFO: running hybridGM3@0 for 194 with 1189 remaining (b = 0.18)
410.69/111.49	c INFO: running hybridGM3@0 for 97 with 1089 remaining (b = 0.08)
411.09/111.50	c INFO: running hybridGM3@0 for 97 with 1089 remaining (b = 0.08)
411.09/111.51	c INFO: running hybridGM3@0 for 97 with 1089 remaining (b = 0.08)
789.94/206.73	c INFO: running hybridGM3@100 for 97 with 993 remaining (b = 0.08)
789.94/206.75	c INFO: running TNM@0 for 97 with 993 remaining (b = 0.08)
792.96/207.78	c INFO: running hybridGM3@95 for 97 with 992 remaining (b = 0.07)
813.82/212.76	c INFO: running hybridGM3@99 for 97 with 987 remaining (b = 0.07)
1189.88/307.02	c INFO: running hybridGM3@100 for 97 with 893 remaining (b = 0.07)
1193.52/307.99	c INFO: running hybridGM3@0 for 97 with 892 remaining (b = 0.06)
1193.92/308.04	c INFO: running hybridGM3@0 for 194 with 892 remaining (b = 0.09)
1225.69/316.04	c INFO: running hybridGM3@0 for 194 with 884 remaining (b = 0.07)
1576.58/404.29	c INFO: running hybridGM3@0 for 194 with 796 remaining (b = 0.05)
1593.28/408.44	c INFO: running hybridGM3@100 for 97 with 792 remaining (b = 0.02)
1972.03/503.69	c INFO: running TNM@0 for 97 with 696 remaining (b = 0.02)
1976.37/504.72	c INFO: running hybridGM3@102 for 97 with 695 remaining (b = 0.02)
2028.50/517.84	c INFO: running hybridGM3@96 for 97 with 682 remaining (b = 0.02)
2355.15/599.97	c INFO: running hybridGM3@300 for 97 with 600 remaining (b = 0.02)
2359.52/601.03	c INFO: running hybridGM3@100 for 97 with 599 remaining (b = 0.02)
2371.11/603.99	c INFO: running hybridGM3@0 for 194 with 596 remaining (b = 0.04)
2407.24/613.09	c INFO: running hybridGM3@191 for 97 with 587 remaining (b = 0.02)
2508.68/638.59	c INFO: running TNM@194 for 562 with 562 remaining (b = 0.07)
2754.16/700.24	c INFO: running TNM@0 for 97 with 500 remaining (b = 0.02)
2762.53/702.32	c INFO: running hybridGM3@0 for 194 with 498 remaining (b = 0.03)
3153.14/800.54	c INFO: running hybridGM3@0 for 194 with 400 remaining (b = 0.02)
3153.95/800.75	c INFO: running hybridGM3@94 for 97 with 399 remaining (b = 0.01)
3556.82/902.04	c INFO: running TNM@0 for 97 with 298 remaining (b = 0.01)
3569.11/905.13	c INFO: running hybridGM3@98 for 97 with 295 remaining (b = 0.01)
3935.69/997.31	c INFO: running hybridGM3@0 for 203 with 203 remaining (b = 0.02)
3947.98/1000.40	c INFO: running hybridGM3@292 for 97 with 200 remaining (b = 0.01)
3955.53/1002.34	c INFO: running TNM@0 for 97 with 198 remaining (b = 0.01)
4349.94/1101.70	c INFO: running hybridGM3@392 for 98 with 98 remaining (b = 0.01)
4353.50/1102.65	c INFO: running TNM@0 for 98 with 98 remaining (b = 0.01)
4728.11/1197.06	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3378289-1305464517/watcher-3378289-1305464517 -o /tmp/evaluation-result-3378289-1305464517/solver-3378289-1305464517 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378289-1305464517.cnf -seed 1113523622 -budget 4800 -cores 4 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.91 8.13 7.01 5/332 18602
/proc/meminfo: memFree=20288172/32951124 swapFree=66457188/67111528
[pid=18602] ppid=18600 vsize=15776 CPUtime=0 cores=0,2,4,6
/proc/18602/stat : 18602 (python) R 18600 18602 18093 0 -1 4202496 897 0 0 0 0 0 0 0 25 0 1 0 658049935 16154624 778 33554432000 4194304 5683644 140734404324192 18446744073709551615 4516195 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/18602/statm: 3944 779 343 364 0 431 0

[startup+0.0775389 s]
/proc/loadavg: 7.91 8.13 7.01 5/332 18602
/proc/meminfo: memFree=20288172/32951124 swapFree=66457188/67111528
[pid=18602] ppid=18600 vsize=45824 CPUtime=0.03 cores=0,2,4,6
/proc/18602/stat : 18602 (python) R 18600 18602 18093 0 -1 4202496 1913 0 0 0 3 0 0 0 25 0 1 0 658049935 46923776 1687 33554432000 4194304 5683644 140734404324192 18446744073709551615 4540219 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/18602/statm: 11456 1687 470 364 0 1205 0
Current children cumulated CPU time (s) 0.03
Current children cumulated vsize (KiB) 45824

[startup+0.100505 s]
/proc/loadavg: 7.91 8.13 7.01 5/332 18602
/proc/meminfo: memFree=20288172/32951124 swapFree=66457188/67111528
[pid=18602] ppid=18600 vsize=87708 CPUtime=0.05 cores=0,2,4,6
/proc/18602/stat : 18602 (python) S 18600 18602 18093 0 -1 4202496 2637 0 0 0 4 1 0 0 18 0 1 0 658049935 89812992 2354 33554432000 4194304 5683644 140734404324192 18446744073709551615 232335890432 0 2147483391 16781312 2 18446744071699453118 0 0 17 6 0 0 0
/proc/18602/statm: 21927 2354 717 364 0 1623 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 87708

[startup+0.300474 s]
/proc/loadavg: 7.91 8.13 7.01 5/332 18602
/proc/meminfo: memFree=20288172/32951124 swapFree=66457188/67111528
[pid=18602] ppid=18600 vsize=135752 CPUtime=0.15 cores=0,2,4,6
/proc/18602/stat : 18602 (python) R 18600 18602 18093 0 -1 4202496 5962 932 1 0 13 2 0 0 17 0 1 0 658049935 139010048 4151 33554432000 4194304 5683644 140734404324192 18446744073709551615 232335888389 0 2147483391 16781312 2 0 0 0 17 6 0 0 0
/proc/18602/statm: 33938 4151 1032 364 0 3124 0
Current children cumulated CPU time (s) 0.15
Current children cumulated vsize (KiB) 135752

[startup+0.700388 s]
/proc/loadavg: 7.91 8.13 7.01 5/332 18602
/proc/meminfo: memFree=20288172/32951124 swapFree=66457188/67111528
[pid=18602] ppid=18600 vsize=283476 CPUtime=0.37 cores=0,2,4,6
/proc/18602/stat : 18602 (python) R 18600 18602 18093 0 -1 4202496 10508 932 1 0 30 7 0 0 16 0 1 0 658049935 290279424 8192 33554432000 4194304 5683644 140734404324192 18446744073709551615 4602421 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/18602/statm: 70869 8192 1761 364 0 6793 0
Current children cumulated CPU time (s) 0.37
Current children cumulated vsize (KiB) 283476

[startup+1.50031 s]
/proc/loadavg: 7.91 8.13 7.01 7/333 18607
/proc/meminfo: memFree=20243020/32951124 swapFree=66457188/67111528
[pid=18602] ppid=18600 vsize=328000 CPUtime=1.17 cores=0,2,4,6
/proc/18602/stat : 18602 (python) R 18600 18602 18093 0 -1 4202496 27451 932 1 0 107 10 0 0 17 0 1 0 658049935 335872000 19384 33554432000 4194304 5683644 140734404324192 18446744073709551615 232339301033 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/18602/statm: 82000 19384 1826 364 0 17924 0
Current children cumulated CPU time (s) 1.17
Current children cumulated vsize (KiB) 328000

[startup+3.10103 s]
/proc/loadavg: 7.91 8.13 7.01 7/333 18607
/proc/meminfo: memFree=20215740/32951124 swapFree=66457188/67111528
[pid=18602] ppid=18600 vsize=328000 CPUtime=2.78 cores=0,2,4,6
/proc/18602/stat : 18602 (python) R 18600 18602 18093 0 -1 4202496 27451 932 1 0 268 10 0 0 19 0 1 0 658049935 335872000 19384 33554432000 4194304 5683644 140734404324192 18446744073709551615 4940555 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/18602/statm: 82000 19384 1826 364 0 17924 0
Current children cumulated CPU time (s) 2.78
Current children cumulated vsize (KiB) 328000

[startup+6.30043 s]
/proc/loadavg: 7.75 8.09 7.01 7/333 18607
/proc/meminfo: memFree=20215468/32951124 swapFree=66457188/67111528
[pid=18602] ppid=18600 vsize=328000 CPUtime=5.98 cores=0,2,4,6
/proc/18602/stat : 18602 (python) R 18600 18602 18093 0 -1 4202496 27452 932 1 0 588 10 0 0 25 0 1 0 658049935 335872000 19385 33554432000 4194304 5683644 140734404324192 18446744073709551615 232335551109 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/18602/statm: 82000 19385 1827 364 0 17924 0
Current children cumulated CPU time (s) 5.98
Current children cumulated vsize (KiB) 328000

[startup+12.7013 s]
/proc/loadavg: 7.53 8.04 7.00 10/345 18619
/proc/meminfo: memFree=20138876/32951124 swapFree=66457188/67111528
[pid=18602] ppid=18600 vsize=365132 CPUtime=10.81 cores=0,2,4,6
/proc/18602/stat : 18602 (python) S 18600 18602 18093 0 -1 4202496 30183 932 1 0 1070 11 0 0 22 0 5 0 658049935 373895168 18471 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347719115 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/18602/statm: 91283 18471 1873 364 0 27192 0
[pid=18608] ppid=18602 vsize=324120 CPUtime=0 cores=0,2,4,6
/proc/18608/stat : 18608 (python) S 18602 18602 18093 0 -1 4202560 731 0 0 0 0 0 0 0 18 0 1 0 658051047 331898880 16935 33554432000 4194304 5683644 140734404324192 18446744073709551615 232335912463 0 0 16781312 514 9325326730736435199 0 0 17 2 0 0 0
/proc/18608/statm: 81030 16935 353 364 0 16948 0
[pid=18610] ppid=18608 vsize=276240 CPUtime=1.57 cores=0,2,4,6
/proc/18610/stat : 18610 (hybridGM3) R 18608 18610 18610 0 -1 4202496 4940 0 0 0 157 0 0 0 25 0 1 0 658051047 282869760 4675 33554432000 134512640 135204385 4293327264 18446744073709551615 134543699 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/18610/statm: 69060 4675 57 169 0 68890 0
[pid=18611] ppid=18602 vsize=334376 CPUtime=0 cores=0,2,4,6
/proc/18611/stat : 18611 (python) S 18602 18602 18093 0 -1 4202560 733 0 0 0 0 0 0 0 18 0 1 0 658051048 342401024 16933 33554432000 4194304 5683644 140734404324192 18446744073709551615 232335912518 0 0 16781312 514 556607974647267327 0 0 17 4 0 0 0
/proc/18611/statm: 83594 16933 347 364 0 19509 0
[pid=18613] ppid=18611 vsize=276240 CPUtime=1.56 cores=0,2,4,6
/proc/18613/stat : 18613 (hybridGM3) R 18611 18613 18613 0 -1 4202496 4930 0 0 0 156 0 0 0 25 0 1 0 658051048 282869760 4666 33554432000 134512640 135204385 4287411936 18446744073709551615 134543975 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/18613/statm: 69060 4666 57 169 0 68890 0
[pid=18614] ppid=18602 vsize=344632 CPUtime=0 cores=0,2,4,6
/proc/18614/stat : 18614 (python) S 18602 18602 18093 0 -1 4202560 739 0 0 0 0 0 0 0 18 0 1 0 658051049 352903168 16937 33554432000 4194304 5683644 140734404324192 18446744073709551615 232335912518 0 0 16781312 514 1183452747781898239 0 0 17 6 0 0 0
/proc/18614/statm: 86158 16937 347 364 0 22070 0
[pid=18616] ppid=18614 vsize=276232 CPUtime=1.55 cores=0,2,4,6
/proc/18616/stat : 18616 (hybridGM3) R 18614 18616 18616 0 -1 4202496 4918 0 0 0 155 0 0 0 25 0 1 0 658051049 282861568 4658 33554432000 134512640 135204385 4290982560 18446744073709551615 134543968 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/18616/statm: 69058 4658 57 169 0 68888 0
[pid=18617] ppid=18602 vsize=354888 CPUtime=0 cores=0,2,4,6
/proc/18617/stat : 18617 (python) S 18602 18602 18093 0 -1 4202560 748 0 0 0 0 0 0 0 18 0 1 0 658051050 363405312 16941 33554432000 4194304 5683644 140734404324192 18446744073709551615 232335912518 0 0 16781312 514 5436539371002003455 0 0 17 4 0 0 0
/proc/18617/statm: 88722 16941 347 364 0 24631 0
[pid=18619] ppid=18617 vsize=276240 CPUtime=1.52 cores=0,2,4,6
/proc/18619/stat : 18619 (hybridGM3) R 18617 18619 18619 0 -1 4202496 4902 0 0 0 152 0 0 0 25 0 1 0 658051050 282869760 4642 33554432000 134512640 135204385 4294158640 18446744073709551615 134543975 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/18619/statm: 69060 4642 57 169 0 68890 0
[pid=18602/tid=18609] ppid=18600 vsize=365132 CPUtime=0 cores=0,2,4,6
/proc/18602/task/18609/stat : 18609 (python) S 18600 18602 18093 0 -1 4202560 27 932 0 0 0 0 0 0 25 0 5 0 658051047 373895168 18471 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 2 18446604442171689664 0 0 -1 2 0 0 0
[pid=18602/tid=18612] ppid=18600 vsize=365132 CPUtime=0 cores=0,2,4,6
/proc/18602/task/18612/stat : 18612 (python) S 18600 18602 18093 0 -1 4202560 26 932 0 0 0 0 0 0 25 0 5 0 658051048 373895168 18471 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 2 18446604442171689664 0 0 -1 2 0 0 0
[pid=18602/tid=18615] ppid=18600 vsize=365132 CPUtime=0 cores=0,2,4,6
/proc/18602/task/18615/stat : 18615 (python) S 18600 18602 18093 0 -1 4202560 23 932 0 0 0 0 0 0 20 0 5 0 658051049 373895168 18471 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 2 18446604442171689664 0 0 -1 6 0 0 0
[pid=18602/tid=18618] ppid=18600 vsize=365132 CPUtime=0 cores=0,2,4,6
/proc/18602/task/18618/stat : 18618 (python) S 18600 18602 18093 0 -1 4202560 28 932 0 0 0 0 0 0 22 0 5 0 658051050 373895168 18471 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 2 18446604442171689664 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 17.01
Current children cumulated vsize (KiB) 2828100

[startup+25.501 s]
/proc/loadavg: 7.64 8.04 7.01 11/345 18620
/proc/meminfo: memFree=20055404/32951124 swapFree=66457188/67111528
[pid=18602] ppid=18600 vsize=365132 CPUtime=10.81 cores=0,2,4,6
/proc/18602/stat : 18602 (python) S 18600 18602 18093 0 -1 4202496 30183 932 1 0 1070 11 0 0 22 0 5 0 658049935 373895168 18471 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347719115 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/18602/statm: 91283 18471 1873 364 0 27192 0
[pid=18608] ppid=18602 vsize=324120 CPUtime=0.02 cores=0,2,4,6
/proc/18608/stat : 18608 (python) S 18602 18602 18093 0 -1 4202560 905 0 0 0 1 1 0 0 15 0 1 0 658051047 331898880 16958 33554432000 4194304 5683644 140734404324192 18446744073709551615 232335912463 0 0 16781312 514 186582725187043587 0 0 17 2 0 0 0
/proc/18608/statm: 81030 16958 376 364 0 16948 0
[pid=18610] ppid=18608 vsize=276244 CPUtime=14.31 cores=0,2,4,6
/proc/18610/stat : 18610 (hybridGM3) R 18608 18610 18610 0 -1 4202496 9514 0 0 0 1430 1 0 0 25 0 1 0 658051047 282873856 9249 33554432000 134512640 135204385 4293327264 18446744073709551615 134536663 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/18610/statm: 69061 9249 64 169 0 68891 0
[pid=18611] ppid=18602 vsize=334376 CPUtime=0.02 cores=0,2,4,6
/proc/18611/stat : 18611 (python) S 18602 18602 18093 0 -1 4202560 914 0 0 0 1 1 0 0 15 0 1 0 658051048 342401024 16955 33554432000 4194304 5683644 140734404324192 18446744073709551615 232335912518 0 0 16781312 514 186582725187043588 0 0 17 4 0 0 0
/proc/18611/statm: 83594 16955 369 364 0 19509 0
[pid=18613] ppid=18611 vsize=276244 CPUtime=14.33 cores=0,2,4,6
/proc/18613/stat : 18613 (hybridGM3) R 18611 18613 18613 0 -1 4202496 9515 0 0 0 1432 1 0 0 25 0 1 0 658051048 282873856 9251 33554432000 134512640 135204385 4287411936 18446744073709551615 134536737 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/18613/statm: 69061 9251 64 169 0 68891 0
[pid=18614] ppid=18602 vsize=344632 CPUtime=0.02 cores=0,2,4,6
/proc/18614/stat : 18614 (python) S 18602 18602 18093 0 -1 4202560 909 0 0 0 1 1 0 0 15 0 1 0 658051049 352903168 16960 33554432000 4194304 5683644 140734404324192 18446744073709551615 232335912518 0 0 16781312 514 186582725187043588 0 0 17 6 0 0 0
/proc/18614/statm: 86158 16960 370 364 0 22070 0
[pid=18616] ppid=18614 vsize=276236 CPUtime=14.35 cores=0,2,4,6
/proc/18616/stat : 18616 (hybridGM3) R 18614 18616 18616 0 -1 4202496 9507 0 0 0 1434 1 0 0 25 0 1 0 658051049 282865664 9247 33554432000 134512640 135204385 4290982560 18446744073709551615 134537383 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/18616/statm: 69059 9247 64 169 0 68889 0
[pid=18617] ppid=18602 vsize=354888 CPUtime=0.02 cores=0,2,4,6
/proc/18617/stat : 18617 (python) S 18602 18602 18093 0 -1 4202560 925 0 0 0 1 1 0 0 15 0 1 0 658051050 363405312 16964 33554432000 4194304 5683644 140734404324192 18446744073709551615 232335912518 0 0 16781312 514 186582725187043586 0 0 17 4 0 0 0
/proc/18617/statm: 88722 16964 370 364 0 24631 0
[pid=18619] ppid=18617 vsize=276244 CPUtime=14.16 cores=0,2,4,6
/proc/18619/stat : 18619 (hybridGM3) R 18617 18619 18619 0 -1 4202496 9513 0 0 0 1415 1 0 0 25 0 1 0 658051050 282873856 9253 33554432000 134512640 135204385 4294158640 18446744073709551615 134547728 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/18619/statm: 69061 9253 64 169 0 68891 0
[pid=18602/tid=18609] ppid=18600 vsize=365132 CPUtime=0 cores=0,2,4,6
/proc/18602/task/18609/stat : 18609 (python) S 18600 18602 18093 0 -1 4202560 27 932 0 0 0 0 0 0 25 0 5 0 658051047 373895168 18471 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 2 18446604442171689664 0 0 -1 2 0 0 0
[pid=18602/tid=18612] ppid=18600 vsize=365132 CPUtime=0 cores=0,2,4,6
/proc/18602/task/18612/stat : 18612 (python) S 18600 18602 18093 0 -1 4202560 26 932 0 0 0 0 0 0 25 0 5 0 658051048 373895168 18471 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 2 18446604442171689664 0 0 -1 2 0 0 0
[pid=18602/tid=18615] ppid=18600 vsize=365132 CPUtime=0 cores=0,2,4,6
/proc/18602/task/18615/stat : 18615 (python) S 18600 18602 18093 0 -1 4202560 23 932 0 0 0 0 0 0 20 0 5 0 658051049 373895168 18471 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 2 18446604442171689664 0 0 -1 6 0 0 0
[pid=18602/tid=18618] ppid=18600 vsize=365132 CPUtime=0 cores=0,2,4,6
/proc/18602/task/18618/stat : 18618 (python) S 18600 18602 18093 0 -1 4202560 28 932 0 0 0 0 0 0 22 0 5 0 658051050 373895168 18471 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 2 18446604442171689664 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 68.04
Current children cumulated vsize (KiB) 2828116

[startup+51.1014 s]
/proc/loadavg: 8.00 8.08 7.05 9/361 18636
/proc/meminfo: memFree=20021472/32951124 swapFree=66457188/67111528
[pid=18602] ppid=18600 vsize=365132 CPUtime=10.81 cores=0,2,4,6
/proc/18602/stat : 18602 (python) S 18600 18602 18093 0 -1 4202496 30183 932 1 0 1070 11 0 0 22 0 5 0 658049935 373895168 18471 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347719115 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/18602/statm: 91283 18471 1873 364 0 27192 0
[pid=18608] ppid=18602 vsize=324120 CPUtime=0.09 cores=0,2,4,6
/proc/18608/stat : 18608 (python) S 18602 18602 18093 0 -1 4202560 916 0 0 0 4 5 0 0 15 0 1 0 658051047 331898880 16958 33554432000 4194304 5683644 140734404324192 18446744073709551615 232335912463 0 0 16781312 514 186582725187043587 0 0 17 2 0 0 0
/proc/18608/statm: 81030 16958 376 364 0 16948 0
[pid=18610] ppid=18608 vsize=276244 CPUtime=39.79 cores=0,2,4,6
/proc/18610/stat : 18610 (hybridGM3) R 18608 18610 18610 0 -1 4202496 9514 0 0 0 3978 1 0 0 25 0 1 0 658051047 282873856 9249 33554432000 134512640 135204385 4293327264 18446744073709551615 134537403 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/18610/statm: 69061 9249 64 169 0 68891 0
[pid=18611] ppid=18602 vsize=334376 CPUtime=0.08 cores=0,2,4,6
/proc/18611/stat : 18611 (python) S 18602 18602 18093 0 -1 4202560 926 0 0 0 4 4 0 0 15 0 1 0 658051048 342401024 16956 33554432000 4194304 5683644 140734404324192 18446744073709551615 232335912518 0 0 16781312 514 186582725187043588 0 0 17 4 0 0 0
/proc/18611/statm: 83594 16956 370 364 0 19509 0
[pid=18613] ppid=18611 vsize=276244 CPUtime=39.86 cores=0,2,4,6
/proc/18613/stat : 18613 (hybridGM3) R 18611 18613 18613 0 -1 4202496 9515 0 0 0 3985 1 0 0 25 0 1 0 658051048 282873856 9251 33554432000 134512640 135204385 4287411936 18446744073709551615 134537405 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/18613/statm: 69061 9251 64 169 0 68891 0
[pid=18614] ppid=18602 vsize=344632 CPUtime=0.08 cores=0,2,4,6
/proc/18614/stat : 18614 (python) S 18602 18602 18093 0 -1 4202560 918 0 0 0 4 4 0 0 15 0 1 0 658051049 352903168 16960 33554432000 4194304 5683644 140734404324192 18446744073709551615 232335912518 0 0 16781312 514 186582725187043588 0 0 17 6 0 0 0
/proc/18614/statm: 86158 16960 370 364 0 22070 0
[pid=18616] ppid=18614 vsize=276236 CPUtime=39.95 cores=0,2,4,6
/proc/18616/stat : 18616 (hybridGM3) R 18614 18616 18616 0 -1 4202496 9508 0 0 0 3994 1 0 0 25 0 1 0 658051049 282865664 9248 33554432000 134512640 135204385 4290982560 18446744073709551615 134536656 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/18616/statm: 69059 9248 64 169 0 68889 0
[pid=18617] ppid=18602 vsize=354888 CPUtime=0.08 cores=0,2,4,6
/proc/18617/stat : 18617 (python) S 18602 18602 18093 0 -1 4202560 933 0 0 0 4 4 0 0 15 0 1 0 658051050 363405312 16964 33554432000 4194304 5683644 140734404324192 18446744073709551615 232335912518 0 0 16781312 514 186582725187043586 0 0 17 4 0 0 0
/proc/18617/statm: 88722 16964 370 364 0 24631 0

################
# More data... #
################

[pid=18602/tid=19098] ppid=18600 vsize=508716 CPUtime=318.24 cores=0,2,4,6
/proc/18602/task/19098/stat : 19098 (python) S 18600 18602 18093 0 -1 4202560 41 11784 0 0 0 0 31706 118 18 0 19 0 658120168 520925184 18564 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 2 18446604442171689664 0 0 -1 2 0 0 0
[pid=18602/tid=19113] ppid=18600 vsize=508716 CPUtime=318.24 cores=0,2,4,6
/proc/18602/task/19113/stat : 19113 (python) S 18600 18602 18093 0 -1 4202560 29 11784 0 0 0 0 31706 118 20 0 19 0 658129990 520925184 18564 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 2 18446604442171689664 0 0 -1 2 0 0 0
[pid=18602/tid=19124] ppid=18600 vsize=508716 CPUtime=318.24 cores=0,2,4,6
/proc/18602/task/19124/stat : 19124 (python) S 18600 18602 18093 0 -1 4202560 31 11784 0 0 0 0 31706 118 20 0 19 0 658140140 520925184 18564 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 2 18446604442171689664 0 0 -1 2 0 0 0
[pid=18602/tid=19135] ppid=18600 vsize=508716 CPUtime=318.24 cores=0,2,4,6
/proc/18602/task/19135/stat : 19135 (python) S 18600 18602 18093 0 -1 4202560 27 11784 0 0 0 0 31706 118 20 0 19 0 658149667 520925184 18564 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 2 18446604442171689664 0 0 -1 2 0 0 0
[pid=18602/tid=19139] ppid=18600 vsize=508716 CPUtime=318.24 cores=0,2,4,6
/proc/18602/task/19139/stat : 19139 (python) S 18600 18602 18093 0 -1 4202560 25 11784 0 0 0 0 31706 118 20 0 19 0 658150171 520925184 18564 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 2 18446604442171689664 0 0 -1 2 0 0 0
[pid=18602/tid=19148] ppid=18600 vsize=508716 CPUtime=318.24 cores=0,2,4,6
/proc/18602/task/19148/stat : 19148 (python) S 18600 18602 18093 0 -1 4202560 27 11784 0 0 0 0 31706 118 20 0 19 0 658160202 520925184 18564 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 2 18446604442171689664 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4727.65
Current children cumulated vsize (KiB) 13391716

[startup+1197 s]
/proc/loadavg: 8.20 8.24 7.86 9/464 19151
/proc/meminfo: memFree=19158012/32951124 swapFree=66457188/67111528
[pid=18602] ppid=18600 vsize=508716 CPUtime=921.36 cores=0,2,4,6
/proc/18602/stat : 18602 (python) S 18600 18602 18093 0 -1 4202496 44184 33213 1 0 1101 17 90820 198 17 0 19 0 658049935 520925184 18564 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347722559 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/18602/statm: 127179 18564 1910 364 0 63046 0
[pid=18611] ppid=18602 vsize=334376 CPUtime=0.79 cores=0,2,4,6
/proc/18611/stat : 18611 (python) S 18602 18602 18093 0 -1 4202560 1142 0 0 0 40 39 0 0 15 0 2 0 658051048 342401024 16986 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347719115 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/18611/statm: 83594 16986 399 364 0 19509 0
[pid=18613] ppid=18611 vsize=276244 CPUtime=296.31 cores=0,2,4,6
/proc/18613/stat : 18613 (hybridGM3) T 18611 18613 18613 0 -1 4202496 9515 0 0 0 29630 1 0 0 25 0 1 0 658051048 282873856 9251 33554432000 134512640 135204385 4287411936 18446744073709551615 134536744 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/18613/statm: 69061 9251 64 169 0 68891 0
[pid=18611/tid=18640] ppid=18602 vsize=334376 CPUtime=0 cores=0,2,4,6
/proc/18611/task/18640/stat : 18640 (python) S 18602 18602 18093 0 -1 4202560 46 0 0 0 0 0 0 0 15 0 2 0 658061084 342401024 16986 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 514 18446604448747321344 0 0 -1 4 0 0 0
[pid=18617] ppid=18602 vsize=0 CPUtime=297.78 cores=0,2,4,6
/proc/18617/stat : 18617 (python) R 18602 18602 18093 0 -1 4202564 1184 9513 0 0 42 37 29698 1 15 0 1 0 658051050 0 0 33554432000 0 0 0 0 0 0 0 16781824 2 18446744073709551615 0 0 17 4 0 0 0
/proc/18617/statm: 0 0 0 0 0 0 0
[pid=18664] ppid=18602 vsize=365144 CPUtime=1.89 cores=0,2,4,6
/proc/18664/stat : 18664 (python) S 18602 18602 18093 0 -1 4202560 1155 0 0 0 101 88 0 0 18 0 2 0 658070611 373907456 16998 33554432000 4194304 5683644 140734404324192 18446744073709551615 232335912518 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/18664/statm: 91286 16998 399 364 0 27192 0
[pid=18666] ppid=18664 vsize=471116 CPUtime=657.63 cores=0,2,4,6
/proc/18666/stat : 18666 (TNM) R 18664 18666 18666 0 -1 4202496 8435 0 0 0 65760 3 0 0 25 0 1 0 658070611 482422784 8173 33554432000 134512640 135008748 4291282480 18446744073709551615 134527327 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/18666/statm: 117779 8173 46 122 0 116679 0
[pid=18664/tid=18682] ppid=18602 vsize=365144 CPUtime=0 cores=0,2,4,6
/proc/18664/task/18682/stat : 18682 (python) S 18602 18602 18093 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 658080637 373907456 16998 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 514 18446604447887335744 0 0 -1 2 0 0 0
[pid=18683] ppid=18602 vsize=375400 CPUtime=0.52 cores=0,2,4,6
/proc/18683/stat : 18683 (python) S 18602 18602 18093 0 -1 4202560 1145 0 0 0 26 26 0 0 15 0 2 0 658080735 384409600 17002 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347719115 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/18683/statm: 93850 17002 399 364 0 29753 0
[pid=18685] ppid=18683 vsize=276240 CPUtime=194.63 cores=0,2,4,6
/proc/18685/stat : 18685 (hybridGM3) T 18683 18685 18685 0 -1 4202496 9514 0 0 0 19461 2 0 0 25 0 1 0 658080735 282869760 9254 33554432000 134512640 135204385 4292601200 18446744073709551615 134537292 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/18685/statm: 69060 9254 64 169 0 68890 0
[pid=18683/tid=18703] ppid=18602 vsize=375400 CPUtime=0 cores=0,2,4,6
/proc/18683/task/18703/stat : 18703 (python) S 18602 18602 18093 0 -1 4202560 47 0 0 0 0 0 0 0 15 0 2 0 658090778 384409600 17002 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 514 18446604441293582528 0 0 -1 2 0 0 0
[pid=18686] ppid=18602 vsize=385656 CPUtime=1.06 cores=0,2,4,6
/proc/18686/stat : 18686 (python) S 18602 18602 18093 0 -1 4202560 1165 0 0 0 57 49 0 0 16 0 2 0 658080740 394911744 17006 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347719115 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/18686/statm: 96414 17006 399 364 0 32314 0
[pid=18688] ppid=18686 vsize=276244 CPUtime=391.65 cores=0,2,4,6
/proc/18688/stat : 18688 (hybridGM3) T 18686 18688 18688 0 -1 4202496 9514 0 0 0 39162 3 0 0 25 0 1 0 658080740 282873856 9251 33554432000 134512640 135204385 4290979824 18446744073709551615 134537440 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/18688/statm: 69061 9251 64 169 0 68891 0
[pid=18686/tid=19049] ppid=18602 vsize=385656 CPUtime=0 cores=0,2,4,6
/proc/18686/task/19049/stat : 19049 (python) S 18602 18602 18093 0 -1 4202560 50 0 0 0 0 0 0 0 15 0 2 0 658100407 394911744 17006 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 514 18446604441293583424 0 0 -1 2 0 0 0
[pid=18700] ppid=18602 vsize=406168 CPUtime=0.8 cores=0,2,4,6
/proc/18700/stat : 18700 (python) S 18602 18602 18093 0 -1 4202560 1198 0 0 0 45 35 0 0 16 0 2 0 658090365 415916032 17020 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347719115 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/18700/statm: 101542 17020 405 364 0 37436 0
[pid=18702] ppid=18700 vsize=276236 CPUtime=295.58 cores=0,2,4,6
/proc/18702/stat : 18702 (hybridGM3) T 18700 18702 18702 0 -1 4202496 9518 0 0 0 29556 2 0 0 25 0 1 0 658090366 282865664 9251 33554432000 134512640 135204385 4294667952 18446744073709551615 134536744 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/18702/statm: 69059 9251 64 169 0 68889 0
[pid=18700/tid=19070] ppid=18602 vsize=406168 CPUtime=0 cores=0,2,4,6
/proc/18700/task/19070/stat : 19070 (python) S 18602 18602 18093 0 -1 4202560 50 0 0 0 0 0 0 0 15 0 2 0 658110038 415916032 17020 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 514 18446604448246613952 0 0 -1 2 0 0 0
[pid=19046] ppid=18602 vsize=416424 CPUtime=0.27 cores=0,2,4,6
/proc/19046/stat : 19046 (python) S 18602 18602 18093 0 -1 4202560 1123 0 0 0 14 13 0 0 19 0 2 0 658100305 426418176 17018 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347719115 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/19046/statm: 104106 17018 399 364 0 39997 0
[pid=19048] ppid=19046 vsize=471116 CPUtime=98.14 cores=0,2,4,6
/proc/19048/stat : 19048 (TNM) T 19046 19048 19048 0 -1 4202496 8437 0 0 0 9812 2 0 0 25 0 1 0 658100305 482422784 8173 33554432000 134512640 135008748 4287919152 18446744073709551615 134529448 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/19048/statm: 117779 8173 46 122 0 116679 0
[pid=19046/tid=19071] ppid=18602 vsize=416424 CPUtime=0 cores=0,2,4,6
/proc/19046/task/19071/stat : 19071 (python) S 18602 18602 18093 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 658110333 426418176 17018 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 514 18446604449460671744 0 0 -1 2 0 0 0
[pid=19072] ppid=18602 vsize=426680 CPUtime=0.84 cores=0,2,4,6
/proc/19072/stat : 19072 (python) S 18602 18602 18093 0 -1 4202560 1183 0 0 0 45 39 0 0 16 0 2 0 658110334 436920320 17022 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347719115 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/19072/statm: 106670 17022 399 364 0 42558 0
[pid=19074] ppid=19072 vsize=276240 CPUtime=296.73 cores=0,2,4,6
/proc/19074/stat : 19074 (hybridGM3) T 19072 19074 19074 0 -1 4202496 9521 0 0 0 29671 2 0 0 25 0 1 0 658110335 282869760 9254 33554432000 134512640 135204385 4287968176 18446744073709551615 134536993 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/19074/statm: 69060 9254 64 169 0 68890 0
[pid=19072/tid=19115] ppid=18602 vsize=426680 CPUtime=0 cores=0,2,4,6
/proc/19072/task/19115/stat : 19115 (python) S 18602 18602 18093 0 -1 4202560 48 0 0 0 0 0 0 0 15 0 2 0 658130010 436920320 17022 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 514 18446604441293585216 0 0 -1 2 0 0 0
[pid=19094] ppid=18602 vsize=436936 CPUtime=0.28 cores=0,2,4,6
/proc/19094/stat : 19094 (python) S 18602 18602 18093 0 -1 4202560 1135 0 0 0 15 13 0 0 19 0 2 0 658119960 447422464 17026 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347719115 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/19094/statm: 109234 17026 399 364 0 45119 0
[pid=19096] ppid=19094 vsize=471116 CPUtime=98.12 cores=0,2,4,6
/proc/19096/stat : 19096 (TNM) T 19094 19096 19096 0 -1 4202496 8443 0 0 0 9810 2 0 0 25 0 1 0 658119961 482422784 8172 33554432000 134512640 135008748 4292335744 18446744073709551615 134527773 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/19096/statm: 117779 8172 46 122 0 116679 0
[pid=19094/tid=19111] ppid=18602 vsize=436936 CPUtime=0 cores=0,2,4,6
/proc/19094/task/19111/stat : 19111 (python) S 18602 18602 18093 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 658129988 447422464 17026 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 514 18446604449169238912 0 0 -1 2 0 0 0
[pid=19097] ppid=18602 vsize=447192 CPUtime=1.45 cores=0,2,4,6
/proc/19097/stat : 19097 (python) S 18602 18602 18093 0 -1 4202560 1187 0 0 0 75 70 0 0 15 0 2 0 658120168 457924608 17030 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347719115 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/19097/statm: 111798 17030 399 364 0 47680 0
[pid=19099] ppid=19097 vsize=276240 CPUtime=488.68 cores=0,2,4,6
/proc/19099/stat : 19099 (hybridGM3) T 19097 19099 19099 0 -1 4202496 9521 0 0 0 48865 3 0 0 25 0 1 0 658120168 282869760 9250 33554432000 134512640 135204385 4291351472 18446744073709551615 134536744 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/19099/statm: 69060 9250 64 169 0 68890 0
[pid=19097/tid=19126] ppid=18602 vsize=447192 CPUtime=0 cores=0,2,4,6
/proc/19097/task/19126/stat : 19126 (python) S 18602 18602 18093 0 -1 4202560 50 0 0 0 0 0 0 0 15 0 2 0 658140447 457924608 17030 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 514 18446604449169236224 0 0 -1 2 0 0 0
[pid=19112] ppid=18602 vsize=457448 CPUtime=0.56 cores=0,2,4,6
/proc/19112/stat : 19112 (python) S 18602 18602 18093 0 -1 4202560 1197 0 0 0 29 27 0 0 16 0 2 0 658129989 468426752 17034 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347719115 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/19112/statm: 114362 17034 399 364 0 50241 0
[pid=19114] ppid=19112 vsize=276236 CPUtime=194.5 cores=0,2,4,6
/proc/19114/stat : 19114 (hybridGM3) T 19112 19114 19114 0 -1 4202496 9523 0 0 0 19449 1 0 0 25 0 1 0 658129990 282865664 9254 33554432000 134512640 135204385 4293727328 18446744073709551615 134536641 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/19114/statm: 69059 9254 64 169 0 68889 0
[pid=19112/tid=19133] ppid=18602 vsize=457448 CPUtime=0 cores=0,2,4,6
/proc/19112/task/19133/stat : 19133 (python) S 18602 18602 18093 0 -1 4202560 51 0 0 0 0 0 0 0 16 0 2 0 658149666 468426752 17034 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 514 18446604453684229440 0 0 -1 2 0 0 0
[pid=19123] ppid=18602 vsize=467704 CPUtime=0.3 cores=0,2,4,6
/proc/19123/stat : 19123 (python) S 18602 18602 18093 0 -1 4202560 1221 0 0 0 15 15 0 0 19 0 2 0 658140139 478928896 17046 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347719115 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/19123/statm: 116926 17046 407 364 0 52802 0
[pid=19125] ppid=19123 vsize=471116 CPUtime=100.27 cores=0,2,4,6
/proc/19125/stat : 19125 (TNM) T 19123 19125 19125 0 -1 4202496 8446 0 0 0 10026 1 0 0 25 0 1 0 658140140 482422784 8172 33554432000 134512640 135008748 4291406400 18446744073709551615 134529176 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/19125/statm: 117779 8172 46 122 0 116679 0
[pid=19123/tid=19137] ppid=18602 vsize=467704 CPUtime=0 cores=0,2,4,6
/proc/19123/task/19137/stat : 19137 (python) S 18602 18602 18093 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 658150169 478928896 17046 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 514 18446604453684228544 0 0 -1 4 0 0 0
[pid=19134] ppid=18602 vsize=477960 CPUtime=0.58 cores=0,2,4,6
/proc/19134/stat : 19134 (python) S 18602 18602 18093 0 -1 4202560 1068 0 0 0 29 29 0 0 15 0 1 0 658149667 489431040 17020 33554432000 4194304 5683644 140734404324192 18446744073709551615 232335912518 0 0 16781312 514 186582725187043588 0 0 17 2 0 0 0
/proc/19134/statm: 119490 17020 378 364 0 55363 0
[pid=19136] ppid=19134 vsize=276240 CPUtime=195.53 cores=0,2,4,6
/proc/19136/stat : 19136 (hybridGM3) R 19134 19136 19136 0 -1 4202496 9526 0 0 0 19551 2 0 0 25 0 1 0 658149667 282869760 9253 33554432000 134512640 135204385 4286725840 18446744073709551615 134547366 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/19136/statm: 69060 9253 64 169 0 68890 0
[pid=19138] ppid=18602 vsize=488216 CPUtime=0.3 cores=0,2,4,6
/proc/19138/stat : 19138 (python) S 18602 18602 18093 0 -1 4202560 1197 0 0 0 15 15 0 0 19 0 2 0 658150170 499933184 17054 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347719115 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/19138/statm: 122054 17054 407 364 0 57924 0
[pid=19140] ppid=19138 vsize=471116 CPUtime=100.28 cores=0,2,4,6
/proc/19140/stat : 19140 (TNM) T 19138 19140 19140 0 -1 4202496 8447 0 0 0 10027 1 0 0 25 0 1 0 658150171 482422784 8171 33554432000 134512640 135008748 4290139632 18446744073709551615 134527693 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/19140/statm: 117779 8171 46 122 0 116679 0
[pid=19138/tid=19146] ppid=18602 vsize=488216 CPUtime=0 cores=0,2,4,6
/proc/19138/task/19146/stat : 19146 (python) S 18602 18602 18093 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 658160200 499933184 17054 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 514 18446604440904372480 0 0 -1 4 0 0 0
[pid=19147] ppid=18602 vsize=498472 CPUtime=0.28 cores=0,2,4,6
/proc/19147/stat : 19147 (python) S 18602 18602 18093 0 -1 4202560 1021 0 0 0 15 13 0 0 18 0 1 0 658160201 510435328 17022 33554432000 4194304 5683644 140734404324192 18446744073709551615 232335912518 0 0 16781312 514 186582725187043586 0 0 17 6 0 0 0
/proc/19147/statm: 124618 17022 372 364 0 60485 0
[pid=19149] ppid=19147 vsize=471116 CPUtime=91 cores=0,2,4,6
/proc/19149/stat : 19149 (TNM) R 19147 19149 19149 0 -1 4202496 8446 0 0 0 9099 1 0 0 25 0 1 0 658160202 482422784 8170 33554432000 134512640 135008748 4291829968 18446744073709551615 134529176 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/19149/statm: 117779 8170 46 122 0 116679 0
[pid=18602/tid=18609] ppid=18600 vsize=508716 CPUtime=1207.98 cores=0,2,4,6
/proc/18602/task/18609/stat : 18609 (python) S 18600 18602 18093 0 -1 4202560 32 43910 0 0 0 0 120561 237 15 0 19 0 658051047 520925184 18564 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 2 18446604442171689664 0 0 -1 2 0 0 0
[pid=18602/tid=18612] ppid=18600 vsize=508716 CPUtime=1207.98 cores=0,2,4,6
/proc/18602/task/18612/stat : 18612 (python) S 18600 18602 18093 0 -1 4202560 36 43910 0 0 0 0 120561 237 16 0 19 0 658051048 520925184 18564 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 2 18446604442171689664 0 0 -1 6 0 0 0
[pid=18602/tid=18615] ppid=18600 vsize=508716 CPUtime=1207.98 cores=0,2,4,6
/proc/18602/task/18615/stat : 18615 (python) S 18600 18602 18093 0 -1 4202560 30 43910 0 0 0 0 120561 237 18 0 19 0 658051049 520925184 18564 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 2 18446604442171689664 0 0 -1 6 0 0 0
[pid=18602/tid=18618] ppid=18600 vsize=508716 CPUtime=1207.98 cores=0,2,4,6
/proc/18602/task/18618/stat : 18618 (python) S 18600 18602 18093 0 -1 4202560 32 43910 0 0 0 0 120561 237 18 0 19 0 658051050 520925184 18564 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 2 18446604442171689664 0 0 -1 4 0 0 0
[pid=18602/tid=18665] ppid=18600 vsize=508716 CPUtime=1207.98 cores=0,2,4,6
/proc/18602/task/18665/stat : 18665 (python) S 18600 18602 18093 0 -1 4202560 31 43910 0 0 0 0 120561 237 18 0 19 0 658070611 520925184 18564 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 2 18446604442171689664 0 0 -1 2 0 0 0
[pid=18602/tid=18684] ppid=18600 vsize=508716 CPUtime=1207.98 cores=0,2,4,6
/proc/18602/task/18684/stat : 18684 (python) S 18600 18602 18093 0 -1 4202560 34 43910 0 0 0 0 120561 237 18 0 19 0 658080735 520925184 18564 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 2 18446604442171689664 0 0 -1 2 0 0 0
[pid=18602/tid=18687] ppid=18600 vsize=508716 CPUtime=1207.98 cores=0,2,4,6
/proc/18602/task/18687/stat : 18687 (python) S 18600 18602 18093 0 -1 4202560 34 43910 0 0 0 0 120561 237 18 0 19 0 658080740 520925184 18564 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 2 18446604442171689664 0 0 -1 2 0 0 0
[pid=18602/tid=18691] ppid=18600 vsize=508716 CPUtime=1207.98 cores=0,2,4,6
/proc/18602/task/18691/stat : 18691 (python) S 18600 18602 18093 0 -1 4202560 36 43910 0 0 0 0 120561 237 18 0 19 0 658081540 520925184 18564 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 2 18446604442171689664 0 0 -1 2 0 0 0
[pid=18602/tid=18701] ppid=18600 vsize=508716 CPUtime=1207.98 cores=0,2,4,6
/proc/18602/task/18701/stat : 18701 (python) S 18600 18602 18093 0 -1 4202560 25 43910 0 0 0 0 120561 237 18 0 19 0 658090366 520925184 18564 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 2 18446604442171689664 0 0 -1 2 0 0 0
[pid=18602/tid=19047] ppid=18600 vsize=508716 CPUtime=1207.98 cores=0,2,4,6
/proc/18602/task/19047/stat : 19047 (python) S 18600 18602 18093 0 -1 4202560 25 43910 0 0 0 0 120561 237 20 0 19 0 658100305 520925184 18564 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 2 18446604442171689664 0 0 -1 2 0 0 0
[pid=18602/tid=19073] ppid=18600 vsize=508716 CPUtime=1207.98 cores=0,2,4,6
/proc/18602/task/19073/stat : 19073 (python) S 18600 18602 18093 0 -1 4202560 30 43910 0 0 0 0 120561 237 18 0 19 0 658110335 520925184 18564 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 2 18446604442171689664 0 0 -1 2 0 0 0
[pid=18602/tid=19095] ppid=18600 vsize=508716 CPUtime=1207.98 cores=0,2,4,6
/proc/18602/task/19095/stat : 19095 (python) S 18600 18602 18093 0 -1 4202560 28 43910 0 0 0 0 120561 237 20 0 19 0 658119960 520925184 18564 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 2 18446604442171689664 0 0 -1 2 0 0 0
[pid=18602/tid=19098] ppid=18600 vsize=508716 CPUtime=1207.98 cores=0,2,4,6
/proc/18602/task/19098/stat : 19098 (python) S 18600 18602 18093 0 -1 4202560 41 43910 0 0 0 0 120561 237 18 0 19 0 658120168 520925184 18564 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 2 18446604442171689664 0 0 -1 2 0 0 0
[pid=18602/tid=19113] ppid=18600 vsize=508716 CPUtime=1207.98 cores=0,2,4,6
/proc/18602/task/19113/stat : 19113 (python) S 18600 18602 18093 0 -1 4202560 29 43910 0 0 0 0 120561 237 20 0 19 0 658129990 520925184 18564 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 2 18446604442171689664 0 0 -1 2 0 0 0
[pid=18602/tid=19124] ppid=18600 vsize=508716 CPUtime=1207.98 cores=0,2,4,6
/proc/18602/task/19124/stat : 19124 (python) S 18600 18602 18093 0 -1 4202560 31 43910 0 0 0 0 120561 237 20 0 19 0 658140140 520925184 18564 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 2 18446604442171689664 0 0 -1 2 0 0 0
[pid=18602/tid=19135] ppid=18600 vsize=508716 CPUtime=1207.98 cores=0,2,4,6
/proc/18602/task/19135/stat : 19135 (python) S 18600 18602 18093 0 -1 4202560 27 43910 0 0 0 0 120561 237 20 0 19 0 658149667 520925184 18564 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 2 18446604442171689664 0 0 -1 2 0 0 0
[pid=18602/tid=19139] ppid=18600 vsize=508716 CPUtime=1207.98 cores=0,2,4,6
/proc/18602/task/19139/stat : 19139 (python) S 18600 18602 18093 0 -1 4202560 25 43910 0 0 0 0 120561 237 20 0 19 0 658150171 520925184 18564 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 2 18446604442171689664 0 0 -1 2 0 0 0
[pid=18602/tid=19148] ppid=18600 vsize=508716 CPUtime=1207.98 cores=0,2,4,6
/proc/18602/task/19148/stat : 19148 (python) S 18600 18602 18093 0 -1 4202560 27 43910 0 0 0 0 120561 237 20 0 19 0 658160202 520925184 18564 33554432000 4194304 5683644 140734404324192 18446744073709551615 232347716017 0 0 16781312 2 18446604442171689664 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4728.11
Current children cumulated vsize (KiB) 11529108

Child status: 0
Real time (s): 1197.1
CPU time (s): 4728.68
CPU user time (s): 4720.93
CPU system time (s): 7.74582
CPU usage (%): 395.013
Max. virtual memory (cumulated for all children) (KiB): 13391720

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4720.93
system time used= 7.74582
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 240713
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 9588
involuntary context switches= 69545

runsolver used 7.09092 second user time and 17.9633 second system time

The end

Launcher Data 

Begin job on node132 at 2011-05-15 15:01:58
IDJOB=3378289
IDBENCH=82698
IDSOLVER=1859
FILE ID=node132/3378289-1305464517
RUNJOBID= node132-1305463312-18111
PBS_JOBID= 13324991
Free space on /tmp= 73316 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/large/unif-k3-r4.2-v50000-c210000-S1170024351-015-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3378289-1305464517/watcher-3378289-1305464517 -o /tmp/evaluation-result-3378289-1305464517/solver-3378289-1305464517 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378289-1305464517.cnf -seed 1113523622 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 0f7796213ec78265bd44aa83c0fa283d
RANDOM SEED=1113523622

node132.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      20288328 kB
Buffers:        302920 kB
Cached:        5992880 kB
SwapCached:       6904 kB
Active:        5907972 kB
Inactive:      6431428 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      20288328 kB
SwapTotal:    67111528 kB
SwapFree:     66457188 kB
Dirty:           36148 kB
Writeback:           0 kB
AnonPages:     6041788 kB
Mapped:          27744 kB
Slab:           226548 kB
PageTables:      33128 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 53661356 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73240 MiB
End job on node132 at 2011-05-15 15:21:55