Trace number 3321859

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-random? (MO) 8313.35 1183.71

General information on the benchmark

Namerandom/large/
unif-k5-r20-v2000-c40000-S1993071608-084-UNKNOWN.cnf
MD5SUM6a416a8eca39558555cbe083cf0c9cff
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark559.28
Satisfiable
(Un)Satisfiability was proved
Number of variables2000
Number of clauses40000
Sum of the clauses size200000
Maximum clause length5
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 540000
Number of clauses of size over 50

Solver Data 

0.35/0.68	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.35/0.68	c INFO: solving HOME/instance-3321859-1304368483.cnf
0.35/0.68	c INFO: computing task features
0.53/0.89	c INFO: computed variable-clause graph statistics
0.53/0.89	c INFO: computed clause balance statistics
1.63/1.95	c INFO: computed variable balance statistics
2.23/2.53	c INFO: computed small-clause counts
2.23/2.53	c INFO: computed Horn-clause counts
2.23/2.58	c INFO: computed variable graph statistics
2.23/2.58	c INFO: collected features for HOME/instance-3321859-1304368483.cnf in 1.89 s
2.33/2.62	c INFO: running gnovelty+2@0 for 97 with 39998 remaining (b = 0.38)
2.33/2.65	c INFO: running gnovelty+2@0 for 97 with 39998 remaining (b = 0.13)
2.33/2.69	c INFO: running adaptg2wsat2009++@0 for 97 with 39998 remaining (b = 0.08)
2.43/2.73	c INFO: running adaptg2wsat2009++@0 for 97 with 39997 remaining (b = 0.07)
2.43/2.77	c INFO: running adaptg2wsat2009++@0 for 97 with 39997 remaining (b = 0.05)
2.52/2.80	c INFO: running gnovelty+2@0 for 97 with 39997 remaining (b = 0.05)
2.52/2.84	c INFO: running FH@0 for 97 with 39997 remaining (b = 0.03)
2.60/2.91	c INFO: running adaptg2wsat2009++@0 for 97 with 39997 remaining (b = 0.03)
701.86/102.88	c INFO: running adaptg2wsat2009++@0 for 97 with 39897 remaining (b = 0.02)
702.57/102.93	c INFO: running FH@0 for 97 with 39897 remaining (b = 0.02)
702.57/102.98	c INFO: running hybridGM3@0 for 194 with 39897 remaining (b = 0.04)
703.10/103.03	c INFO: running gnovelty+2@0 for 97 with 39897 remaining (b = 0.02)
703.10/103.09	c INFO: running hybridGM3@0 for 194 with 39897 remaining (b = 0.03)
703.51/103.14	c INFO: running adaptg2wsat2009++@0 for 97 with 39897 remaining (b = 0.01)
1038.37/151.05	c INFO: running hybridGM3@0 for 194 with 39849 remaining (b = 0.03)
1083.04/157.41	c INFO: running hybridGM3@0 for 194 with 39843 remaining (b = 0.02)
1401.87/203.12	c INFO: running TNM@0 for 1552 with 39797 remaining (b = 0.19)
1403.11/203.40	c INFO: running FH@0 for 97 with 39797 remaining (b = 0.01)
1487.68/215.41	c INFO: running adaptg2wsat2009++@0 for 97 with 39785 remaining (b = 0.01)
1560.36/225.26	c INFO: running adaptg2wsat2009++@0 for 97 with 39775 remaining (b = 0.01)
2133.79/305.85	c INFO: running hybridGM3@100 for 97 with 39694 remaining (b = 0.01)
2223.35/318.68	c INFO: running adaptg2wsat2009++@0 for 97 with 39682 remaining (b = 0.01)
2280.08/326.77	c INFO: running adaptg2wsat2009++@100 for 97 with 39673 remaining (b = 0.01)
2331.42/333.64	c INFO: running hybridGM3@0 for 194 with 39667 remaining (b = 0.02)
2513.46/359.64	c INFO: running hybridGM3@102 for 97 with 39641 remaining (b = 0.01)
2677.21/383.03	c INFO: running hybridGM3@99 for 97 with 39617 remaining (b = 0.01)
2717.07/388.76	c INFO: running hybridGM3@99 for 97 with 39611 remaining (b = 0.01)
2907.51/415.94	c INFO: running hybridGM3@0 for 194 with 39584 remaining (b = 0.02)
2964.20/424.04	c INFO: running TNM@0 for 1552 with 39576 remaining (b = 0.15)
3139.87/449.19	c INFO: running adaptg2wsat2009++@100 for 97 with 39551 remaining (b = 0.01)
3385.56/484.30	c INFO: running hybridGM3@295 for 97 with 39516 remaining (b = 0.01)
3426.13/490.02	c INFO: running FH@0 for 291 with 39510 remaining (b = 0.03)
3478.19/496.98	c INFO: running gnovelty+2@0 for 97 with 39503 remaining (b = 0.01)
3817.09/542.30	c INFO: running hybridGM3@296 for 97 with 39458 remaining (b = 0.01)
3971.79/564.49	c INFO: running iPAWS@0 for 97 with 39436 remaining (b = 0.01)
4070.04/578.58	c INFO: running hybridGM3@396 for 97 with 39422 remaining (b = 0.01)
4200.24/597.31	c INFO: running adaptg2wsat2009++@100 for 97 with 39403 remaining (b = 0.01)
4508.72/641.57	c INFO: running hybridGM3@96 for 97 with 39359 remaining (b = 0.01)
4578.35/651.60	c INFO: running hybridGM3@391 for 97 with 39349 remaining (b = 0.01)
4769.17/678.93	c INFO: running hybridGM3@0 for 194 with 39321 remaining (b = 0.02)
4878.46/694.62	c INFO: running adaptg2wsat2009++@294 for 291 with 39306 remaining (b = 0.02)
5005.30/712.86	c INFO: running hybridGM3@0 for 194 with 39287 remaining (b = 0.01)
5215.19/742.84	c INFO: running TNM@0 for 1552 with 39257 remaining (b = 0.11)
5494.33/782.94	c INFO: running FH@100 for 291 with 39217 remaining (b = 0.02)
5942.80/844.78	c INFO: running iPAWS@0 for 97 with 39155 remaining (b = 0.01)
6156.52/875.49	c INFO: running hybridGM3@96 for 97 with 39125 remaining (b = 0.01)
6644.70/945.52	c INFO: running hybridGM3@100 for 97 with 39055 remaining (b = 0.01)
6682.98/951.10	c INFO: running gnovelty+2@0 for 97 with 39049 remaining (b = 0.01)
7081.34/1006.85	c INFO: running hybridGM3@0 for 194 with 38993 remaining (b = 0.01)
7279.69/1035.39	c INFO: running adaptg2wsat2009++@391 for 194 with 38965 remaining (b = 0.01)
7404.58/1052.84	c INFO: running FH@0 for 97 with 38947 remaining (b = 0.01)
7437.27/1057.51	c INFO: running hybridGM3@0 for 194 with 38943 remaining (b = 0.01)
8197.14/1166.84	c INFO: running NCVWr@0 for 485 with 38833 remaining (b = 0.03)
8311.84/1183.31	c INFO: running hybridGM3@0 for 194 with 38817 remaining (b = 0.01)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3321859-1304368483/watcher-3321859-1304368483 -o /tmp/evaluation-result-3321859-1304368483/solver-3321859-1304368483 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3321859-1304368483.cnf -seed 206775748 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.24 8.40 7.29 2/241 7042
/proc/meminfo: memFree=17410780/32951124 swapFree=62859076/67111528
[pid=7042] ppid=7040 vsize=15380 CPUtime=0 cores=0-7
/proc/7042/stat : 7042 (python) S 7040 7042 6630 0 -1 4202496 819 0 0 0 0 0 0 0 23 0 1 0 548448769 15749120 704 33554432000 4194304 5683644 140734935248912 18446744073709551615 244446943232 0 2147483391 16781312 2 18446744071699199166 0 0 17 4 0 0 0
/proc/7042/statm: 3845 704 334 364 0 332 0

[startup+0.0163109 s]
/proc/loadavg: 8.24 8.40 7.29 2/241 7042
/proc/meminfo: memFree=17410780/32951124 swapFree=62859076/67111528
[pid=7042] ppid=7040 vsize=15904 CPUtime=0 cores=0-7
/proc/7042/stat : 7042 (python) S 7040 7042 6630 0 -1 4202496 957 0 0 0 0 0 0 0 23 0 1 0 548448769 16285696 833 33554432000 4194304 5683644 140734935248912 18446744073709551615 244446943232 0 2147483391 16781312 2 18446744071699199166 0 0 17 4 0 0 0
/proc/7042/statm: 3976 833 357 364 0 463 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 15904

[startup+0.100267 s]
/proc/loadavg: 8.24 8.40 7.29 2/241 7042
/proc/meminfo: memFree=17410780/32951124 swapFree=62859076/67111528
[pid=7042] ppid=7040 vsize=85616 CPUtime=0.07 cores=0-7
/proc/7042/stat : 7042 (python) S 7040 7042 6630 0 -1 4202496 2628 0 0 0 5 2 0 0 18 0 1 0 548448769 87670784 2347 33554432000 4194304 5683644 140734935248912 18446744073709551615 244446943232 0 2147483391 16781312 2 18446744071699199166 0 0 17 4 0 0 0
/proc/7042/statm: 21404 2347 713 364 0 1621 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 85616

[startup+0.300266 s]
/proc/loadavg: 8.24 8.40 7.29 2/241 7042
/proc/meminfo: memFree=17410780/32951124 swapFree=62859076/67111528
[pid=7042] ppid=7040 vsize=135748 CPUtime=0.2 cores=0-7
/proc/7042/stat : 7042 (python) R 7040 7042 6630 0 -1 4202496 5962 937 1 0 14 6 0 0 17 0 1 0 548448769 139005952 4151 33554432000 4194304 5683644 140734935248912 18446744073709551615 4620076 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/7042/statm: 33937 4151 1032 364 0 3123 0
Current children cumulated CPU time (s) 0.2
Current children cumulated vsize (KiB) 135748

[startup+0.700167 s]
/proc/loadavg: 8.24 8.40 7.29 2/241 7042
/proc/meminfo: memFree=17410780/32951124 swapFree=62859076/67111528
[pid=7042] ppid=7040 vsize=280156 CPUtime=0.43 cores=0-7
/proc/7042/stat : 7042 (python) R 7040 7042 6630 0 -1 4202496 9671 937 1 0 31 12 0 0 16 0 1 0 548448769 286879744 7354 33554432000 4194304 5683644 140734935248912 18446744073709551615 4602711 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/7042/statm: 70039 7354 1761 364 0 5963 0
Current children cumulated CPU time (s) 0.43
Current children cumulated vsize (KiB) 280156

[startup+1.50101 s]
/proc/loadavg: 8.24 8.40 7.29 3/242 7047
/proc/meminfo: memFree=17376488/32951124 swapFree=62859076/67111528
[pid=7042] ppid=7040 vsize=290796 CPUtime=1.23 cores=0-7
/proc/7042/stat : 7042 (python) R 7040 7042 6630 0 -1 4202496 14007 937 1 0 110 13 0 0 17 0 1 0 548448769 297775104 10084 33554432000 4194304 5683644 140734935248912 18446744073709551615 4500451 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/7042/statm: 72699 10084 1826 364 0 8623 0
Current children cumulated CPU time (s) 1.23
Current children cumulated vsize (KiB) 290796

[startup+3.10068 s]
/proc/loadavg: 7.82 8.31 7.26 3/242 7047
/proc/meminfo: memFree=17375992/32951124 swapFree=62859076/67111528
[pid=7042] ppid=7040 vsize=432068 CPUtime=2.61 cores=0-7
/proc/7042/stat : 7042 (python) S 7040 7042 6630 0 -1 4202496 20156 937 1 0 246 15 0 0 22 0 9 0 548448769 442437632 8743 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458771915 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/7042/statm: 108017 8743 1891 364 0 43914 0
Current children cumulated CPU time (s) 2.61
Current children cumulated vsize (KiB) 432068

[startup+6.30108 s]
/proc/loadavg: 7.82 8.31 7.26 10/266 7071
/proc/meminfo: memFree=17299052/32951124 swapFree=62859076/67111528
[pid=7042] ppid=7040 vsize=432068 CPUtime=2.61 cores=0-7
/proc/7042/stat : 7042 (python) S 7040 7042 6630 0 -1 4202496 20156 937 1 0 246 15 0 0 22 0 9 0 548448769 442437632 8743 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458771915 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/7042/statm: 108017 8743 1891 364 0 43914 0
[pid=7048] ppid=7042 vsize=284980 CPUtime=0 cores=0-7
/proc/7048/stat : 7048 (python) S 7042 7042 6630 0 -1 4202560 738 0 0 0 0 0 0 0 18 0 1 0 548449031 291819520 6898 33554432000 4194304 5683644 140734935248912 18446744073709551615 244446965263 0 0 16781312 514 4092137916417933569 0 0 17 4 0 0 0
/proc/7048/statm: 71245 6898 355 364 0 7163 0
[pid=7050] ppid=7048 vsize=1020308 CPUtime=3.67 cores=0-7
/proc/7050/stat : 7050 (gnovelty+2) R 7048 7050 7050 0 -1 4202496 1157 0 0 0 367 0 0 0 25 0 1 0 548449031 1044795392 892 33554432000 134512640 135663704 4288254880 18446744073709551615 134525328 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/7050/statm: 255077 892 113 282 0 254793 0
[pid=7051] ppid=7042 vsize=294900 CPUtime=0 cores=0-7
/proc/7051/stat : 7051 (python) S 7042 7042 6630 0 -1 4202560 739 0 0 0 0 0 0 0 18 0 1 0 548449034 301977600 7031 33554432000 4194304 5683644 140734935248912 18446744073709551615 244446965318 0 0 16781312 514 7030462070248145155 0 0 17 1 0 0 0
/proc/7051/statm: 73725 7031 349 364 0 9640 0
[pid=7053] ppid=7051 vsize=1020308 CPUtime=3.63 cores=0-7
/proc/7053/stat : 7053 (gnovelty+2) R 7051 7053 7053 0 -1 4202496 1152 0 0 0 363 0 0 0 25 0 1 0 548449035 1044795392 891 33554432000 134512640 135663704 4288938880 18446744073709551615 134525286 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/7053/statm: 255077 891 113 282 0 254793 0
[pid=7054] ppid=7042 vsize=305764 CPUtime=0 cores=0-7
/proc/7054/stat : 7054 (python) S 7042 7042 6630 0 -1 4202560 742 0 0 0 0 0 0 0 18 0 1 0 548449038 313102336 7035 33554432000 4194304 5683644 140734935248912 18446744073709551615 244446965318 0 0 16781312 514 5501623587418636548 0 0 17 1 0 0 0
/proc/7054/statm: 76441 7035 349 364 0 12353 0
[pid=7056] ppid=7054 vsize=423400 CPUtime=3.59 cores=0-7
/proc/7056/stat : 7056 (adaptg2wsat2009) R 7054 7056 7056 0 -1 4202496 2621 0 0 0 359 0 0 0 25 0 1 0 548449038 433561600 2358 33554432000 134512640 135107411 4293403728 18446744073709551615 134531600 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/7056/statm: 105850 2358 59 146 0 104726 0
[pid=7057] ppid=7042 vsize=380800 CPUtime=0 cores=0-7
/proc/7057/stat : 7057 (python) S 7042 7042 6630 0 -1 4202560 707 0 0 0 0 0 0 0 18 0 1 0 548449042 389939200 7210 33554432000 4194304 5683644 140734935248912 18446744073709551615 244446965318 0 0 16781312 514 4129475956907671812 0 0 17 1 0 0 0
/proc/7057/statm: 95200 7210 348 364 0 31109 0
[pid=7059] ppid=7057 vsize=423400 CPUtime=3.5 cores=0-7
/proc/7059/stat : 7059 (adaptg2wsat2009) R 7057 7059 7059 0 -1 4202496 2608 0 0 0 350 0 0 0 25 0 1 0 548449042 433561600 2357 33554432000 134512640 135107411 4294335888 18446744073709551615 134531600 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/7059/statm: 105850 2357 59 146 0 104726 0
[pid=7060] ppid=7042 vsize=391056 CPUtime=0 cores=0-7
/proc/7060/stat : 7060 (python) S 7042 7042 6630 0 -1 4202560 720 0 0 0 0 0 0 0 18 0 1 0 548449046 400441344 7184 33554432000 4194304 5683644 140734935248912 18446744073709551615 244446965318 0 0 16781312 514 4165754068188233985 0 0 17 1 0 0 0
/proc/7060/statm: 97764 7184 348 364 0 33670 0
[pid=7062] ppid=7060 vsize=423400 CPUtime=3.49 cores=0-7
/proc/7062/stat : 7062 (adaptg2wsat2009) R 7060 7062 7062 0 -1 4202496 2616 0 0 0 349 0 0 0 25 0 1 0 548449046 433561600 2359 33554432000 134512640 135107411 4289196336 18446744073709551615 134531736 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/7062/statm: 105850 2359 59 146 0 104726 0
[pid=7063] ppid=7042 vsize=401312 CPUtime=0 cores=0-7
/proc/7063/stat : 7063 (python) S 7042 7042 6630 0 -1 4202560 727 0 0 0 0 0 0 0 18 0 1 0 548449050 410943488 7155 33554432000 4194304 5683644 140734935248912 18446744073709551615 244446965318 0 0 16781312 514 4058894732107940100 0 0 17 1 0 0 0
/proc/7063/statm: 100328 7155 348 364 0 36231 0
[pid=7065] ppid=7063 vsize=1020308 CPUtime=3.44 cores=0-7
/proc/7065/stat : 7065 (gnovelty+2) R 7063 7065 7065 0 -1 4202496 1148 0 0 0 344 0 0 0 25 0 1 0 548449052 1044795392 892 33554432000 134512640 135663704 4288828688 18446744073709551615 134525286 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/7065/statm: 255077 892 113 282 0 254793 0
[pid=7066] ppid=7042 vsize=411568 CPUtime=0 cores=0-7
/proc/7066/stat : 7066 (python) S 7042 7042 6630 0 -1 4202560 734 0 0 0 0 0 0 0 18 0 1 0 548449053 421445632 7287 33554432000 4194304 5683644 140734935248912 18446744073709551615 244446965318 0 0 16781312 514 3022209748498874625 0 0 17 1 0 0 0
/proc/7066/statm: 102892 7287 348 364 0 38792 0
[pid=7068] ppid=7066 vsize=253032 CPUtime=1.81 cores=0-7
/proc/7068/stat : 7068 (FH) R 7066 7068 7068 0 -1 4202496 2491 0 0 0 181 0 0 0 25 0 1 0 548449054 259104768 2235 33554432000 134512640 134936988 4288319584 18446744073709551615 134555005 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/7068/statm: 63258 2235 41 104 0 62762 0
[pid=7069] ppid=7042 vsize=421824 CPUtime=0 cores=0-7
/proc/7069/stat : 7069 (python) S 7042 7042 6630 0 -1 4202560 766 0 0 0 0 0 0 0 18 0 1 0 548449060 431947776 7202 33554432000 4194304 5683644 140734935248912 18446744073709551615 244446965318 0 0 16781312 514 9973428362170827012 0 0 17 1 0 0 0
/proc/7069/statm: 105456 7202 354 364 0 41353 0
[pid=7071] ppid=7069 vsize=423400 CPUtime=1.6 cores=0-7
/proc/7071/stat : 7071 (adaptg2wsat2009) R 7069 7071 7071 0 -1 4202496 2620 0 0 0 160 0 0 0 25 0 1 0 548449070 433561600 2358 33554432000 134512640 135107411 4290017296 18446744073709551615 134531781 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/7071/statm: 105850 2358 59 146 0 104726 0
[pid=7042/tid=7049] ppid=7040 vsize=432068 CPUtime=0 cores=0-7
/proc/7042/task/7049/stat : 7049 (python) S 7040 7042 6630 0 -1 4202560 27 937 0 0 0 0 0 0 20 0 9 0 548449031 442437632 8743 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 4 0 0 0
[pid=7042/tid=7052] ppid=7040 vsize=432068 CPUtime=0 cores=0-7
/proc/7042/task/7052/stat : 7052 (python) S 7040 7042 6630 0 -1 4202560 24 937 0 0 0 0 0 0 21 0 9 0 548449035 442437632 8743 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 0 0 0 0
[pid=7042/tid=7055] ppid=7040 vsize=432068 CPUtime=0 cores=0-7
/proc/7042/task/7055/stat : 7055 (python) S 7040 7042 6630 0 -1 4202560 22 937 0 0 0 0 0 0 18 0 9 0 548449038 442437632 8743 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 5 0 0 0
[pid=7042/tid=7058] ppid=7040 vsize=432068 CPUtime=0 cores=0-7
/proc/7042/task/7058/stat : 7058 (python) S 7040 7042 6630 0 -1 4202560 94 937 0 0 0 0 0 0 19 0 9 0 548449042 442437632 8743 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 0 0 0 0
[pid=7042/tid=7061] ppid=7040 vsize=432068 CPUtime=0 cores=0-7
/proc/7042/task/7061/stat : 7061 (python) S 7040 7042 6630 0 -1 4202560 31 937 0 0 0 0 0 0 18 0 9 0 548449046 442437632 8743 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 7 0 0 0
[pid=7042/tid=7064] ppid=7040 vsize=432068 CPUtime=0 cores=0-7
/proc/7042/task/7064/stat : 7064 (python) S 7040 7042 6630 0 -1 4202560 31 937 0 0 0 0 0 0 21 0 9 0 548449050 442437632 8743 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 2 0 0 0
[pid=7042/tid=7067] ppid=7040 vsize=432068 CPUtime=0 cores=0-7
/proc/7042/task/7067/stat : 7067 (python) S 7040 7042 6630 0 -1 4202560 31 937 0 0 0 0 0 0 22 0 9 0 548449054 442437632 8743 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 1 0 0 0
[pid=7042/tid=7070] ppid=7040 vsize=432068 CPUtime=0 cores=0-7
/proc/7042/task/7070/stat : 7070 (python) S 7040 7042 6630 0 -1 4202560 31 937 0 0 0 0 0 0 23 0 9 0 548449060 442437632 8743 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 27.34
Current children cumulated vsize (KiB) 8331828

[startup+12.7019 s]
/proc/loadavg: 8.00 8.34 7.28 10/266 7071
/proc/meminfo: memFree=17298172/32951124 swapFree=62859076/67111528
[pid=7042] ppid=7040 vsize=432068 CPUtime=2.61 cores=0-7
/proc/7042/stat : 7042 (python) S 7040 7042 6630 0 -1 4202496 20156 937 1 0 246 15 0 0 22 0 9 0 548448769 442437632 8743 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458771915 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/7042/statm: 108017 8743 1891 364 0 43914 0
[pid=7048] ppid=7042 vsize=284980 CPUtime=0.02 cores=0-7
/proc/7048/stat : 7048 (python) S 7042 7042 6630 0 -1 4202560 904 0 0 0 1 1 0 0 18 0 1 0 548449031 291819520 6921 33554432000 4194304 5683644 140734935248912 18446744073709551615 244446965263 0 0 16781312 514 718412101495783681 0 0 17 4 0 0 0
/proc/7048/statm: 71245 6921 378 364 0 7163 0
[pid=7050] ppid=7048 vsize=1020308 CPUtime=10.05 cores=0-7
/proc/7050/stat : 7050 (gnovelty+2) R 7048 7050 7050 0 -1 4202496 1157 0 0 0 1005 0 0 0 25 0 1 0 548449031 1044795392 892 33554432000 134512640 135663704 4288254880 18446744073709551615 134545631 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/7050/statm: 255077 892 113 282 0 254793 0
[pid=7051] ppid=7042 vsize=294900 CPUtime=0 cores=0-7
/proc/7051/stat : 7051 (python) S 7042 7042 6630 0 -1 4202560 898 0 0 0 0 0 0 0 18 0 1 0 548449034 301977600 7054 33554432000 4194304 5683644 140734935248912 18446744073709551615 244446965318 0 0 16781312 514 7030462070248145155 0 0 17 1 0 0 0
/proc/7051/statm: 73725 7054 372 364 0 9640 0
[pid=7053] ppid=7051 vsize=1020308 CPUtime=10.03 cores=0-7
/proc/7053/stat : 7053 (gnovelty+2) R 7051 7053 7053 0 -1 4202496 1152 0 0 0 1003 0 0 0 25 0 1 0 548449035 1044795392 891 33554432000 134512640 135663704 4288938880 18446744073709551615 134524918 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/7053/statm: 255077 891 113 282 0 254793 0
[pid=7054] ppid=7042 vsize=305764 CPUtime=0 cores=0-7
/proc/7054/stat : 7054 (python) S 7042 7042 6630 0 -1 4202560 918 0 0 0 0 0 0 0 18 0 1 0 548449038 313102336 7057 33554432000 4194304 5683644 140734935248912 18446744073709551615 244446965318 0 0 16781312 514 5501623587418636548 0 0 17 1 0 0 0
/proc/7054/statm: 76441 7057 371 364 0 12353 0
[pid=7056] ppid=7054 vsize=423400 CPUtime=9.99 cores=0-7
/proc/7056/stat : 7056 (adaptg2wsat2009) R 7054 7056 7056 0 -1 4202496 2621 0 0 0 999 0 0 0 25 0 1 0 548449038 433561600 2358 33554432000 134512640 135107411 4293403728 18446744073709551615 134539327 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/7056/statm: 105850 2358 59 146 0 104726 0
[pid=7057] ppid=7042 vsize=380800 CPUtime=0 cores=0-7
/proc/7057/stat : 7057 (python) S 7042 7042 6630 0 -1 4202560 784 0 0 0 0 0 0 0 18 0 1 0 548449042 389939200 7233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244446965318 0 0 16781312 514 4129475956907671812 0 0 17 1 0 0 0
/proc/7057/statm: 95200 7233 371 364 0 31109 0
[pid=7059] ppid=7057 vsize=423400 CPUtime=6.91 cores=0-7
/proc/7059/stat : 7059 (adaptg2wsat2009) R 7057 7059 7059 0 -1 4202496 2608 0 0 0 691 0 0 0 25 0 1 0 548449042 433561600 2357 33554432000 134512640 135107411 4294335888 18446744073709551615 134539363 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/7059/statm: 105850 2357 59 146 0 104726 0
[pid=7060] ppid=7042 vsize=391056 CPUtime=0 cores=0-7
/proc/7060/stat : 7060 (python) S 7042 7042 6630 0 -1 4202560 797 0 0 0 0 0 0 0 18 0 1 0 548449046 400441344 7207 33554432000 4194304 5683644 140734935248912 18446744073709551615 244446965318 0 0 16781312 514 4165754068188233985 0 0 17 1 0 0 0
/proc/7060/statm: 97764 7207 371 364 0 33670 0
[pid=7062] ppid=7060 vsize=423400 CPUtime=9.83 cores=0-7
/proc/7062/stat : 7062 (adaptg2wsat2009) R 7060 7062 7062 0 -1 4202496 2616 0 0 0 983 0 0 0 25 0 1 0 548449046 433561600 2359 33554432000 134512640 135107411 4289196336 18446744073709551615 134532277 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/7062/statm: 105850 2359 59 146 0 104726 0
[pid=7063] ppid=7042 vsize=401312 CPUtime=0 cores=0-7
/proc/7063/stat : 7063 (python) S 7042 7042 6630 0 -1 4202560 803 0 0 0 0 0 0 0 18 0 1 0 548449050 410943488 7178 33554432000 4194304 5683644 140734935248912 18446744073709551615 244446965318 0 0 16781312 514 4058894732107940100 0 0 17 1 0 0 0
/proc/7063/statm: 100328 7178 371 364 0 36231 0
[pid=7065] ppid=7063 vsize=1020308 CPUtime=9.84 cores=0-7
/proc/7065/stat : 7065 (gnovelty+2) R 7063 7065 7065 0 -1 4202496 1148 0 0 0 984 0 0 0 25 0 1 0 548449052 1044795392 892 33554432000 134512640 135663704 4288828688 18446744073709551615 134544212 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/7065/statm: 255077 892 113 282 0 254793 0
[pid=7066] ppid=7042 vsize=411568 CPUtime=0 cores=0-7
/proc/7066/stat : 7066 (python) S 7042 7042 6630 0 -1 4202560 817 0 0 0 0 0 0 0 18 0 1 0 548449053 421445632 7310 33554432000 4194304 5683644 140734935248912 18446744073709551615 244446965318 0 0 16781312 514 3022209748498874625 0 0 17 1 0 0 0
/proc/7066/statm: 102892 7310 371 364 0 38792 0

################
# More data... #
################

/proc/7138/stat : 7138 (TNM) R 7136 7138 7138 0 -1 4202496 2468 0 0 0 67993 0 0 0 25 0 1 0 548491173 472588288 2195 33554432000 134512640 135008748 4290320304 18446744073709551615 134527526 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/7138/statm: 115378 2195 46 122 0 114278 0
[pid=7141] ppid=7042 vsize=565408 CPUtime=0.2 cores=0-7
/proc/7141/stat : 7141 (python) S 7042 7042 6630 0 -1 4202560 1175 0 0 0 10 10 0 0 19 0 2 0 548498467 578977792 7556 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/7141/statm: 141352 7556 407 364 0 77207 0
[pid=7143] ppid=7141 vsize=1020308 CPUtime=99.56 cores=0-7
/proc/7143/stat : 7143 (gnovelty+2) T 7141 7143 7143 0 -1 4202496 1170 0 0 0 9956 0 0 0 25 0 1 0 548498467 1044795392 891 33554432000 134512640 135663704 4291083744 18446744073709551615 134544212 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/7143/statm: 255077 891 113 282 0 254793 0
[pid=7141/tid=7151] ppid=7042 vsize=565408 CPUtime=0 cores=0-7
/proc/7141/task/7151/stat : 7151 (python) S 7042 7042 6630 0 -1 4202560 90 0 0 0 0 0 0 0 19 0 2 0 548508491 578977792 7556 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 514 18446604448992303936 0 0 -1 6 0 0 0
[pid=7147] ppid=7042 vsize=575664 CPUtime=0.3 cores=0-7
/proc/7147/stat : 7147 (python) S 7042 7042 6630 0 -1 4202560 1184 0 0 0 16 14 0 0 19 0 2 0 548505218 589479936 7608 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/7147/statm: 143916 7608 407 364 0 79768 0
[pid=7149] ppid=7147 vsize=335996 CPUtime=97.4 cores=0-7
/proc/7149/stat : 7149 (iPAWS) T 7147 7149 7149 0 -1 4202496 2054 0 0 0 9740 0 0 0 25 0 1 0 548505219 344059904 1773 33554432000 134512640 135114423 4290321728 18446744073709551615 134515508 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/7149/statm: 83999 1773 59 147 0 83851 0
[pid=7147/tid=7159] ppid=7042 vsize=575664 CPUtime=0 cores=0-7
/proc/7147/task/7159/stat : 7159 (python) S 7042 7042 6630 0 -1 4202560 87 0 0 0 0 0 0 0 19 0 2 0 548520047 589479936 7608 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 514 18446604448992301248 0 0 -1 1 0 0 0
[pid=7155] ppid=7042 vsize=585920 CPUtime=0.67 cores=0-7
/proc/7155/stat : 7155 (python) S 7042 7042 6630 0 -1 4202560 1217 0 0 0 35 32 0 0 18 0 2 0 548516662 599982080 7524 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/7155/statm: 146480 7524 406 364 0 82329 0
[pid=7157] ppid=7155 vsize=268424 CPUtime=291.87 cores=0-7
/proc/7157/stat : 7157 (hybridGM3) T 7155 7157 7157 0 -1 4202496 6204 0 0 0 29186 1 0 0 25 0 1 0 548516663 274866176 5921 33554432000 134512640 135204385 4289717584 18446744073709551615 134538284 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/7157/statm: 67106 5921 63 169 0 66936 0
[pid=7155/tid=7173] ppid=7042 vsize=585920 CPUtime=0 cores=0-7
/proc/7155/task/7173/stat : 7173 (python) S 7042 7042 6630 0 -1 4202560 92 0 0 0 0 0 0 0 18 0 2 0 548536308 599982080 7524 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 514 18446604453000608640 0 0 -1 1 0 0 0
[pid=7160] ppid=7042 vsize=596176 CPUtime=0.72 cores=0-7
/proc/7160/stat : 7160 (python) S 7042 7042 6630 0 -1 4202560 1195 0 0 0 40 32 0 0 18 0 2 0 548520056 610484224 7572 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/7160/statm: 149044 7572 400 364 0 84890 0
[pid=7162] ppid=7160 vsize=268428 CPUtime=291.47 cores=0-7
/proc/7162/stat : 7162 (hybridGM3) T 7160 7162 7162 0 -1 4202496 6200 0 0 0 29146 1 0 0 25 0 1 0 548520056 274870272 5920 33554432000 134512640 135204385 4290169008 18446744073709551615 134545449 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/7162/statm: 67107 5920 63 169 0 66937 0
[pid=7160/tid=7176] ppid=7042 vsize=596176 CPUtime=0 cores=0-7
/proc/7160/task/7176/stat : 7176 (python) S 7042 7042 6630 0 -1 4202560 87 0 0 0 0 0 0 0 18 0 2 0 548543312 610484224 7572 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 514 18446604453000605952 0 0 -1 0 0 0 0
[pid=7164] ppid=7042 vsize=606432 CPUtime=0.93 cores=0-7
/proc/7164/stat : 7164 (python) S 7042 7042 6630 0 -1 4202560 997 0 0 0 49 44 0 0 18 0 1 0 548523054 620986368 7599 33554432000 4194304 5683644 140734935248912 18446744073709551615 244446965318 0 0 16781312 514 718412101495783682 0 0 17 2 0 0 0
/proc/7164/statm: 151608 7599 373 364 0 87451 0
[pid=7166] ppid=7164 vsize=461508 CPUtime=408.53 cores=0-7
/proc/7166/stat : 7166 (TNM) R 7164 7166 7166 0 -1 4202496 2480 0 0 0 40853 0 0 0 25 0 1 0 548523054 472584192 2195 33554432000 134512640 135008748 4294850704 18446744073709551615 134527773 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/7166/statm: 115377 2195 46 122 0 114277 0
[pid=7169] ppid=7042 vsize=616688 CPUtime=0.23 cores=0-7
/proc/7169/stat : 7169 (python) S 7042 7042 6630 0 -1 4202560 1192 0 0 0 12 11 0 0 19 0 2 0 548533247 631488512 7607 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/7169/statm: 154172 7607 400 364 0 90012 0
[pid=7171] ppid=7169 vsize=335996 CPUtime=97.12 cores=0-7
/proc/7171/stat : 7171 (iPAWS) T 7169 7171 7171 0 -1 4202496 2053 0 0 0 9712 0 0 0 25 0 1 0 548533248 344059904 1774 33554432000 134512640 135114423 4288050736 18446744073709551615 134516669 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/7171/statm: 83999 1774 59 147 0 83851 0
[pid=7169/tid=7177] ppid=7042 vsize=616688 CPUtime=0 cores=0-7
/proc/7169/task/7177/stat : 7177 (python) S 7042 7042 6630 0 -1 4202560 90 0 0 0 0 0 0 0 19 0 2 0 548543870 631488512 7607 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 514 18446604453220684096 0 0 -1 4 0 0 0
[pid=7178] ppid=7042 vsize=626944 CPUtime=0.23 cores=0-7
/proc/7178/stat : 7178 (python) S 7042 7042 6630 0 -1 4202560 1213 0 0 0 12 11 0 0 19 0 2 0 548543879 641990656 7595 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/7178/statm: 156736 7595 401 364 0 92573 0
[pid=7180] ppid=7178 vsize=1020308 CPUtime=100.83 cores=0-7
/proc/7180/stat : 7180 (gnovelty+2) T 7178 7180 7180 0 -1 4202496 1176 0 0 0 10083 0 0 0 25 0 1 0 548543881 1044795392 892 33554432000 134512640 135663704 4287767952 18446744073709551615 134545183 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/7180/statm: 255077 892 113 282 0 254793 0
[pid=7178/tid=7190] ppid=7042 vsize=626944 CPUtime=0 cores=0-7
/proc/7178/task/7190/stat : 7190 (python) S 7042 7042 6630 0 -1 4202560 89 0 0 0 0 0 0 0 19 0 2 0 548554508 641990656 7595 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 514 18446604448056947072 0 0 -1 6 0 0 0
[pid=7183] ppid=7042 vsize=637200 CPUtime=0.37 cores=0-7
/proc/7183/stat : 7183 (python) S 7042 7042 6630 0 -1 4202560 1057 0 0 0 19 18 0 0 18 0 1 0 548549454 652492800 7631 33554432000 4194304 5683644 140734935248912 18446744073709551615 244446965318 0 0 16781312 514 718412101495783681 0 0 17 1 0 0 0
/proc/7183/statm: 159300 7631 378 364 0 95134 0
[pid=7185] ppid=7183 vsize=268420 CPUtime=150.36 cores=0-7
/proc/7185/stat : 7185 (hybridGM3) R 7183 7185 7185 0 -1 4202496 6210 0 0 0 15036 0 0 0 25 0 1 0 548549455 274862080 5919 33554432000 134512640 135204385 4292960112 18446744073709551615 134538584 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/7185/statm: 67105 5919 63 169 0 66935 0
[pid=7187] ppid=7042 vsize=647456 CPUtime=0.28 cores=0-7
/proc/7187/stat : 7187 (python) S 7042 7042 6630 0 -1 4202560 1253 0 0 0 17 11 0 0 19 0 2 0 548554053 662994944 7640 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/7187/statm: 161864 7640 407 364 0 97695 0
[pid=7189] ppid=7187 vsize=253036 CPUtime=98.49 cores=0-7
/proc/7189/stat : 7189 (FH) T 7187 7189 7189 0 -1 4202496 2521 0 0 0 9849 0 0 0 25 0 1 0 548554054 259108864 2234 33554432000 134512640 134936988 4294453936 18446744073709551615 134554229 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/7189/statm: 63259 2234 41 104 0 62763 0
[pid=7187/tid=7200] ppid=7042 vsize=647456 CPUtime=0 cores=0-7
/proc/7187/task/7200/stat : 7200 (python) S 7042 7042 6630 0 -1 4202560 91 0 0 0 0 0 0 0 19 0 2 0 548567089 662994944 7640 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 514 18446604453220686784 0 0 -1 1 0 0 0
[pid=7191] ppid=7042 vsize=657712 CPUtime=0.27 cores=0-7
/proc/7191/stat : 7191 (python) S 7042 7042 6630 0 -1 4202560 1064 0 0 0 15 12 0 0 18 0 1 0 548554520 673497088 7738 33554432000 4194304 5683644 140734935248912 18446744073709551615 244446965318 0 0 16781312 514 718412101495783683 0 0 17 2 0 0 0
/proc/7191/statm: 164428 7738 379 364 0 100256 0
[pid=7193] ppid=7191 vsize=268428 CPUtime=110 cores=0-7
/proc/7193/stat : 7193 (hybridGM3) R 7191 7193 7193 0 -1 4202496 6212 0 0 0 10999 1 0 0 25 0 1 0 548554522 274870272 5920 33554432000 134512640 135204385 4290623088 18446744073709551615 134545302 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/7193/statm: 67107 5920 63 169 0 66937 0
[pid=7197] ppid=7042 vsize=667968 CPUtime=0.03 cores=0-7
/proc/7197/stat : 7197 (python) S 7042 7042 6630 0 -1 4202560 1073 0 0 0 1 2 0 0 18 0 1 0 548565453 683999232 7600 33554432000 4194304 5683644 140734935248912 18446744073709551615 244446965318 0 0 16781312 514 718412101495783682 0 0 17 1 0 0 0
/proc/7197/statm: 166992 7600 379 364 0 102817 0
[pid=7199] ppid=7197 vsize=423316 CPUtime=8.48 cores=0-7
/proc/7199/stat : 7199 (NCVWr) R 7197 7199 7199 0 -1 4202496 2556 0 0 0 848 0 0 0 25 0 1 0 548565454 433475584 2267 33554432000 134512640 135113558 4293899328 18446744073709551615 134531745 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/7199/statm: 105829 2267 58 147 0 104704 0
[pid=7201] ppid=7042 vsize=678224 CPUtime=0 cores=0-7
/proc/7201/stat : 7201 (python) S 7042 7042 6630 0 -1 4202560 947 0 0 0 0 0 0 0 18 0 1 0 548567101 694501376 7643 33554432000 4194304 5683644 140734935248912 18446744073709551615 244446965318 0 0 16781312 514 4129472108617007103 0 0 17 1 0 0 0
/proc/7201/statm: 169556 7643 350 364 0 105378 0
[pid=7203] ppid=7201 vsize=268428 CPUtime=0.11 cores=0-7
/proc/7203/stat : 7203 (hybridGM3) R 7201 7203 7203 0 -1 4202496 6214 0 0 0 10 1 0 0 21 0 1 0 548567101 274870272 5920 33554432000 134512640 135204385 4288674032 18446744073709551615 134545440 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/7203/statm: 67107 5920 63 169 0 66937 0
[pid=7042/tid=7049] ppid=7040 vsize=688468 CPUtime=0 cores=0-7
/proc/7042/task/7049/stat : 7049 (python) S 7040 7042 6630 0 -1 4202560 27 937 0 0 0 0 0 0 20 0 34 0 548449031 704991232 9233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 4 0 0 0
[pid=7042/tid=7052] ppid=7040 vsize=688468 CPUtime=0 cores=0-7
/proc/7042/task/7052/stat : 7052 (python) S 7040 7042 6630 0 -1 4202560 24 937 0 0 0 0 0 0 21 0 34 0 548449035 704991232 9233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 0 0 0 0
[pid=7042/tid=7055] ppid=7040 vsize=688468 CPUtime=0 cores=0-7
/proc/7042/task/7055/stat : 7055 (python) S 7040 7042 6630 0 -1 4202560 28 937 0 0 0 0 0 0 18 0 34 0 548449038 704991232 9233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 5 0 0 0
[pid=7042/tid=7058] ppid=7040 vsize=688468 CPUtime=0 cores=0-7
/proc/7042/task/7058/stat : 7058 (python) S 7040 7042 6630 0 -1 4202560 98 937 0 0 0 0 0 0 18 0 34 0 548449042 704991232 9233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 0 0 0 0
[pid=7042/tid=7061] ppid=7040 vsize=688468 CPUtime=0 cores=0-7
/proc/7042/task/7061/stat : 7061 (python) S 7040 7042 6630 0 -1 4202560 35 937 0 0 0 0 0 0 18 0 34 0 548449046 704991232 9233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 7 0 0 0
[pid=7042/tid=7064] ppid=7040 vsize=688468 CPUtime=0 cores=0-7
/proc/7042/task/7064/stat : 7064 (python) S 7040 7042 6630 0 -1 4202560 31 937 0 0 0 0 0 0 21 0 34 0 548449050 704991232 9233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 2 0 0 0
[pid=7042/tid=7067] ppid=7040 vsize=688468 CPUtime=0 cores=0-7
/proc/7042/task/7067/stat : 7067 (python) S 7040 7042 6630 0 -1 4202560 36 937 0 0 0 0 0 0 18 0 34 0 548449054 704991232 9233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 2 0 0 0
[pid=7042/tid=7070] ppid=7040 vsize=688468 CPUtime=0 cores=0-7
/proc/7042/task/7070/stat : 7070 (python) S 7040 7042 6630 0 -1 4202560 38 937 0 0 0 0 0 0 18 0 34 0 548449060 704991232 9233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 2 0 0 0
[pid=7042/tid=7079] ppid=7040 vsize=688468 CPUtime=0 cores=0-7
/proc/7042/task/7079/stat : 7079 (python) S 7040 7042 6630 0 -1 4202560 41 937 0 0 0 0 0 0 18 0 34 0 548459057 704991232 9233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 6 0 0 0
[pid=7042/tid=7082] ppid=7040 vsize=688468 CPUtime=0 cores=0-7
/proc/7042/task/7082/stat : 7082 (python) S 7040 7042 6630 0 -1 4202560 38 937 0 0 0 0 0 0 18 0 34 0 548459062 704991232 9233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 1 0 0 0
[pid=7042/tid=7086] ppid=7040 vsize=688468 CPUtime=0 cores=0-7
/proc/7042/task/7086/stat : 7086 (python) S 7040 7042 6630 0 -1 4202560 39 937 0 0 0 0 0 0 18 0 34 0 548459067 704991232 9233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 2 0 0 0
[pid=7042/tid=7090] ppid=7040 vsize=688468 CPUtime=0 cores=0-7
/proc/7042/task/7090/stat : 7090 (python) S 7040 7042 6630 0 -1 4202560 34 937 0 0 0 0 0 0 25 0 34 0 548459073 704991232 9233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 2 0 0 0
[pid=7042/tid=7094] ppid=7040 vsize=688468 CPUtime=0 cores=0-7
/proc/7042/task/7094/stat : 7094 (python) S 7040 7042 6630 0 -1 4202560 40 937 0 0 0 0 0 0 18 0 34 0 548459078 704991232 9233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 2 0 0 0
[pid=7042/tid=7096] ppid=7040 vsize=688468 CPUtime=0 cores=0-7
/proc/7042/task/7096/stat : 7096 (python) S 7040 7042 6630 0 -1 4202560 53 937 0 0 0 0 0 0 18 0 34 0 548459084 704991232 9233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 2 0 0 0
[pid=7042/tid=7102] ppid=7040 vsize=688468 CPUtime=0 cores=0-7
/proc/7042/task/7102/stat : 7102 (python) S 7040 7042 6630 0 -1 4202560 48 937 0 0 0 0 0 0 18 0 34 0 548463874 704991232 9233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 5 0 0 0
[pid=7042/tid=7106] ppid=7040 vsize=688468 CPUtime=0 cores=0-7
/proc/7042/task/7106/stat : 7106 (python) S 7040 7042 6630 0 -1 4202560 51 937 0 0 0 0 0 0 18 0 34 0 548464510 704991232 9233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 2 0 0 0
[pid=7042/tid=7111] ppid=7040 vsize=688468 CPUtime=0 cores=0-7
/proc/7042/task/7111/stat : 7111 (python) S 7040 7042 6630 0 -1 4202560 36 937 0 0 0 0 0 0 20 0 34 0 548469082 704991232 9233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 2 0 0 0
[pid=7042/tid=7115] ppid=7040 vsize=688468 CPUtime=0 cores=0-7
/proc/7042/task/7115/stat : 7115 (python) S 7040 7042 6630 0 -1 4202560 33 937 0 0 0 0 0 0 20 0 34 0 548469110 704991232 9233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 2 0 0 0
[pid=7042/tid=7126] ppid=7040 vsize=688468 CPUtime=0 cores=0-7
/proc/7042/task/7126/stat : 7126 (python) S 7040 7042 6630 0 -1 4202560 38 937 0 0 0 0 0 0 19 0 34 0 548482133 704991232 9233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 2 0 0 0
[pid=7042/tid=7134] ppid=7040 vsize=688468 CPUtime=0 cores=0-7
/proc/7042/task/7134/stat : 7134 (python) S 7040 7042 6630 0 -1 4202560 42 937 0 0 0 0 0 0 18 0 34 0 548490363 704991232 9233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 2 0 0 0
[pid=7042/tid=7137] ppid=7040 vsize=688468 CPUtime=0 cores=0-7
/proc/7042/task/7137/stat : 7137 (python) S 7040 7042 6630 0 -1 4202560 36 937 0 0 0 0 0 0 19 0 34 0 548491173 704991232 9233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 2 0 0 0
[pid=7042/tid=7142] ppid=7040 vsize=688468 CPUtime=0 cores=0-7
/proc/7042/task/7142/stat : 7142 (python) S 7040 7042 6630 0 -1 4202560 38 937 0 0 0 0 0 0 20 0 34 0 548498467 704991232 9233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 2 0 0 0
[pid=7042/tid=7148] ppid=7040 vsize=688468 CPUtime=0 cores=0-7
/proc/7042/task/7148/stat : 7148 (python) S 7040 7042 6630 0 -1 4202560 34 937 0 0 0 0 0 0 19 0 34 0 548505219 704991232 9233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 2 0 0 0
[pid=7042/tid=7156] ppid=7040 vsize=688468 CPUtime=0 cores=0-7
/proc/7042/task/7156/stat : 7156 (python) S 7040 7042 6630 0 -1 4202560 43 937 0 0 0 0 0 0 18 0 34 0 548516662 704991232 9233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 2 0 0 0
[pid=7042/tid=7161] ppid=7040 vsize=688468 CPUtime=0 cores=0-7
/proc/7042/task/7161/stat : 7161 (python) S 7040 7042 6630 0 -1 4202560 41 937 0 0 0 0 0 0 18 0 34 0 548520056 704991232 9233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 2 0 0 0
[pid=7042/tid=7165] ppid=7040 vsize=688468 CPUtime=0 cores=0-7
/proc/7042/task/7165/stat : 7165 (python) S 7040 7042 6630 0 -1 4202560 33 937 0 0 0 0 0 0 20 0 34 0 548523054 704991232 9233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 2 0 0 0
[pid=7042/tid=7170] ppid=7040 vsize=688468 CPUtime=0 cores=0-7
/proc/7042/task/7170/stat : 7170 (python) S 7040 7042 6630 0 -1 4202560 32 937 0 0 0 0 0 0 20 0 34 0 548533248 704991232 9233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 2 0 0 0
[pid=7042/tid=7179] ppid=7040 vsize=688468 CPUtime=0 cores=0-7
/proc/7042/task/7179/stat : 7179 (python) S 7040 7042 6630 0 -1 4202560 35 937 0 0 0 0 0 0 20 0 34 0 548543880 704991232 9233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 2 0 0 0
[pid=7042/tid=7184] ppid=7040 vsize=688468 CPUtime=0 cores=0-7
/proc/7042/task/7184/stat : 7184 (python) S 7040 7042 6630 0 -1 4202560 33 937 0 0 0 0 0 0 19 0 34 0 548549454 704991232 9233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 2 0 0 0
[pid=7042/tid=7188] ppid=7040 vsize=688468 CPUtime=0 cores=0-7
/proc/7042/task/7188/stat : 7188 (python) S 7040 7042 6630 0 -1 4202560 35 937 0 0 0 0 0 0 19 0 34 0 548554054 704991232 9233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 2 0 0 0
[pid=7042/tid=7192] ppid=7040 vsize=688468 CPUtime=0 cores=0-7
/proc/7042/task/7192/stat : 7192 (python) S 7040 7042 6630 0 -1 4202560 32 937 0 0 0 0 0 0 20 0 34 0 548554520 704991232 9233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 2 0 0 0
[pid=7042/tid=7198] ppid=7040 vsize=688468 CPUtime=0 cores=0-7
/proc/7042/task/7198/stat : 7198 (python) S 7040 7042 6630 0 -1 4202560 33 937 0 0 0 0 0 0 19 0 34 0 548565453 704991232 9233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 2 0 0 0
[pid=7042/tid=7202] ppid=7040 vsize=688468 CPUtime=0 cores=0-7
/proc/7042/task/7202/stat : 7202 (python) S 7040 7042 6630 0 -1 4202560 31 937 0 0 0 0 0 0 19 0 34 0 548567101 704991232 9233 33554432000 4194304 5683644 140734935248912 18446744073709551615 244458768817 0 0 16781312 2 18446604453855406976 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 8313.35
Current children cumulated vsize (KiB) 32576580

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 7042 and gives
#  childrusage.ru_utime.tv_sec=6
#  childrusage.ru_utime.tv_usec=166062
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=263959
# CPU time returned by wait4() is 6.43002
# while last known CPU time is 8313.35
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1183.71
CPU time (s): 8313.35
CPU user time (s): 8303.92
CPU system time (s): 9.43
CPU usage (%): 702.316
Max. virtual memory (cumulated for all children) (KiB): 32576580

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 6.16606
system time used= 0.263959
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 67400
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3188
involuntary context switches= 127

runsolver used 12.1212 second user time and 26.227 second system time

The end

Launcher Data 

Begin job on node105 at 2011-05-02 22:34:43
IDJOB=3321859
IDBENCH=82637
IDSOLVER=1712
FILE ID=node105/3321859-1304368483
RUNJOBID= node105-1304366469-6644
PBS_JOBID= 13181765
Free space on /tmp= 73652 MiB

SOLVER NAME= borg-sat 11.03.02-random
BENCH NAME= SAT11/random/large/unif-k5-r20-v2000-c40000-S1993071608-084-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3321859-1304368483/watcher-3321859-1304368483 -o /tmp/evaluation-result-3321859-1304368483/solver-3321859-1304368483 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3321859-1304368483.cnf -seed 206775748 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 6a416a8eca39558555cbe083cf0c9cff
RANDOM SEED=206775748

node105.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5410.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      17410944 kB
Buffers:          9844 kB
Cached:         215044 kB
SwapCached:      26824 kB
Active:        3793964 kB
Inactive:     11548520 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      17410944 kB
SwapTotal:    67111528 kB
SwapFree:     62859076 kB
Dirty:           32704 kB
Writeback:           0 kB
AnonPages:    15093188 kB
Mapped:          16952 kB
Slab:            81552 kB
PageTables:      50872 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 31164960 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73620 MiB
End job on node105 at 2011-05-02 22:54:29