Trace number 3321823

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-random? (MO) 10473 1481.81

General information on the benchmark

Namerandom/large/
unif-k7-r85-v400-c34000-S927774663-037-UNKNOWN.cnf
MD5SUM950830e22ef92fb7f22c5b3c987a21f5
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables400
Number of clauses34000
Sum of the clauses size238000
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 534000

Solver Data 

0.31/0.46	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.31/0.47	c INFO: solving HOME/instance-3321823-1304367982.cnf
0.31/0.47	c INFO: computing task features
0.51/0.70	c INFO: computed variable-clause graph statistics
0.60/0.70	c INFO: computed clause balance statistics
1.70/1.88	c INFO: computed variable balance statistics
2.20/2.35	c INFO: computed small-clause counts
2.20/2.35	c INFO: computed Horn-clause counts
2.20/2.39	c INFO: computed variable graph statistics
2.20/2.39	c INFO: collected features for HOME/instance-3321823-1304367982.cnf in 1.92 s
2.31/2.43	c INFO: running gnovelty+2@0 for 97 with 39998 remaining (b = 0.38)
2.31/2.47	c INFO: running gnovelty+2@0 for 97 with 39998 remaining (b = 0.13)
2.40/2.51	c INFO: running adaptg2wsat2009++@0 for 97 with 39997 remaining (b = 0.08)
2.40/2.54	c INFO: running adaptg2wsat2009++@0 for 97 with 39997 remaining (b = 0.07)
2.40/2.58	c INFO: running adaptg2wsat2009++@0 for 97 with 39997 remaining (b = 0.05)
2.50/2.62	c INFO: running gnovelty+2@0 for 97 with 39997 remaining (b = 0.04)
2.50/2.66	c INFO: running FH@0 for 97 with 39997 remaining (b = 0.03)
2.60/2.70	c INFO: running adaptg2wsat2009++@0 for 97 with 39997 remaining (b = 0.03)
717.59/102.75	c INFO: running adaptg2wsat2009++@0 for 97 with 39897 remaining (b = 0.02)
718.28/102.80	c INFO: running FH@0 for 97 with 39897 remaining (b = 0.02)
718.28/102.88	c INFO: running hybridGM3@0 for 194 with 39897 remaining (b = 0.04)
719.30/103.01	c INFO: running gnovelty+2@0 for 97 with 39897 remaining (b = 0.02)
845.73/121.08	c INFO: running hybridGM3@0 for 194 with 39879 remaining (b = 0.03)
886.19/126.86	c INFO: running adaptg2wsat2009++@0 for 97 with 39873 remaining (b = 0.01)
886.91/126.92	c INFO: running hybridGM3@0 for 194 with 39873 remaining (b = 0.03)
887.51/127.08	c INFO: running hybridGM3@0 for 194 with 39873 remaining (b = 0.02)
1447.74/203.53	c INFO: running TNM@0 for 1552 with 39796 remaining (b = 0.18)
1479.72/208.13	c INFO: running FH@0 for 97 with 39792 remaining (b = 0.01)
1480.41/208.20	c INFO: running adaptg2wsat2009++@0 for 97 with 39792 remaining (b = 0.01)
1864.68/263.30	c INFO: running adaptg2wsat2009++@0 for 97 with 39737 remaining (b = 0.01)
2173.73/305.52	c INFO: running adaptg2wsat2009++@100 for 97 with 39695 remaining (b = 0.01)
2258.78/317.71	c INFO: running hybridGM3@99 for 97 with 39682 remaining (b = 0.01)
2300.60/323.79	c INFO: running hybridGM3@100 for 97 with 39676 remaining (b = 0.01)
2425.35/341.61	c INFO: running hybridGM3@99 for 97 with 39658 remaining (b = 0.01)
2427.43/341.92	c INFO: running hybridGM3@97 for 97 with 39658 remaining (b = 0.01)
2529.90/356.63	c INFO: running hybridGM3@0 for 194 with 39643 remaining (b = 0.02)
2641.40/372.64	c INFO: running hybridGM3@0 for 194 with 39627 remaining (b = 0.02)
2922.95/413.01	c INFO: running hybridGM3@296 for 97 with 39587 remaining (b = 0.01)
2928.52/413.87	c INFO: running TNM@0 for 1552 with 39586 remaining (b = 0.14)
3131.32/442.94	c INFO: running hybridGM3@294 for 97 with 39557 remaining (b = 0.01)
3216.35/455.17	c INFO: running adaptg2wsat2009++@0 for 97 with 39545 remaining (b = 0.01)
3258.84/461.28	c INFO: running adaptg2wsat2009++@99 for 97 with 39539 remaining (b = 0.01)
3837.29/544.26	c INFO: running iPAWS@0 for 97 with 39456 remaining (b = 0.01)
3973.09/562.45	c INFO: running hybridGM3@295 for 97 with 39438 remaining (b = 0.01)
4072.17/576.64	c INFO: running gnovelty+2@0 for 97 with 39423 remaining (b = 0.01)
4076.77/577.36	c INFO: running hybridGM3@391 for 97 with 39423 remaining (b = 0.01)
4106.43/581.59	c INFO: running hybridGM3@396 for 97 with 39418 remaining (b = 0.01)
4147.01/587.35	c INFO: running hybridGM3@96 for 97 with 39413 remaining (b = 0.01)
4588.43/650.61	c INFO: running hybridGM3@95 for 97 with 39349 remaining (b = 0.01)
4687.41/664.82	c INFO: running hybridGM3@393 for 97 with 39335 remaining (b = 0.01)
4788.23/678.91	c INFO: running FH@0 for 291 with 39321 remaining (b = 0.02)
4817.61/683.18	c INFO: running adaptg2wsat2009++@296 for 291 with 39317 remaining (b = 0.02)
4968.04/704.75	c INFO: running FH@99 for 291 with 39295 remaining (b = 0.02)
4995.90/708.73	c INFO: running hybridGM3@193 for 97 with 39291 remaining (b = 0.01)
5393.87/765.97	c INFO: running hybridGM3@189 for 97 with 39234 remaining (b = 0.01)
5736.98/814.26	c INFO: running TNM@0 for 1552 with 39186 remaining (b = 0.11)
5966.16/846.14	c INFO: running iPAWS@0 for 97 with 39154 remaining (b = 0.01)
6239.32/885.36	c INFO: running adaptg2wsat2009++@98 for 97 with 39115 remaining (b = 0.01)
6707.47/952.51	c INFO: running gnovelty+2@0 for 97 with 39048 remaining (b = 0.01)
7041.58/1000.55	c INFO: running hybridGM3@0 for 194 with 38999 remaining (b = 0.01)
7142.49/1015.07	c INFO: running adaptg2wsat2009++@390 for 194 with 38985 remaining (b = 0.01)
7155.03/1016.81	c INFO: running FH@197 for 194 with 38983 remaining (b = 0.01)
7377.79/1048.88	c INFO: running FH@0 for 97 with 38951 remaining (b = 0.01)
7408.29/1052.93	c INFO: running hybridGM3@0 for 194 with 38947 remaining (b = 0.01)
8098.76/1149.33	c INFO: running NCVWr@0 for 485 with 38851 remaining (b = 0.03)
8448.55/1197.25	c INFO: running hybridGM3@0 for 194 with 38803 remaining (b = 0.01)
8540.49/1210.41	c INFO: running FH@296 for 97 with 38790 remaining (b = 0.01)
8807.14/1248.76	c INFO: running adaptg2wsat2009++@681 for 97 with 38751 remaining (b = 0.01)
8977.52/1272.76	c INFO: running hybridGM3@97 for 97 with 38727 remaining (b = 0.01)
9217.97/1305.76	c INFO: running iPAWS@0 for 97 with 38694 remaining (b = 0.01)
9637.70/1362.16	c INFO: running adaptg2wsat2009++@876 for 97 with 38638 remaining (b = 0.01)
9766.78/1380.14	c INFO: running hybridGM3@192 for 97 with 38620 remaining (b = 0.01)
10114.69/1430.20	c INFO: running FH@99 for 291 with 38570 remaining (b = 0.02)
10305.36/1457.52	c INFO: running adaptg2wsat2009++@98 for 97 with 38543 remaining (b = 0.01)
10316.41/1459.16	c INFO: running hybridGM3@191 for 97 with 38541 remaining (b = 0.01)
10472.18/1481.51	c INFO: running hybridGM3@0 for 194 with 38519 remaining (b = 0.01)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3321823-1304367982/watcher-3321823-1304367982 -o /tmp/evaluation-result-3321823-1304367982/solver-3321823-1304367982 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3321823-1304367982.cnf -seed 1833446829 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.40 8.35 6.52 2/306 18125
/proc/meminfo: memFree=17993476/32951124 swapFree=67102128/67111528
[pid=18125] ppid=18123 vsize=15904 CPUtime=0 cores=0-7
/proc/18125/stat : 18125 (python) S 18123 18125 17541 0 -1 4202496 944 0 0 0 0 0 0 0 25 0 1 0 627028853 16285696 820 33554432000 4194304 5683644 140737095724672 18446744073709551615 216735176704 0 2147483391 16781312 2 18446744071699199166 0 0 17 4 0 0 0
/proc/18125/statm: 3976 820 357 364 0 463 0

[startup+0.0939019 s]
/proc/loadavg: 8.40 8.35 6.52 2/306 18125
/proc/meminfo: memFree=17993476/32951124 swapFree=67102128/67111528
[pid=18125] ppid=18123 vsize=101576 CPUtime=0.06 cores=0-7
/proc/18125/stat : 18125 (python) S 18123 18125 17541 0 -1 4202496 2897 0 0 0 5 1 0 0 17 0 1 0 627028853 104013824 2593 33554432000 4194304 5683644 140737095724672 18446744073709551615 216735176704 0 2147483391 16781312 2 18446744071699199166 0 0 17 4 0 0 0
/proc/18125/statm: 25394 2593 837 364 0 1694 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 101576

[startup+0.100906 s]
/proc/loadavg: 8.40 8.35 6.52 2/306 18125
/proc/meminfo: memFree=17993476/32951124 swapFree=67102128/67111528
[pid=18125] ppid=18123 vsize=104224 CPUtime=0.06 cores=0-7
/proc/18125/stat : 18125 (python) R 18123 18125 17541 0 -1 4202496 3066 0 0 0 5 1 0 0 17 0 1 0 627028853 106725376 2747 33554432000 4194304 5683644 140737095724672 18446744073709551615 216735174501 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/18125/statm: 26056 2747 847 364 0 1804 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 104224

[startup+0.300872 s]
/proc/loadavg: 8.40 8.35 6.52 2/306 18125
/proc/meminfo: memFree=17993476/32951124 swapFree=67102128/67111528
[pid=18125] ppid=18123 vsize=170752 CPUtime=0.24 cores=0-7
/proc/18125/stat : 18125 (python) S 18123 18125 17541 0 -1 4202496 6742 933 1 0 18 6 0 0 17 0 1 0 627028853 174850048 4809 33554432000 4194304 5683644 140737095724672 18446744073709551615 216735176704 0 2147483391 16781312 2 18446744071699199166 0 0 17 4 0 0 0
/proc/18125/statm: 42688 4809 1336 364 0 3453 0
Current children cumulated CPU time (s) 0.24
Current children cumulated vsize (KiB) 170752

[startup+0.700791 s]
/proc/loadavg: 8.40 8.35 6.52 2/306 18125
/proc/meminfo: memFree=17993476/32951124 swapFree=67102128/67111528
[pid=18125] ppid=18123 vsize=289504 CPUtime=0.6 cores=0-7
/proc/18125/stat : 18125 (python) R 18123 18125 17541 0 -1 4202496 13878 933 1 0 52 8 0 0 17 0 1 0 627028853 296452096 9753 33554432000 4194304 5683644 140737095724672 18446744073709551615 47056800431207 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/18125/statm: 72376 9753 1818 364 0 8300 0
Current children cumulated CPU time (s) 0.6
Current children cumulated vsize (KiB) 289504

[startup+1.50065 s]
/proc/loadavg: 8.40 8.35 6.52 3/307 18130
/proc/meminfo: memFree=17960732/32951124 swapFree=67102128/67111528
[pid=18125] ppid=18123 vsize=289504 CPUtime=1.4 cores=0-7
/proc/18125/stat : 18125 (python) R 18123 18125 17541 0 -1 4202496 13886 933 1 0 132 8 0 0 19 0 1 0 627028853 296452096 9761 33554432000 4194304 5683644 140737095724672 18446744073709551615 47056768156640 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/18125/statm: 72376 9761 1826 364 0 8300 0
Current children cumulated CPU time (s) 1.4
Current children cumulated vsize (KiB) 289504

[startup+3.10033 s]
/proc/loadavg: 7.89 8.25 6.50 3/307 18130
/proc/meminfo: memFree=17960360/32951124 swapFree=67102128/67111528
[pid=18125] ppid=18123 vsize=429964 CPUtime=2.61 cores=0-7
/proc/18125/stat : 18125 (python) S 18123 18125 17541 0 -1 4202496 20015 933 1 0 250 11 0 0 25 0 9 0 627028853 440283136 8219 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747005387 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/18125/statm: 107491 8219 1891 364 0 43388 0
Current children cumulated CPU time (s) 2.61
Current children cumulated vsize (KiB) 429964

[startup+6.30177 s]
/proc/loadavg: 7.89 8.25 6.50 10/331 18154
/proc/meminfo: memFree=17899904/32951124 swapFree=67102128/67111528
[pid=18125] ppid=18123 vsize=429964 CPUtime=2.61 cores=0-7
/proc/18125/stat : 18125 (python) S 18123 18125 17541 0 -1 4202496 20015 933 1 0 250 11 0 0 25 0 9 0 627028853 440283136 8219 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747005387 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/18125/statm: 107491 8219 1891 364 0 43388 0
[pid=18131] ppid=18125 vsize=282248 CPUtime=0 cores=0-7
/proc/18131/stat : 18131 (python) S 18125 18125 17541 0 -1 4202560 732 0 0 0 0 0 0 0 18 0 1 0 627029096 289021952 6333 33554432000 4194304 5683644 140737095724672 18446744073709551615 216735198735 0 0 16781312 514 5858721049742377219 0 0 17 4 0 0 0
/proc/18131/statm: 70562 6333 355 364 0 6480 0
[pid=18133] ppid=18131 vsize=1020188 CPUtime=3.86 cores=0-7
/proc/18133/stat : 18133 (gnovelty+2) R 18131 18133 18133 0 -1 4202496 1139 0 0 0 386 0 0 0 25 0 1 0 627029097 1044672512 877 33554432000 134512640 135663704 4293718992 18446744073709551615 134524744 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/18133/statm: 255047 877 113 282 0 254763 0
[pid=18134] ppid=18125 vsize=293132 CPUtime=0 cores=0-7
/proc/18134/stat : 18134 (python) S 18125 18125 17541 0 -1 4202560 732 0 0 0 0 0 0 0 18 0 1 0 627029101 300167168 6370 33554432000 4194304 5683644 140737095724672 18446744073709551615 216735198790 0 0 16781312 514 7242444613548474627 0 0 17 0 0 0 0
/proc/18134/statm: 73283 6370 349 364 0 9198 0
[pid=18136] ppid=18134 vsize=1020188 CPUtime=3.81 cores=0-7
/proc/18136/stat : 18136 (gnovelty+2) R 18134 18136 18136 0 -1 4202496 1136 0 0 0 381 0 0 0 25 0 1 0 627029101 1044672512 876 33554432000 134512640 135663704 4291418720 18446744073709551615 134524498 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/18136/statm: 255047 876 113 282 0 254763 0
[pid=18137] ppid=18125 vsize=302904 CPUtime=0 cores=0-7
/proc/18137/stat : 18137 (python) S 18125 18125 17541 0 -1 4202560 735 0 0 0 0 0 0 0 18 0 1 0 627029104 310173696 6507 33554432000 4194304 5683644 140737095724672 18446744073709551615 216735198790 0 0 16781312 514 7242440765257777410 0 0 17 6 0 0 0
/proc/18137/statm: 75726 6507 347 364 0 11638 0
[pid=18139] ppid=18137 vsize=421140 CPUtime=3.78 cores=0-7
/proc/18139/stat : 18139 (adaptg2wsat2009) R 18137 18139 18139 0 -1 4202496 1805 0 0 0 378 0 0 0 25 0 1 0 627029105 431247360 1539 33554432000 134512640 135107411 4288628080 18446744073709551615 134523442 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/18139/statm: 105285 1539 49 146 0 104161 0
[pid=18140] ppid=18125 vsize=313160 CPUtime=0 cores=0-7
/proc/18140/stat : 18140 (python) S 18125 18125 17541 0 -1 4202560 751 0 0 0 0 0 0 0 18 0 1 0 627029108 320675840 6510 33554432000 4194304 5683644 140737095724672 18446744073709551615 216735198790 0 0 16781312 514 15501798665151807746 0 0 17 1 0 0 0
/proc/18140/statm: 78290 6510 347 364 0 14199 0
[pid=18142] ppid=18140 vsize=421140 CPUtime=2.01 cores=0-7
/proc/18142/stat : 18142 (adaptg2wsat2009) R 18140 18142 18142 0 -1 4202496 1803 0 0 0 201 0 0 0 25 0 1 0 627029108 431247360 1539 33554432000 134512640 135107411 4292578976 18446744073709551615 134523285 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/18142/statm: 105285 1539 49 146 0 104161 0
[pid=18143] ppid=18125 vsize=388952 CPUtime=0 cores=0-7
/proc/18143/stat : 18143 (python) S 18125 18125 17541 0 -1 4202560 723 0 0 0 0 0 0 0 18 0 1 0 627029112 398286848 6688 33554432000 4194304 5683644 140737095724672 18446744073709551615 216735198790 0 0 16781312 514 2740705909607989507 0 0 17 1 0 0 0
/proc/18143/statm: 97238 6688 348 364 0 33144 0
[pid=18145] ppid=18143 vsize=421140 CPUtime=3.66 cores=0-7
/proc/18145/stat : 18145 (adaptg2wsat2009) R 18143 18145 18145 0 -1 4202496 1794 0 0 0 366 0 0 0 25 0 1 0 627029112 431247360 1538 33554432000 134512640 135107411 4290272336 18446744073709551615 134523261 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/18145/statm: 105285 1538 49 146 0 104161 0
[pid=18146] ppid=18125 vsize=399208 CPUtime=0 cores=0-7
/proc/18146/stat : 18146 (python) S 18125 18125 17541 0 -1 4202560 732 0 0 0 0 0 0 0 18 0 1 0 627029116 408788992 6664 33554432000 4194304 5683644 140737095724672 18446744073709551615 216735198790 0 0 16781312 514 16561852668897624321 0 0 17 1 0 0 0
/proc/18146/statm: 99802 6664 348 364 0 35705 0
[pid=18148] ppid=18146 vsize=1020188 CPUtime=3.63 cores=0-7
/proc/18148/stat : 18148 (gnovelty+2) R 18146 18148 18148 0 -1 4202496 1131 0 0 0 363 0 0 0 25 0 1 0 627029117 1044672512 876 33554432000 134512640 135663704 4287653056 18446744073709551615 134524548 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/18148/statm: 255047 876 113 282 0 254763 0
[pid=18149] ppid=18125 vsize=409464 CPUtime=0 cores=0-7
/proc/18149/stat : 18149 (python) S 18125 18125 17541 0 -1 4202560 741 0 0 0 0 0 0 0 18 0 1 0 627029120 419291136 6670 33554432000 4194304 5683644 140737095724672 18446744073709551615 216735198790 0 0 16781312 514 12007513328684335362 0 0 17 4 0 0 0
/proc/18149/statm: 102366 6670 348 364 0 38266 0
[pid=18153] ppid=18149 vsize=249144 CPUtime=1.72 cores=0-7
/proc/18153/stat : 18153 (FH) R 18149 18153 18153 0 -1 4202496 1555 0 0 0 172 0 0 0 25 0 1 0 627029124 255123456 1296 33554432000 134512640 134936988 4289230192 18446744073709551615 134523657 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/18153/statm: 62286 1296 37 104 0 61790 0
[pid=18151] ppid=18125 vsize=419720 CPUtime=0 cores=0-7
/proc/18151/stat : 18151 (python) S 18125 18125 17541 0 -1 4202560 781 0 0 0 0 0 0 0 18 0 1 0 627029124 429793280 6678 33554432000 4194304 5683644 140737095724672 18446744073709551615 216735198790 0 0 16781312 514 12007524873556427012 0 0 17 4 0 0 0
/proc/18151/statm: 104930 6678 354 364 0 40827 0
[pid=18154] ppid=18151 vsize=421140 CPUtime=3.58 cores=0-7
/proc/18154/stat : 18154 (adaptg2wsat2009) R 18151 18154 18154 0 -1 4202496 1802 0 0 0 358 0 0 0 25 0 1 0 627029124 431247360 1538 33554432000 134512640 135107411 4291393152 18446744073709551615 134523312 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/18154/statm: 105285 1538 49 146 0 104161 0
[pid=18125/tid=18132] ppid=18123 vsize=429964 CPUtime=0 cores=0-7
/proc/18125/task/18132/stat : 18132 (python) S 18123 18125 17541 0 -1 4202560 26 933 0 0 0 0 0 0 22 0 9 0 627029097 440283136 8219 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 0 0 0 0
[pid=18125/tid=18135] ppid=18123 vsize=429964 CPUtime=0 cores=0-7
/proc/18125/task/18135/stat : 18135 (python) S 18123 18125 17541 0 -1 4202560 24 933 0 0 0 0 0 0 24 0 9 0 627029101 440283136 8219 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 4 0 0 0
[pid=18125/tid=18138] ppid=18123 vsize=429964 CPUtime=0 cores=0-7
/proc/18125/task/18138/stat : 18138 (python) S 18123 18125 17541 0 -1 4202560 23 933 0 0 0 0 0 0 20 0 9 0 627029105 440283136 8219 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 6 0 0 0
[pid=18125/tid=18141] ppid=18123 vsize=429964 CPUtime=0 cores=0-7
/proc/18125/task/18141/stat : 18141 (python) S 18123 18125 17541 0 -1 4202560 19 933 0 0 0 0 0 0 21 0 9 0 627029108 440283136 8219 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 0 0 0 0
[pid=18125/tid=18144] ppid=18123 vsize=429964 CPUtime=0 cores=0-7
/proc/18125/task/18144/stat : 18144 (python) S 18123 18125 17541 0 -1 4202560 96 933 0 0 0 0 0 0 22 0 9 0 627029112 440283136 8219 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 7 0 0 0
[pid=18125/tid=18147] ppid=18123 vsize=429964 CPUtime=0 cores=0-7
/proc/18125/task/18147/stat : 18147 (python) S 18123 18125 17541 0 -1 4202560 32 933 0 0 0 0 0 0 23 0 9 0 627029116 440283136 8219 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 1 0 0 0
[pid=18125/tid=18150] ppid=18123 vsize=429964 CPUtime=0 cores=0-7
/proc/18125/task/18150/stat : 18150 (python) S 18123 18125 17541 0 -1 4202560 52 933 0 0 0 0 0 0 25 0 9 0 627029120 440283136 8219 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 4 0 0 0
[pid=18125/tid=18152] ppid=18123 vsize=429964 CPUtime=0 cores=0-7
/proc/18125/task/18152/stat : 18152 (python) S 18123 18125 17541 0 -1 4202560 71 933 0 0 0 0 0 0 25 0 9 0 627029124 440283136 8219 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 28.66
Current children cumulated vsize (KiB) 8233020

[startup+12.7016 s]
/proc/loadavg: 8.06 8.27 6.52 10/331 18154
/proc/meminfo: memFree=17895328/32951124 swapFree=67102128/67111528
[pid=18125] ppid=18123 vsize=429964 CPUtime=2.61 cores=0-7
/proc/18125/stat : 18125 (python) S 18123 18125 17541 0 -1 4202496 20015 933 1 0 250 11 0 0 25 0 9 0 627028853 440283136 8219 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747005387 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/18125/statm: 107491 8219 1891 364 0 43388 0
[pid=18131] ppid=18125 vsize=282248 CPUtime=0.02 cores=0-7
/proc/18131/stat : 18131 (python) S 18125 18125 17541 0 -1 4202560 902 0 0 0 1 1 0 0 18 0 1 0 627029096 289021952 6356 33554432000 4194304 5683644 140737095724672 18446744073709551615 216735198735 0 0 16781312 514 4531782309413617923 0 0 17 4 0 0 0
/proc/18131/statm: 70562 6356 378 364 0 6480 0
[pid=18133] ppid=18131 vsize=1020188 CPUtime=10.26 cores=0-7
/proc/18133/stat : 18133 (gnovelty+2) R 18131 18133 18133 0 -1 4202496 1139 0 0 0 1026 0 0 0 25 0 1 0 627029097 1044672512 877 33554432000 134512640 135663704 4293718992 18446744073709551615 134524882 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/18133/statm: 255047 877 113 282 0 254763 0
[pid=18134] ppid=18125 vsize=293132 CPUtime=0.02 cores=0-7
/proc/18134/stat : 18134 (python) S 18125 18125 17541 0 -1 4202560 895 0 0 0 1 1 0 0 18 0 1 0 627029101 300167168 6393 33554432000 4194304 5683644 140737095724672 18446744073709551615 216735198790 0 0 16781312 514 4531782309413617923 0 0 17 0 0 0 0
/proc/18134/statm: 73283 6393 372 364 0 9198 0
[pid=18136] ppid=18134 vsize=1020188 CPUtime=10.21 cores=0-7
/proc/18136/stat : 18136 (gnovelty+2) R 18134 18136 18136 0 -1 4202496 1136 0 0 0 1021 0 0 0 25 0 1 0 627029101 1044672512 876 33554432000 134512640 135663704 4291418720 18446744073709551615 134524888 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/18136/statm: 255047 876 113 282 0 254763 0
[pid=18137] ppid=18125 vsize=302904 CPUtime=0.02 cores=0-7
/proc/18137/stat : 18137 (python) S 18125 18125 17541 0 -1 4202560 915 0 0 0 1 1 0 0 18 0 1 0 627029104 310173696 6530 33554432000 4194304 5683644 140737095724672 18446744073709551615 216735198790 0 0 16781312 514 4531782309413617922 0 0 17 6 0 0 0
/proc/18137/statm: 75726 6530 370 364 0 11638 0
[pid=18139] ppid=18137 vsize=421804 CPUtime=10.15 cores=0-7
/proc/18139/stat : 18139 (adaptg2wsat2009) R 18137 18139 18139 0 -1 4202496 2138 0 0 0 1015 0 0 0 25 0 1 0 627029105 431927296 1872 33554432000 134512640 135107411 4288628080 18446744073709551615 134531696 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/18139/statm: 105451 1872 59 146 0 104327 0
[pid=18140] ppid=18125 vsize=313160 CPUtime=0.02 cores=0-7
/proc/18140/stat : 18140 (python) S 18125 18125 17541 0 -1 4202560 937 0 0 0 1 1 0 0 18 0 1 0 627029108 320675840 6533 33554432000 4194304 5683644 140737095724672 18446744073709551615 216735198790 0 0 16781312 514 4531782309413617922 0 0 17 1 0 0 0
/proc/18140/statm: 78290 6533 370 364 0 14199 0
[pid=18142] ppid=18140 vsize=421140 CPUtime=5.19 cores=0-7
/proc/18142/stat : 18142 (adaptg2wsat2009) R 18140 18142 18142 0 -1 4202496 1803 0 0 0 519 0 0 0 25 0 1 0 627029108 431247360 1539 33554432000 134512640 135107411 4292578976 18446744073709551615 134523300 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/18142/statm: 105285 1539 49 146 0 104161 0
[pid=18143] ppid=18125 vsize=388952 CPUtime=0.02 cores=0-7
/proc/18143/stat : 18143 (python) S 18125 18125 17541 0 -1 4202560 799 0 0 0 1 1 0 0 18 0 1 0 627029112 398286848 6711 33554432000 4194304 5683644 140737095724672 18446744073709551615 216735198790 0 0 16781312 514 4531782309413617923 0 0 17 1 0 0 0
/proc/18143/statm: 97238 6711 371 364 0 33144 0
[pid=18145] ppid=18143 vsize=421804 CPUtime=9.98 cores=0-7
/proc/18145/stat : 18145 (adaptg2wsat2009) R 18143 18145 18145 0 -1 4202496 2127 0 0 0 998 0 0 0 25 0 1 0 627029112 431927296 1871 33554432000 134512640 135107411 4290272336 18446744073709551615 134531600 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/18145/statm: 105451 1871 59 146 0 104327 0
[pid=18146] ppid=18125 vsize=399208 CPUtime=0 cores=0-7
/proc/18146/stat : 18146 (python) S 18125 18125 17541 0 -1 4202560 815 0 0 0 0 0 0 0 18 0 1 0 627029116 408788992 6687 33554432000 4194304 5683644 140737095724672 18446744073709551615 216735198790 0 0 16781312 514 16561852668897624321 0 0 17 1 0 0 0
/proc/18146/statm: 99802 6687 371 364 0 35705 0
[pid=18148] ppid=18146 vsize=1020188 CPUtime=9.96 cores=0-7
/proc/18148/stat : 18148 (gnovelty+2) R 18146 18148 18148 0 -1 4202496 1131 0 0 0 996 0 0 0 25 0 1 0 627029117 1044672512 876 33554432000 134512640 135663704 4287653056 18446744073709551615 134524548 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/18148/statm: 255047 876 113 282 0 254763 0
[pid=18149] ppid=18125 vsize=409464 CPUtime=0 cores=0-7
/proc/18149/stat : 18149 (python) S 18125 18125 17541 0 -1 4202560 818 0 0 0 0 0 0 0 18 0 1 0 627029120 419291136 6693 33554432000 4194304 5683644 140737095724672 18446744073709551615 216735198790 0 0 16781312 514 12007513328684335362 0 0 17 4 0 0 0
/proc/18149/statm: 102366 6693 371 364 0 38266 0

################
# More data... #
################

/proc/18225/task/18236/stat : 18236 (python) S 18125 18125 17541 0 -1 4202560 83 0 0 0 0 0 0 0 19 0 2 0 627093906 576823296 7016 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 514 18446604449878003776 0 0 -1 1 0 0 0
[pid=18230] ppid=18125 vsize=573560 CPUtime=0.27 cores=0-7
/proc/18230/stat : 18230 (python) S 18125 18125 17541 0 -1 4202560 1178 0 0 0 12 15 0 0 19 0 2 0 627086518 587325440 6976 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747005387 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/18230/statm: 143390 6976 406 364 0 79242 0
[pid=18232] ppid=18230 vsize=1020188 CPUtime=101.83 cores=0-7
/proc/18232/stat : 18232 (gnovelty+2) T 18230 18232 18232 0 -1 4202496 1158 0 0 0 10183 0 0 0 25 0 1 0 627086519 1044672512 877 33554432000 134512640 135663704 4293739584 18446744073709551615 134524548 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/18232/statm: 255047 877 113 282 0 254763 0
[pid=18230/tid=18238] ppid=18125 vsize=573560 CPUtime=0 cores=0-7
/proc/18230/task/18238/stat : 18238 (python) S 18125 18125 17541 0 -1 4202560 80 0 0 0 0 0 0 0 19 0 2 0 627097163 587325440 6976 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 514 18446604453914101760 0 0 -1 6 0 0 0
[pid=18246] ppid=18125 vsize=583816 CPUtime=1.76 cores=0-7
/proc/18246/stat : 18246 (python) S 18125 18125 17541 0 -1 4202560 995 0 0 0 85 91 0 0 18 0 1 0 627110279 597827584 6969 33554432000 4194304 5683644 140737095724672 18446744073709551615 216735198790 0 0 16781312 514 4531782309413617923 0 0 17 1 0 0 0
/proc/18246/statm: 145954 6969 379 364 0 81803 0
[pid=18248] ppid=18246 vsize=459764 CPUtime=555.6 cores=0-7
/proc/18248/stat : 18248 (TNM) R 18246 18248 18248 0 -1 4202496 1941 0 0 0 55560 0 0 0 25 0 1 0 627110280 470798336 1657 33554432000 134512640 135008748 4293735552 18446744073709551615 134527553 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/18248/statm: 114941 1657 46 122 0 113841 0
[pid=18250] ppid=18125 vsize=594072 CPUtime=0.28 cores=0-7
/proc/18250/stat : 18250 (python) S 18125 18125 17541 0 -1 4202560 1166 0 0 0 15 13 0 0 19 0 2 0 627113468 608329728 7077 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747005387 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/18250/statm: 148518 7077 401 364 0 84364 0
[pid=18252] ppid=18250 vsize=335996 CPUtime=101.02 cores=0-7
/proc/18252/stat : 18252 (iPAWS) T 18250 18252 18252 0 -1 4202496 2035 0 0 0 10102 0 0 0 25 0 1 0 627113469 344059904 1751 33554432000 134512640 135114423 4289859744 18446744073709551615 134514774 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/18252/statm: 83999 1751 59 147 0 83851 0
[pid=18250/tid=18256] ppid=18125 vsize=594072 CPUtime=0 cores=0-7
/proc/18250/task/18256/stat : 18256 (python) S 18125 18125 17541 0 -1 4202560 82 0 0 0 0 0 0 0 19 0 2 0 627124096 608329728 7077 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 514 18446604453914099968 0 0 -1 2 0 0 0
[pid=18257] ppid=18125 vsize=604328 CPUtime=0.26 cores=0-7
/proc/18257/stat : 18257 (python) S 18125 18125 17541 0 -1 4202560 1183 0 0 0 14 12 0 0 19 0 2 0 627124105 618831872 7053 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747005387 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/18257/statm: 151082 7053 401 364 0 86925 0
[pid=18259] ppid=18257 vsize=1020188 CPUtime=99.78 cores=0-7
/proc/18259/stat : 18259 (gnovelty+2) T 18257 18259 18259 0 -1 4202496 1158 0 0 0 9978 0 0 0 25 0 1 0 627124106 1044672512 876 33554432000 134512640 135663704 4292375536 18446744073709551615 134525316 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/18259/statm: 255047 876 113 282 0 254763 0
[pid=18257/tid=18268] ppid=18125 vsize=604328 CPUtime=0 cores=0-7
/proc/18257/task/18268/stat : 18268 (python) S 18125 18125 17541 0 -1 4202560 86 0 0 0 0 0 0 0 19 0 2 0 627134136 618831872 7053 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 514 18446604450474049984 0 0 -1 6 0 0 0
[pid=18261] ppid=18125 vsize=614584 CPUtime=0.52 cores=0-7
/proc/18261/stat : 18261 (python) S 18125 18125 17541 0 -1 4202560 1197 0 0 0 25 27 0 0 19 0 2 0 627128909 629334016 7162 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747005387 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/18261/statm: 153646 7162 401 364 0 89486 0
[pid=18263] ppid=18261 vsize=268420 CPUtime=196.03 cores=0-7
/proc/18263/stat : 18263 (hybridGM3) T 18261 18263 18263 0 -1 4202496 5660 0 0 0 19603 0 0 0 25 0 1 0 627128909 274862080 5377 33554432000 134512640 135204385 4293014048 18446744073709551615 134538041 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/18263/statm: 67105 5377 63 169 0 66935 0
[pid=18261/tid=18280] ppid=18125 vsize=614584 CPUtime=0 cores=0-7
/proc/18261/task/18280/stat : 18280 (python) S 18125 18125 17541 0 -1 4202560 81 0 0 0 0 0 0 0 19 0 2 0 627148568 629334016 7162 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 514 18446604445857503104 0 0 -1 4 0 0 0
[pid=18265] ppid=18125 vsize=624840 CPUtime=0.27 cores=0-7
/proc/18265/stat : 18265 (python) S 18125 18125 17541 0 -1 4202560 1209 0 0 0 13 14 0 0 19 0 2 0 627133741 639836160 7068 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747005387 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/18265/statm: 156210 7068 401 364 0 92047 0
[pid=18267] ppid=18265 vsize=251288 CPUtime=97.71 cores=0-7
/proc/18267/stat : 18267 (FH) T 18265 18267 18267 0 -1 4202496 1978 0 0 0 9771 0 0 0 25 0 1 0 627133751 257318912 1693 33554432000 134512640 134936988 4286575344 18446744073709551615 134552080 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/18267/statm: 62822 1693 41 104 0 62326 0
[pid=18265/tid=18275] ppid=18125 vsize=624840 CPUtime=0 cores=0-7
/proc/18265/task/18275/stat : 18275 (python) S 18125 18125 17541 0 -1 4202560 85 0 0 0 0 0 0 0 19 0 2 0 627143777 639836160 7068 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 514 18446604445857502208 0 0 -1 4 0 0 0
[pid=18269] ppid=18125 vsize=635096 CPUtime=1.15 cores=0-7
/proc/18269/stat : 18269 (python) S 18125 18125 17541 0 -1 4202560 1245 0 0 0 57 58 0 0 18 0 2 0 627134147 650338304 7134 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747005387 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/18269/statm: 158774 7134 407 364 0 94608 0
[pid=18271] ppid=18269 vsize=268424 CPUtime=393.25 cores=0-7
/proc/18271/stat : 18271 (hybridGM3) T 18269 18271 18271 0 -1 4202496 5666 0 0 0 39324 1 0 0 25 0 1 0 627134147 274866176 5378 33554432000 134512640 135204385 4293818016 18446744073709551615 134537792 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/18271/statm: 67106 5378 63 169 0 66936 0
[pid=18269/tid=18286] ppid=18125 vsize=635096 CPUtime=0 cores=0-7
/proc/18269/task/18286/stat : 18286 (python) S 18125 18125 17541 0 -1 4202560 88 0 0 0 0 0 0 0 18 0 2 0 627156119 650338304 7134 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 514 18446604450522860352 0 0 -1 2 0 0 0
[pid=18276] ppid=18125 vsize=645352 CPUtime=0.87 cores=0-7
/proc/18276/stat : 18276 (python) S 18125 18125 17541 0 -1 4202560 1042 0 0 0 39 48 0 0 18 0 1 0 627143787 660840448 7036 33554432000 4194304 5683644 140737095724672 18446744073709551615 216735198790 0 0 16781312 514 4531782309413617922 0 0 17 1 0 0 0
/proc/18276/statm: 161338 7036 379 364 0 97169 0
[pid=18278] ppid=18276 vsize=421340 CPUtime=265.25 cores=0-7
/proc/18278/stat : 18278 (NCVWr) R 18276 18278 18278 0 -1 4202496 2000 0 0 0 26523 2 0 0 25 0 1 0 627143788 431452160 1712 33554432000 134512640 135113558 4289806224 18446744073709551615 134533568 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/18278/statm: 105335 1712 58 147 0 104210 0
[pid=18281] ppid=18125 vsize=655608 CPUtime=0.74 cores=0-7
/proc/18281/stat : 18281 (python) S 18125 18125 17541 0 -1 4202560 1258 0 0 0 37 37 0 0 18 0 2 0 627148579 671342592 7139 33554432000 4194304 5683644 140737095724672 18446744073709551615 216735198790 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/18281/statm: 163902 7139 407 364 0 99730 0
[pid=18283] ppid=18281 vsize=268424 CPUtime=220.41 cores=0-7
/proc/18283/stat : 18283 (hybridGM3) R 18281 18283 18283 0 -1 4202496 5668 0 0 0 22040 1 0 0 25 0 1 0 627148579 274866176 5378 33554432000 134512640 135204385 4286809488 18446744073709551615 134537724 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/18283/statm: 67106 5378 63 169 0 66936 0
[pid=18281/tid=18296] ppid=18125 vsize=655608 CPUtime=0 cores=0-7
/proc/18281/task/18296/stat : 18296 (python) S 18125 18125 17541 0 -1 4202560 86 0 0 0 0 0 0 0 19 0 2 0 627174758 671342592 7139 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 514 18446604450522858560 0 0 -1 4 0 0 0
[pid=18288] ppid=18125 vsize=665864 CPUtime=0.34 cores=0-7
/proc/18288/stat : 18288 (python) S 18125 18125 17541 0 -1 4202560 1264 0 0 0 16 18 0 0 19 0 2 0 627159430 681844736 7205 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747005387 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/18288/statm: 166466 7205 407 364 0 102291 0
[pid=18290] ppid=18288 vsize=335996 CPUtime=98.06 cores=0-7
/proc/18290/stat : 18290 (iPAWS) T 18288 18290 18290 0 -1 4202496 2033 0 0 0 9806 0 0 0 25 0 1 0 627159431 344059904 1749 33554432000 134512640 135114423 4287792832 18446744073709551615 134517083 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/18290/statm: 83999 1749 59 147 0 83851 0
[pid=18288/tid=18294] ppid=18125 vsize=665864 CPUtime=0 cores=0-7
/proc/18288/task/18294/stat : 18294 (python) S 18125 18125 17541 0 -1 4202560 89 0 0 0 0 0 0 0 19 0 2 0 627171863 681844736 7205 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 514 18446604453629039808 0 0 -1 0 0 0 0
[pid=18298] ppid=18125 vsize=676120 CPUtime=0 cores=0-7
/proc/18298/stat : 18298 (python) S 18125 18125 17541 0 -1 4202560 941 0 0 0 0 0 0 0 19 0 1 0 627177005 692346880 7152 33554432000 4194304 5683644 140737095724672 18446744073709551615 216735198790 0 0 16781312 514 10306718099218366463 0 0 17 2 0 0 0
/proc/18298/statm: 169030 7152 350 364 0 104852 0
[pid=18300] ppid=18298 vsize=268424 CPUtime=0.1 cores=0-7
/proc/18300/stat : 18300 (hybridGM3) R 18298 18300 18300 0 -1 4202496 5669 0 0 0 10 0 0 0 21 0 1 0 627177005 274866176 5378 33554432000 134512640 135204385 4294646080 18446744073709551615 134538257 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/18300/statm: 67106 5378 63 169 0 66936 0
[pid=18125/tid=18132] ppid=18123 vsize=686364 CPUtime=0 cores=0-7
/proc/18125/task/18132/stat : 18132 (python) S 18123 18125 17541 0 -1 4202560 26 933 0 0 0 0 0 0 22 0 34 0 627029097 702836736 8748 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 0 0 0 0
[pid=18125/tid=18135] ppid=18123 vsize=686364 CPUtime=0 cores=0-7
/proc/18125/task/18135/stat : 18135 (python) S 18123 18125 17541 0 -1 4202560 24 933 0 0 0 0 0 0 24 0 34 0 627029101 702836736 8748 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 4 0 0 0
[pid=18125/tid=18138] ppid=18123 vsize=686364 CPUtime=0 cores=0-7
/proc/18125/task/18138/stat : 18138 (python) S 18123 18125 17541 0 -1 4202560 37 933 0 0 0 0 0 0 15 0 34 0 627029105 702836736 8748 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 6 0 0 0
[pid=18125/tid=18141] ppid=18123 vsize=686364 CPUtime=0 cores=0-7
/proc/18125/task/18141/stat : 18141 (python) S 18123 18125 17541 0 -1 4202560 25 933 0 0 0 0 0 0 18 0 34 0 627029108 702836736 8748 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 0 0 0 0
[pid=18125/tid=18144] ppid=18123 vsize=686364 CPUtime=0 cores=0-7
/proc/18125/task/18144/stat : 18144 (python) S 18123 18125 17541 0 -1 4202560 103 933 0 0 0 0 0 0 18 0 34 0 627029112 702836736 8748 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 7 0 0 0
[pid=18125/tid=18147] ppid=18123 vsize=686364 CPUtime=0 cores=0-7
/proc/18125/task/18147/stat : 18147 (python) S 18123 18125 17541 0 -1 4202560 32 933 0 0 0 0 0 0 23 0 34 0 627029116 702836736 8748 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 1 0 0 0
[pid=18125/tid=18150] ppid=18123 vsize=686364 CPUtime=0 cores=0-7
/proc/18125/task/18150/stat : 18150 (python) S 18123 18125 17541 0 -1 4202560 58 933 0 0 0 0 0 0 18 0 34 0 627029120 702836736 8748 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 4 0 0 0
[pid=18125/tid=18152] ppid=18123 vsize=686364 CPUtime=0 cores=0-7
/proc/18125/task/18152/stat : 18152 (python) S 18123 18125 17541 0 -1 4202560 97 933 0 0 0 0 0 0 18 0 34 0 627029124 702836736 8748 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 4 0 0 0
[pid=18125/tid=18161] ppid=18123 vsize=686364 CPUtime=0 cores=0-7
/proc/18125/task/18161/stat : 18161 (python) S 18123 18125 17541 0 -1 4202560 46 933 0 0 0 0 0 0 18 0 34 0 627039129 702836736 8748 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 1 0 0 0
[pid=18125/tid=18164] ppid=18123 vsize=686364 CPUtime=0 cores=0-7
/proc/18125/task/18164/stat : 18164 (python) S 18123 18125 17541 0 -1 4202560 54 933 0 0 0 0 0 0 18 0 34 0 627039134 702836736 8748 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 4 0 0 0
[pid=18125/tid=18168] ppid=18123 vsize=686364 CPUtime=0 cores=0-7
/proc/18125/task/18168/stat : 18168 (python) S 18123 18125 17541 0 -1 4202560 52 933 0 0 0 0 0 0 18 0 34 0 627039142 702836736 8748 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 4 0 0 0
[pid=18125/tid=18172] ppid=18123 vsize=686364 CPUtime=0 cores=0-7
/proc/18125/task/18172/stat : 18172 (python) S 18123 18125 17541 0 -1 4202560 33 933 0 0 0 0 0 0 23 0 34 0 627039155 702836736 8748 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 4 0 0 0
[pid=18125/tid=18176] ppid=18123 vsize=686364 CPUtime=0 cores=0-7
/proc/18125/task/18176/stat : 18176 (python) S 18123 18125 17541 0 -1 4202560 50 933 0 0 0 0 0 0 18 0 34 0 627040962 702836736 8748 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 4 0 0 0
[pid=18125/tid=18182] ppid=18123 vsize=686364 CPUtime=0 cores=0-7
/proc/18125/task/18182/stat : 18182 (python) S 18123 18125 17541 0 -1 4202560 43 933 0 0 0 0 0 0 18 0 34 0 627041540 702836736 8748 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 5 0 0 0
[pid=18125/tid=18185] ppid=18123 vsize=686364 CPUtime=0 cores=0-7
/proc/18125/task/18185/stat : 18185 (python) S 18123 18125 17541 0 -1 4202560 50 933 0 0 0 0 0 0 18 0 34 0 627041546 702836736 8748 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 0 0 0 0
[pid=18125/tid=18189] ppid=18123 vsize=686364 CPUtime=0 cores=0-7
/proc/18125/task/18189/stat : 18189 (python) S 18123 18125 17541 0 -1 4202560 39 933 0 0 0 0 0 0 18 0 34 0 627041562 702836736 8748 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 4 0 0 0
[pid=18125/tid=18195] ppid=18123 vsize=686364 CPUtime=0 cores=0-7
/proc/18125/task/18195/stat : 18195 (python) S 18123 18125 17541 0 -1 4202560 34 933 0 0 0 0 0 0 19 0 34 0 627049207 702836736 8748 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 4 0 0 0
[pid=18125/tid=18200] ppid=18123 vsize=686364 CPUtime=0 cores=0-7
/proc/18125/task/18200/stat : 18200 (python) S 18123 18125 17541 0 -1 4202560 37 933 0 0 0 0 0 0 18 0 34 0 627049667 702836736 8748 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 4 0 0 0
[pid=18125/tid=18213] ppid=18123 vsize=686364 CPUtime=0 cores=0-7
/proc/18125/task/18213/stat : 18213 (python) S 18123 18125 17541 0 -1 4202560 55 933 0 0 0 0 0 0 18 0 34 0 627064516 702836736 8748 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 0 0 0 0
[pid=18125/tid=18216] ppid=18123 vsize=686364 CPUtime=0 cores=0-7
/proc/18125/task/18216/stat : 18216 (python) S 18123 18125 17541 0 -1 4202560 41 933 0 0 0 0 0 0 18 0 34 0 627066118 702836736 8748 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 4 0 0 0
[pid=18125/tid=18220] ppid=18123 vsize=686364 CPUtime=0 cores=0-7
/proc/18125/task/18220/stat : 18220 (python) S 18123 18125 17541 0 -1 4202560 35 933 0 0 0 0 0 0 19 0 34 0 627070241 702836736 8748 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 4 0 0 0
[pid=18125/tid=18226] ppid=18123 vsize=686364 CPUtime=0 cores=0-7
/proc/18125/task/18226/stat : 18226 (python) S 18123 18125 17541 0 -1 4202560 37 933 0 0 0 0 0 0 19 0 34 0 627083280 702836736 8748 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 0 0 0 0
[pid=18125/tid=18231] ppid=18123 vsize=686364 CPUtime=0 cores=0-7
/proc/18125/task/18231/stat : 18231 (python) S 18123 18125 17541 0 -1 4202560 36 933 0 0 0 0 0 0 19 0 34 0 627086518 702836736 8748 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 0 0 0 0
[pid=18125/tid=18247] ppid=18123 vsize=686364 CPUtime=0 cores=0-7
/proc/18125/task/18247/stat : 18247 (python) S 18123 18125 17541 0 -1 4202560 40 933 0 0 0 0 0 0 19 0 34 0 627110280 702836736 8748 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 0 0 0 0
[pid=18125/tid=18251] ppid=18123 vsize=686364 CPUtime=0 cores=0-7
/proc/18125/task/18251/stat : 18251 (python) S 18123 18125 17541 0 -1 4202560 36 933 0 0 0 0 0 0 19 0 34 0 627113468 702836736 8748 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 0 0 0 0
[pid=18125/tid=18258] ppid=18123 vsize=686364 CPUtime=0 cores=0-7
/proc/18125/task/18258/stat : 18258 (python) S 18123 18125 17541 0 -1 4202560 33 933 0 0 0 0 0 0 20 0 34 0 627124105 702836736 8748 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 0 0 0 0
[pid=18125/tid=18262] ppid=18123 vsize=686364 CPUtime=0 cores=0-7
/proc/18125/task/18262/stat : 18262 (python) S 18123 18125 17541 0 -1 4202560 34 933 0 0 0 0 0 0 20 0 34 0 627128909 702836736 8748 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 0 0 0 0
[pid=18125/tid=18266] ppid=18123 vsize=686364 CPUtime=0 cores=0-7
/proc/18125/task/18266/stat : 18266 (python) S 18123 18125 17541 0 -1 4202560 36 933 0 0 0 0 0 0 19 0 34 0 627133742 702836736 8748 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 0 0 0 0
[pid=18125/tid=18270] ppid=18123 vsize=686364 CPUtime=0 cores=0-7
/proc/18125/task/18270/stat : 18270 (python) S 18123 18125 17541 0 -1 4202560 43 933 0 0 0 0 0 0 18 0 34 0 627134147 702836736 8748 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 0 0 0 0
[pid=18125/tid=18277] ppid=18123 vsize=686364 CPUtime=0 cores=0-7
/proc/18125/task/18277/stat : 18277 (python) S 18123 18125 17541 0 -1 4202560 32 933 0 0 0 0 0 0 19 0 34 0 627143787 702836736 8748 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 3 0 0 0
[pid=18125/tid=18282] ppid=18123 vsize=686364 CPUtime=0 cores=0-7
/proc/18125/task/18282/stat : 18282 (python) S 18123 18125 17541 0 -1 4202560 36 933 0 0 0 0 0 0 18 0 34 0 627148579 702836736 8748 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 0 0 0 0
[pid=18125/tid=18289] ppid=18123 vsize=686364 CPUtime=0 cores=0-7
/proc/18125/task/18289/stat : 18289 (python) S 18123 18125 17541 0 -1 4202560 34 933 0 0 0 0 0 0 19 0 34 0 627159430 702836736 8748 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 0 0 0 0
[pid=18125/tid=18299] ppid=18123 vsize=686364 CPUtime=0 cores=0-7
/proc/18125/task/18299/stat : 18299 (python) S 18123 18125 17541 0 -1 4202560 36 933 0 0 0 0 0 0 20 0 34 0 627177005 702836736 8748 33554432000 4194304 5683644 140737095724672 18446744073709551615 216747002289 0 0 16781312 2 18446604440665913472 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 10473
Current children cumulated vsize (KiB) 32612660

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 18125 and gives
#  childrusage.ru_utime.tv_sec=7
#  childrusage.ru_utime.tv_usec=392876
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=241963
# CPU time returned by wait4() is 7.63484
# while last known CPU time is 10473
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1481.81
CPU time (s): 10473
CPU user time (s): 10457.4
CPU system time (s): 15.62
CPU usage (%): 706.772
Max. virtual memory (cumulated for all children) (KiB): 32612660

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 7.39288
system time used= 0.241963
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 79622
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1366
involuntary context switches= 170

runsolver used 16.3975 second user time and 37.3543 second system time

The end

Launcher Data 

Begin job on node119 at 2011-05-02 22:26:22
IDJOB=3321823
IDBENCH=82601
IDSOLVER=1712
FILE ID=node119/3321823-1304367982
RUNJOBID= node119-1304366467-17555
PBS_JOBID= 13181752
Free space on /tmp= 73408 MiB

SOLVER NAME= borg-sat 11.03.02-random
BENCH NAME= SAT11/random/large/unif-k7-r85-v400-c34000-S927774663-037-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3321823-1304367982/watcher-3321823-1304367982 -o /tmp/evaluation-result-3321823-1304367982/solver-3321823-1304367982 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3321823-1304367982.cnf -seed 1833446829 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 950830e22ef92fb7f22c5b3c987a21f5
RANDOM SEED=1833446829

node119.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.63
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      17993632 kB
Buffers:        332336 kB
Cached:        1609504 kB
SwapCached:       3692 kB
Active:       11256552 kB
Inactive:      3233308 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      17993632 kB
SwapTotal:    67111528 kB
SwapFree:     67102128 kB
Dirty:           33308 kB
Writeback:           8 kB
AnonPages:    12547960 kB
Mapped:          25872 kB
Slab:           353728 kB
PageTables:      46276 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 69617872 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73376 MiB
End job on node119 at 2011-05-02 22:51:06