Trace number 3385286

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)? (MO) 10093.6 1269.61

General information on the benchmark

Namerandom/large/
unif-k7-r85-v300-c25500-S966156467-066-UNKNOWN.cnf
MD5SUM5297bd1d2a420892cfb1fb6d32ccacad
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables300
Number of clauses25500
Sum of the clauses size178500
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 525500

Solver Data 

0.37/0.75	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.37/0.75	c INFO: solving HOME/instance-3385286-1305564967.cnf
0.37/0.75	c INFO: computing task features
0.55/0.92	c INFO: computed variable-clause graph statistics
0.55/0.93	c INFO: computed clause balance statistics
1.34/1.78	c INFO: computed variable balance statistics
1.74/2.12	c INFO: computed small-clause counts
1.74/2.12	c INFO: computed Horn-clause counts
1.74/2.15	c INFO: computed variable graph statistics
1.74/2.15	c INFO: collected features for HOME/instance-3385286-1305564967.cnf in 1.39 s
1.74/2.20	c INFO: running gnovelty+2@0 for 97 with 4998 remaining (b = 0.37)
1.83/2.23	c INFO: running gnovelty+2@0 for 97 with 4998 remaining (b = 0.13)
1.83/2.26	c INFO: running adaptg2wsat2009++@0 for 97 with 4998 remaining (b = 0.08)
1.83/2.29	c INFO: running adaptg2wsat2009++@0 for 97 with 4998 remaining (b = 0.07)
1.94/2.33	c INFO: running adaptg2wsat2009++@0 for 97 with 4998 remaining (b = 0.05)
1.94/2.36	c INFO: running gnovelty+2@0 for 97 with 4998 remaining (b = 0.04)
1.94/2.39	c INFO: running FH@0 for 97 with 4998 remaining (b = 0.03)
2.02/2.42	c INFO: running adaptg2wsat2009++@0 for 97 with 4998 remaining (b = 0.03)
802.12/102.50	c INFO: running adaptg2wsat2009++@0 for 97 with 4898 remaining (b = 0.02)
802.12/102.53	c INFO: running FH@0 for 97 with 4898 remaining (b = 0.02)
802.12/102.57	c INFO: running hybridGM3@0 for 194 with 4898 remaining (b = 0.04)
802.62/102.61	c INFO: running gnovelty+2@0 for 97 with 4898 remaining (b = 0.02)
802.62/102.66	c INFO: running hybridGM3@0 for 194 with 4898 remaining (b = 0.03)
802.93/102.70	c INFO: running adaptg2wsat2009++@0 for 97 with 4898 remaining (b = 0.01)
802.93/102.75	c INFO: running hybridGM3@0 for 194 with 4898 remaining (b = 0.03)
803.05/102.80	c INFO: running hybridGM3@0 for 194 with 4897 remaining (b = 0.02)
1601.14/202.78	c INFO: running TNM@0 for 1552 with 4798 remaining (b = 0.19)
1601.90/202.83	c INFO: running FH@0 for 97 with 4797 remaining (b = 0.01)
1602.53/202.95	c INFO: running adaptg2wsat2009++@0 for 97 with 4797 remaining (b = 0.01)
1603.13/203.03	c INFO: running adaptg2wsat2009++@0 for 97 with 4797 remaining (b = 0.01)
2361.46/298.17	c INFO: running hybridGM3@100 for 97 with 4702 remaining (b = 0.01)
2370.95/299.32	c INFO: running hybridGM3@99 for 97 with 4701 remaining (b = 0.01)
2371.66/299.40	c INFO: running hybridGM3@100 for 97 with 4701 remaining (b = 0.01)
2378.05/300.30	c INFO: running adaptg2wsat2009++@0 for 97 with 4700 remaining (b = 0.01)
2401.14/303.13	c INFO: running adaptg2wsat2009++@100 for 97 with 4697 remaining (b = 0.01)
2417.87/305.25	c INFO: running hybridGM3@99 for 97 with 4695 remaining (b = 0.01)
2425.84/306.25	c INFO: running hybridGM3@0 for 194 with 4694 remaining (b = 0.02)
3146.31/396.62	c INFO: running hybridGM3@0 for 194 with 4604 remaining (b = 0.02)
3160.62/398.48	c INFO: running hybridGM3@296 for 97 with 4602 remaining (b = 0.01)
3176.65/400.42	c INFO: running TNM@0 for 1552 with 4600 remaining (b = 0.15)
3177.35/400.53	c INFO: running adaptg2wsat2009++@100 for 97 with 4600 remaining (b = 0.01)
3177.99/400.62	c INFO: running hybridGM3@295 for 97 with 4600 remaining (b = 0.01)
3177.99/400.69	c INFO: running FH@0 for 291 with 4600 remaining (b = 0.03)
3912.24/492.88	c INFO: running gnovelty+2@0 for 97 with 4507 remaining (b = 0.01)
3936.04/495.90	c INFO: running hybridGM3@393 for 97 with 4504 remaining (b = 0.01)
3951.96/497.83	c INFO: running hybridGM3@396 for 97 with 4502 remaining (b = 0.01)
3991.87/502.84	c INFO: running iPAWS@0 for 97 with 4497 remaining (b = 0.01)
4703.32/592.11	c INFO: running hybridGM3@96 for 97 with 4408 remaining (b = 0.01)
4704.06/592.23	c INFO: running adaptg2wsat2009++@99 for 97 with 4408 remaining (b = 0.01)
4704.06/592.28	c INFO: running FH@99 for 291 with 4408 remaining (b = 0.02)
4711.99/593.22	c INFO: running hybridGM3@100 for 97 with 4407 remaining (b = 0.01)
4782.93/602.17	c INFO: running adaptg2wsat2009++@299 for 291 with 4398 remaining (b = 0.02)
5470.84/688.58	c INFO: running iPAWS@0 for 97 with 4312 remaining (b = 0.01)
5477.99/689.41	c INFO: running hybridGM3@192 for 97 with 4311 remaining (b = 0.01)
5518.54/694.51	c INFO: running hybridGM3@0 for 194 with 4306 remaining (b = 0.02)
5573.44/701.48	c INFO: running TNM@0 for 1552 with 4299 remaining (b = 0.11)
6235.69/784.69	c INFO: running hybridGM3@293 for 97 with 4216 remaining (b = 0.01)
6261.16/787.89	c INFO: running FH@0 for 97 with 4212 remaining (b = 0.01)
7027.67/884.12	c INFO: running FH@294 for 97 with 4116 remaining (b = 0.01)
7041.98/885.99	c INFO: running hybridGM3@392 for 97 with 4114 remaining (b = 0.01)
7089.72/891.92	c INFO: running adaptg2wsat2009++@395 for 194 with 4108 remaining (b = 0.01)
7100.13/893.24	c INFO: running hybridGM3@294 for 194 with 4107 remaining (b = 0.02)
7108.01/894.23	c INFO: running NCVWr@0 for 485 with 4106 remaining (b = 0.03)
7786.16/979.45	c INFO: running FH@290 for 194 with 4021 remaining (b = 0.01)
7800.58/981.29	c INFO: running gnovelty+2@0 for 97 with 4019 remaining (b = 0.01)
8625.10/1084.81	c INFO: running hybridGM3@395 for 97 with 3915 remaining (b = 0.01)
8631.48/1085.69	c INFO: running iPAWS@0 for 97 with 3915 remaining (b = 0.01)
8661.69/1089.47	c INFO: running hybridGM3@0 for 194 with 3911 remaining (b = 0.01)
9326.98/1173.01	c INFO: running FH@385 for 97 with 3827 remaining (b = 0.01)
9414.50/1184.04	c INFO: running adaptg2wsat2009++@99 for 97 with 3816 remaining (b = 0.01)
9423.24/1185.13	c INFO: running hybridGM3@0 for 194 with 3815 remaining (b = 0.01)
10084.76/1268.32	c INFO: running hybridGM3@0 for 194 with 3732 remaining (b = 0.01)
10093.61/1269.50	c INFO: running TNM@0 for 1552 with 3731 remaining (b = 0.08)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3385286-1305564967/watcher-3385286-1305564967 -o /tmp/evaluation-result-3385286-1305564967/solver-3385286-1305564967 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385286-1305564967.cnf -seed 592305601 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.00 0.00 0.00 1/300 25128
/proc/meminfo: memFree=15547612/32951124 swapFree=52307772/67111528
[pid=25128] ppid=25126 vsize=15780 CPUtime=0 cores=0-7
/proc/25128/stat : 25128 (python) R 25126 25128 25102 0 -1 4202496 830 0 0 0 0 0 0 0 20 0 1 0 668099561 16158720 716 33554432000 4194304 5683644 140734980559984 18446744073709551615 4992576 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/25128/statm: 3945 720 334 364 0 432 0

[startup+0.11132 s]
/proc/loadavg: 0.00 0.00 0.00 1/300 25128
/proc/meminfo: memFree=15547612/32951124 swapFree=52307772/67111528
[pid=25128] ppid=25126 vsize=85072 CPUtime=0.06 cores=0-7
/proc/25128/stat : 25128 (python) R 25126 25128 25102 0 -1 4202496 2471 0 0 0 4 2 0 0 17 0 1 0 668099561 87113728 2199 33554432000 4194304 5683644 140734980559984 18446744073709551615 5114052 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/25128/statm: 21268 2199 707 364 0 1485 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 85072

[startup+0.20032 s]
/proc/loadavg: 0.00 0.00 0.00 1/300 25128
/proc/meminfo: memFree=15547612/32951124 swapFree=52307772/67111528
[pid=25128] ppid=25126 vsize=116324 CPUtime=0.09 cores=0-7
/proc/25128/stat : 25128 (python) S 25126 25128 25102 0 -1 4202496 3779 0 0 0 6 3 0 0 17 0 1 0 668099561 119115776 3418 33554432000 4194304 5683644 140734980559984 18446744073709551615 218370955264 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 0
/proc/25128/statm: 29081 3418 895 364 0 2582 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 116324

[startup+0.300301 s]
/proc/loadavg: 0.00 0.00 0.00 1/300 25128
/proc/meminfo: memFree=15547612/32951124 swapFree=52307772/67111528
[pid=25128] ppid=25126 vsize=135748 CPUtime=0.17 cores=0-7
/proc/25128/stat : 25128 (python) R 25126 25128 25102 0 -1 4202496 5962 934 1 0 12 5 0 0 18 0 1 0 668099561 139005952 4151 33554432000 4194304 5683644 140734980559984 18446744073709551615 218370953221 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/25128/statm: 33937 4151 1032 364 0 3123 0
Current children cumulated CPU time (s) 0.17
Current children cumulated vsize (KiB) 135748

[startup+0.700237 s]
/proc/loadavg: 0.00 0.00 0.00 1/300 25128
/proc/meminfo: memFree=15547612/32951124 swapFree=52307772/67111528
[pid=25128] ppid=25126 vsize=249172 CPUtime=0.37 cores=0-7
/proc/25128/stat : 25128 (python) R 25126 25128 25102 0 -1 4202496 8296 934 1 0 27 10 0 0 17 0 1 0 668099561 255152128 6188 33554432000 4194304 5683644 140734980559984 18446744073709551615 218366018303 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/25128/statm: 62293 6188 1627 364 0 4770 0
Current children cumulated CPU time (s) 0.37
Current children cumulated vsize (KiB) 249172

[startup+1.50108 s]
/proc/loadavg: 0.08 0.02 0.01 2/301 25133
/proc/meminfo: memFree=15518208/32951124 swapFree=52307772/67111528
[pid=25128] ppid=25126 vsize=285672 CPUtime=1.14 cores=0-7
/proc/25128/stat : 25128 (python) R 25126 25128 25102 0 -1 4202496 12374 934 1 0 102 12 0 0 17 0 1 0 668099561 292528128 8803 33554432000 4194304 5683644 140734980559984 18446744073709551615 4566996 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/25128/statm: 71418 8803 1826 364 0 7342 0
Current children cumulated CPU time (s) 1.14
Current children cumulated vsize (KiB) 285672

[startup+3.10075 s]
/proc/loadavg: 0.08 0.02 0.01 2/301 25133
/proc/meminfo: memFree=15522920/32951124 swapFree=52307772/67111528
[pid=25128] ppid=25126 vsize=428052 CPUtime=2.06 cores=0-7
/proc/25128/stat : 25128 (python) S 25126 25128 25102 0 -1 4202496 18656 934 1 0 192 14 0 0 21 0 9 0 668099561 438325248 7598 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382783947 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/25128/statm: 107013 7598 1891 364 0 42910 0
[pid=25128/tid=25135] ppid=25126 vsize=428052 CPUtime=0 cores=0-7
/proc/25128/task/25135/stat : 25135 (python) S 25126 25128 25102 0 -1 4202560 24 934 0 0 0 0 0 0 18 0 9 0 668099781 438325248 7598 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 2.06
Current children cumulated vsize (KiB) 428052
heavy processes:

[startup+6.30119 s]
/proc/loadavg: 0.71 0.15 0.05 9/325 25157
/proc/meminfo: memFree=15458968/32951124 swapFree=52307772/67111528
[pid=25128] ppid=25126 vsize=428052 CPUtime=2.06 cores=0-7
/proc/25128/stat : 25128 (python) S 25126 25128 25102 0 -1 4202496 18656 934 1 0 192 14 0 0 21 0 9 0 668099561 438325248 7598 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382783947 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/25128/statm: 107013 7598 1891 364 0 42910 0
[pid=25134] ppid=25128 vsize=280692 CPUtime=0 cores=0-7
/proc/25134/stat : 25134 (python) S 25128 25128 25102 0 -1 4202560 896 0 0 0 0 0 0 0 18 0 1 0 668099781 287428608 5984 33554432000 4194304 5683644 140734980559984 18446744073709551615 218370977295 0 0 16781312 514 3636592107088478464 0 0 17 0 0 0 0
/proc/25134/statm: 70173 5984 378 364 0 6091 0
[pid=25136] ppid=25134 vsize=1019708 CPUtime=4.09 cores=0-7
/proc/25136/stat : 25136 (gnovelty+2) R 25134 25136 25136 0 -1 4202496 953 0 0 0 409 0 0 0 25 0 1 0 668099781 1044180992 690 33554432000 134512640 135663704 4291163824 18446744073709551615 134524540 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/25136/statm: 254927 690 113 282 0 254643 0
[pid=25137] ppid=25128 vsize=357092 CPUtime=0 cores=0-7
/proc/25137/stat : 25137 (python) S 25128 25128 25102 0 -1 4202560 904 0 0 0 0 0 0 0 18 0 1 0 668099784 365662208 6001 33554432000 4194304 5683644 140734980559984 18446744073709551615 218370977350 0 0 16781312 514 3636599803669872900 0 0 17 6 0 0 0
/proc/25137/statm: 89273 6001 372 364 0 25188 0
[pid=25139] ppid=25137 vsize=1019708 CPUtime=4.06 cores=0-7
/proc/25139/stat : 25139 (gnovelty+2) R 25137 25139 25139 0 -1 4202496 955 0 0 0 406 0 0 0 25 0 1 0 668099784 1044180992 690 33554432000 134512640 135663704 4288312816 18446744073709551615 134545682 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/25139/statm: 254927 690 113 282 0 254643 0
[pid=25140] ppid=25128 vsize=366876 CPUtime=0 cores=0-7
/proc/25140/stat : 25140 (python) S 25128 25128 25102 0 -1 4202560 896 0 0 0 0 0 0 0 18 0 1 0 668099787 375681024 6005 33554432000 4194304 5683644 140734980559984 18446744073709551615 218370977350 0 0 16781312 514 13221698246801326340 0 0 17 4 0 0 0
/proc/25140/statm: 91719 6005 372 364 0 27631 0
[pid=25142] ppid=25140 vsize=419820 CPUtime=3.98 cores=0-7
/proc/25142/stat : 25142 (adaptg2wsat2009) R 25140 25142 25142 0 -1 4202496 1441 0 0 0 398 0 0 0 25 0 1 0 668099788 429895680 1172 33554432000 134512640 135107411 4291083184 18446744073709551615 134523294 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/25142/statm: 104955 1172 49 146 0 103831 0
[pid=25143] ppid=25128 vsize=377132 CPUtime=0 cores=0-7
/proc/25143/stat : 25143 (python) S 25128 25128 25102 0 -1 4202560 754 0 0 0 0 0 0 0 18 0 1 0 668099791 386183168 6003 33554432000 4194304 5683644 140734980559984 18446744073709551615 218370977350 0 0 16781312 514 4581132793609683203 0 0 17 6 0 0 0
/proc/25143/statm: 94283 6003 349 364 0 30192 0
[pid=25145] ppid=25143 vsize=420220 CPUtime=3.98 cores=0-7
/proc/25145/stat : 25145 (adaptg2wsat2009) R 25143 25145 25145 0 -1 4202496 1667 0 0 0 398 0 0 0 25 0 1 0 668099791 430305280 1399 33554432000 134512640 135107411 4294214864 18446744073709551615 134531600 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/25145/statm: 105055 1399 59 146 0 103931 0
[pid=25146] ppid=25128 vsize=387456 CPUtime=0 cores=0-7
/proc/25146/stat : 25146 (python) S 25128 25128 25102 0 -1 4202560 756 0 0 0 0 0 0 0 18 0 1 0 668099794 396754944 6007 33554432000 4194304 5683644 140734980559984 18446744073709551615 218370977350 0 0 16781312 514 958413503901696257 0 0 17 4 0 0 0
/proc/25146/statm: 96864 6007 349 364 0 32770 0
[pid=25148] ppid=25146 vsize=419820 CPUtime=3.94 cores=0-7
/proc/25148/stat : 25148 (adaptg2wsat2009) R 25146 25148 25148 0 -1 4202496 1441 0 0 0 394 0 0 0 25 0 1 0 668099794 429895680 1173 33554432000 134512640 135107411 4294872720 18446744073709551615 134523440 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/25148/statm: 104955 1173 49 146 0 103831 0
[pid=25149] ppid=25128 vsize=397780 CPUtime=0 cores=0-7
/proc/25149/stat : 25149 (python) S 25128 25128 25102 0 -1 4202560 764 0 0 0 0 0 0 0 18 0 1 0 668099797 407326720 6028 33554432000 4194304 5683644 140734980559984 18446744073709551615 218370977350 0 0 16781312 514 958409655610999041 0 0 17 0 0 0 0
/proc/25149/statm: 99445 6028 349 364 0 35348 0
[pid=25151] ppid=25149 vsize=1019708 CPUtime=3.91 cores=0-7
/proc/25151/stat : 25151 (gnovelty+2) R 25149 25151 25151 0 -1 4202496 954 0 0 0 391 0 0 0 25 0 1 0 668099798 1044180992 690 33554432000 134512640 135663704 4294670608 18446744073709551615 134525070 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/25151/statm: 254927 690 113 282 0 254643 0
[pid=25152] ppid=25128 vsize=407900 CPUtime=0 cores=0-7
/proc/25152/stat : 25152 (python) S 25128 25128 25102 0 -1 4202560 770 0 0 0 0 0 0 0 18 0 1 0 668099801 417689600 6049 33554432000 4194304 5683644 140734980559984 18446744073709551615 218370977350 0 0 16781312 514 5674618822787563779 0 0 17 1 0 0 0
/proc/25152/statm: 101975 6049 349 364 0 37875 0
[pid=25154] ppid=25152 vsize=247956 CPUtime=3.85 cores=0-7
/proc/25154/stat : 25154 (FH) R 25152 25154 25154 0 -1 4202496 1255 0 0 0 385 0 0 0 25 0 1 0 668099802 253906944 986 33554432000 134512640 134936988 4293805216 18446744073709551615 134523624 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/25154/statm: 61989 986 37 104 0 61493 0
[pid=25155] ppid=25128 vsize=418764 CPUtime=0 cores=0-7
/proc/25155/stat : 25155 (python) S 25128 25128 25102 0 -1 4202560 799 0 0 0 0 0 0 0 18 0 1 0 668099804 428814336 6059 33554432000 4194304 5683644 140734980559984 18446744073709551615 218370977350 0 0 16781312 514 5674614974496866562 0 0 17 1 0 0 0
/proc/25155/statm: 104691 6059 355 364 0 40588 0
[pid=25157] ppid=25155 vsize=419820 CPUtime=3.82 cores=0-7
/proc/25157/stat : 25157 (adaptg2wsat2009) R 25155 25157 25157 0 -1 4202496 1448 0 0 0 382 0 0 0 25 0 1 0 668099806 429895680 1173 33554432000 134512640 135107411 4293198800 18446744073709551615 134523457 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/25157/statm: 104955 1173 49 146 0 103831 0
[pid=25128/tid=25135] ppid=25126 vsize=428052 CPUtime=0 cores=0-7
/proc/25128/task/25135/stat : 25135 (python) S 25126 25128 25102 0 -1 4202560 24 934 0 0 0 0 0 0 18 0 9 0 668099781 438325248 7598 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 1 0 0 0
[pid=25128/tid=25138] ppid=25126 vsize=428052 CPUtime=0 cores=0-7
/proc/25128/task/25138/stat : 25138 (python) S 25126 25128 25102 0 -1 4202560 23 934 0 0 0 0 0 0 18 0 9 0 668099784 438325248 7598 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 6 0 0 0
[pid=25128/tid=25141] ppid=25126 vsize=428052 CPUtime=0 cores=0-7
/proc/25128/task/25141/stat : 25141 (python) S 25126 25128 25102 0 -1 4202560 22 934 0 0 0 0 0 0 18 0 9 0 668099788 438325248 7598 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 4 0 0 0
[pid=25128/tid=25144] ppid=25126 vsize=428052 CPUtime=0 cores=0-7
/proc/25128/task/25144/stat : 25144 (python) S 25126 25128 25102 0 -1 4202560 20 934 0 0 0 0 0 0 20 0 9 0 668099791 438325248 7598 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 6 0 0 0
[pid=25128/tid=25147] ppid=25126 vsize=428052 CPUtime=0 cores=0-7
/proc/25128/task/25147/stat : 25147 (python) S 25126 25128 25102 0 -1 4202560 26 934 0 0 0 0 0 0 18 0 9 0 668099794 438325248 7598 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 4 0 0 0
[pid=25128/tid=25150] ppid=25126 vsize=428052 CPUtime=0 cores=0-7
/proc/25128/task/25150/stat : 25150 (python) S 25126 25128 25102 0 -1 4202560 24 934 0 0 0 0 0 0 20 0 9 0 668099798 438325248 7598 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 0 0 0 0
[pid=25128/tid=25153] ppid=25126 vsize=428052 CPUtime=0 cores=0-7
/proc/25128/task/25153/stat : 25153 (python) S 25126 25128 25102 0 -1 4202560 27 934 0 0 0 0 0 0 21 0 9 0 668099801 438325248 7598 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 1 0 0 0
[pid=25128/tid=25156] ppid=25126 vsize=428052 CPUtime=0 cores=0-7
/proc/25128/task/25156/stat : 25156 (python) S 25126 25128 25102 0 -1 4202560 31 934 0 0 0 0 0 0 22 0 9 0 668099804 438325248 7598 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 33.69
Current children cumulated vsize (KiB) 8408504

[startup+12.7011 s]
/proc/loadavg: 1.30 0.28 0.09 9/325 25158
/proc/meminfo: memFree=15453280/32951124 swapFree=52307772/67111528
[pid=25128] ppid=25126 vsize=428052 CPUtime=2.06 cores=0-7
/proc/25128/stat : 25128 (python) S 25126 25128 25102 0 -1 4202496 18656 934 1 0 192 14 0 0 21 0 9 0 668099561 438325248 7598 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382783947 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/25128/statm: 107013 7598 1891 364 0 42910 0
[pid=25134] ppid=25128 vsize=280692 CPUtime=0.01 cores=0-7
/proc/25134/stat : 25134 (python) S 25128 25128 25102 0 -1 4202560 899 0 0 0 1 0 0 0 18 0 1 0 668099781 287428608 5984 33554432000 4194304 5683644 140734980559984 18446744073709551615 218370977295 0 0 16781312 514 8799400353184317696 0 0 17 0 0 0 0
/proc/25134/statm: 70173 5984 378 364 0 6091 0
[pid=25136] ppid=25134 vsize=1019708 CPUtime=10.49 cores=0-7
/proc/25136/stat : 25136 (gnovelty+2) R 25134 25136 25136 0 -1 4202496 953 0 0 0 1049 0 0 0 25 0 1 0 668099781 1044180992 690 33554432000 134512640 135663704 4291163824 18446744073709551615 134525286 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/25136/statm: 254927 690 113 282 0 254643 0
[pid=25137] ppid=25128 vsize=357092 CPUtime=0.02 cores=0-7
/proc/25137/stat : 25137 (python) S 25128 25128 25102 0 -1 4202560 908 0 0 0 1 1 0 0 18 0 1 0 668099784 365662208 6001 33554432000 4194304 5683644 140734980559984 18446744073709551615 218370977350 0 0 16781312 514 8799400353184317700 0 0 17 6 0 0 0
/proc/25137/statm: 89273 6001 372 364 0 25188 0
[pid=25139] ppid=25137 vsize=1019708 CPUtime=10.46 cores=0-7
/proc/25139/stat : 25139 (gnovelty+2) R 25137 25139 25139 0 -1 4202496 955 0 0 0 1046 0 0 0 25 0 1 0 668099784 1044180992 690 33554432000 134512640 135663704 4288312816 18446744073709551615 134524897 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/25139/statm: 254927 690 113 282 0 254643 0
[pid=25140] ppid=25128 vsize=366876 CPUtime=0.02 cores=0-7
/proc/25140/stat : 25140 (python) S 25128 25128 25102 0 -1 4202560 898 0 0 0 1 1 0 0 18 0 1 0 668099787 375681024 6005 33554432000 4194304 5683644 140734980559984 18446744073709551615 218370977350 0 0 16781312 514 8799400353184317700 0 0 17 4 0 0 0
/proc/25140/statm: 91719 6005 372 364 0 27631 0
[pid=25142] ppid=25140 vsize=420220 CPUtime=10.31 cores=0-7
/proc/25142/stat : 25142 (adaptg2wsat2009) R 25140 25142 25142 0 -1 4202496 1668 0 0 0 1031 0 0 0 25 0 1 0 668099788 430305280 1399 33554432000 134512640 135107411 4291083184 18446744073709551615 134531830 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/25142/statm: 105055 1399 59 146 0 103931 0
[pid=25143] ppid=25128 vsize=377132 CPUtime=0.02 cores=0-7
/proc/25143/stat : 25143 (python) S 25128 25128 25102 0 -1 4202560 917 0 0 0 1 1 0 0 18 0 1 0 668099791 386183168 6026 33554432000 4194304 5683644 140734980559984 18446744073709551615 218370977350 0 0 16781312 514 8799400353184317699 0 0 17 6 0 0 0
/proc/25143/statm: 94283 6026 372 364 0 30192 0
[pid=25145] ppid=25143 vsize=420220 CPUtime=10.38 cores=0-7
/proc/25145/stat : 25145 (adaptg2wsat2009) R 25143 25145 25145 0 -1 4202496 1667 0 0 0 1038 0 0 0 25 0 1 0 668099791 430305280 1399 33554432000 134512640 135107411 4294214864 18446744073709551615 134531239 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/25145/statm: 105055 1399 59 146 0 103931 0
[pid=25146] ppid=25128 vsize=387456 CPUtime=0.02 cores=0-7
/proc/25146/stat : 25146 (python) S 25128 25128 25102 0 -1 4202560 939 0 0 0 1 1 0 0 18 0 1 0 668099794 396754944 6030 33554432000 4194304 5683644 140734980559984 18446744073709551615 218370977350 0 0 16781312 514 8799400353184317697 0 0 17 4 0 0 0
/proc/25146/statm: 96864 6030 372 364 0 32770 0
[pid=25148] ppid=25146 vsize=420220 CPUtime=10.3 cores=0-7
/proc/25148/stat : 25148 (adaptg2wsat2009) R 25146 25148 25148 0 -1 4202496 1668 0 0 0 1030 0 0 0 25 0 1 0 668099794 430305280 1400 33554432000 134512640 135107411 4294872720 18446744073709551615 134532313 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/25148/statm: 105055 1400 59 146 0 103931 0
[pid=25149] ppid=25128 vsize=397780 CPUtime=0.02 cores=0-7
/proc/25149/stat : 25149 (python) S 25128 25128 25102 0 -1 4202560 946 0 0 0 1 1 0 0 18 0 1 0 668099797 407326720 6051 33554432000 4194304 5683644 140734980559984 18446744073709551615 218370977350 0 0 16781312 514 8799400353184317697 0 0 17 0 0 0 0
/proc/25149/statm: 99445 6051 372 364 0 35348 0
[pid=25151] ppid=25149 vsize=1019708 CPUtime=10.27 cores=0-7
/proc/25151/stat : 25151 (gnovelty+2) R 25149 25151 25151 0 -1 4202496 954 0 0 0 1027 0 0 0 25 0 1 0 668099798 1044180992 690 33554432000 134512640 135663704 4294670608 18446744073709551615 134525699 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/25151/statm: 254927 690 113 282 0 254643 0

################
# More data... #
################

/proc/25233/stat : 25233 (TNM) R 25231 25233 25233 0 -1 4202496 1519 0 0 0 86358 2 0 0 25 0 1 0 668139603 469348352 1239 33554432000 134512640 135008748 4288821792 18446744073709551615 134527773 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/25233/statm: 114587 1239 46 122 0 113487 0
[pid=25239] ppid=25128 vsize=562776 CPUtime=0.25 cores=0-7
/proc/25239/stat : 25239 (python) S 25128 25128 25102 0 -1 4202560 1234 0 0 0 13 12 0 0 19 0 2 0 668148849 576282624 6621 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382783947 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/25239/statm: 140694 6621 407 364 0 76549 0
[pid=25241] ppid=25239 vsize=1019708 CPUtime=98.99 cores=0-7
/proc/25241/stat : 25241 (gnovelty+2) T 25239 25241 25241 0 -1 4202496 973 0 0 0 9899 0 0 0 25 0 1 0 668148850 1044180992 690 33554432000 134512640 135663704 4292931216 18446744073709551615 134545670 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/25241/statm: 254927 690 113 282 0 254643 0
[pid=25239/tid=25249] ppid=25128 vsize=562776 CPUtime=0 cores=0-7
/proc/25239/task/25249/stat : 25249 (python) S 25128 25128 25102 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 668158780 576282624 6621 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 514 18446604468983412928 0 0 -1 4 0 0 0
[pid=25244] ppid=25128 vsize=573048 CPUtime=0.25 cores=0-7
/proc/25244/stat : 25244 (python) S 25128 25128 25102 0 -1 4202560 1277 0 0 0 14 11 0 0 19 0 2 0 668149846 586801152 6674 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382783947 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/25244/statm: 143262 6674 407 364 0 79114 0
[pid=25246] ppid=25244 vsize=335996 CPUtime=99.25 cores=0-7
/proc/25246/stat : 25246 (iPAWS) T 25244 25246 25246 0 -1 4202496 1823 0 0 0 9925 0 0 0 25 0 1 0 668149846 344059904 1534 33554432000 134512640 135114423 4288771360 18446744073709551615 134515637 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/25246/statm: 83999 1534 59 147 0 83851 0
[pid=25244/tid=25251] ppid=25128 vsize=573048 CPUtime=0 cores=0-7
/proc/25244/task/25251/stat : 25251 (python) S 25128 25128 25102 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 668159772 586801152 6674 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 514 18446604465031349248 0 0 -1 6 0 0 0
[pid=25255] ppid=25128 vsize=583316 CPUtime=0.24 cores=0-7
/proc/25255/stat : 25255 (python) S 25128 25128 25102 0 -1 4202560 1271 0 0 0 12 12 0 0 19 0 2 0 668168419 597315584 6679 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382783947 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/25255/statm: 145829 6679 408 364 0 81678 0
[pid=25257] ppid=25255 vsize=335996 CPUtime=99.21 cores=0-7
/proc/25257/stat : 25257 (iPAWS) T 25255 25257 25257 0 -1 4202496 1819 0 0 0 9921 0 0 0 25 0 1 0 668168419 344059904 1535 33554432000 134512640 135114423 4287763184 18446744073709551615 134515385 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/25257/statm: 83999 1535 59 147 0 83851 0
[pid=25255/tid=25266] ppid=25128 vsize=583316 CPUtime=0 cores=0-7
/proc/25255/task/25266/stat : 25266 (python) S 25128 25128 25102 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 668178345 597315584 6679 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 514 18446604468773302528 0 0 -1 3 0 0 0
[pid=25258] ppid=25128 vsize=593612 CPUtime=1.31 cores=0-7
/proc/25258/stat : 25258 (python) S 25128 25128 25102 0 -1 4202560 1295 0 0 0 70 61 0 0 18 0 2 0 668169012 607858688 6676 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382783947 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/25258/statm: 148403 6676 401 364 0 84249 0
[pid=25260] ppid=25258 vsize=268428 CPUtime=488.46 cores=0-7
/proc/25260/stat : 25260 (hybridGM3) T 25258 25260 25260 0 -1 4202496 5416 0 0 0 48846 0 0 0 25 0 1 0 668169013 274870272 5127 33554432000 134512640 135204385 4290983776 18446744073709551615 134538284 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/25260/statm: 67107 5127 63 169 0 66937 0
[pid=25258/tid=25273] ppid=25128 vsize=593612 CPUtime=0 cores=0-7
/proc/25258/task/25273/stat : 25273 (python) S 25128 25128 25102 0 -1 4202560 47 0 0 0 0 0 0 0 18 0 2 0 668188878 607858688 6676 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 514 18446604468773304320 0 0 -1 2 0 0 0
[pid=25261] ppid=25128 vsize=603676 CPUtime=1.47 cores=0-7
/proc/25261/stat : 25261 (python) S 25128 25128 25102 0 -1 4202560 970 0 0 0 73 74 0 0 18 0 1 0 668169709 618164224 6750 33554432000 4194304 5683644 140734980559984 18446744073709551615 218370977350 0 0 16781312 514 8799400353184317701 0 0 17 1 0 0 0
/proc/25261/statm: 150919 6750 372 364 0 86762 0
[pid=25263] ppid=25261 vsize=458348 CPUtime=563.88 cores=0-7
/proc/25263/stat : 25263 (TNM) R 25261 25263 25263 0 -1 4202496 1519 0 0 0 56388 0 0 0 25 0 1 0 668169710 469348352 1240 33554432000 134512640 135008748 4291538608 18446744073709551615 134529338 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/25263/statm: 114587 1240 46 122 0 113487 0
[pid=25267] ppid=25128 vsize=613932 CPUtime=1.05 cores=0-7
/proc/25267/stat : 25267 (python) S 25128 25128 25102 0 -1 4202560 1189 0 0 0 52 53 0 0 18 0 2 0 668178351 628666368 6787 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382783947 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/25267/statm: 153483 6787 400 364 0 89323 0
[pid=25269] ppid=25267 vsize=249872 CPUtime=389.22 cores=0-7
/proc/25269/stat : 25269 (FH) T 25267 25269 25269 0 -1 4202496 1547 0 0 0 38922 0 0 0 25 0 1 0 668178351 255868928 1264 33554432000 134512640 134936988 4291982256 18446744073709551615 134553579 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/25269/statm: 62468 1264 41 104 0 61972 0
[pid=25267/tid=25274] ppid=25128 vsize=613932 CPUtime=0 cores=0-7
/proc/25267/task/25274/stat : 25274 (python) S 25128 25128 25102 0 -1 4202560 92 0 0 0 0 0 0 0 18 0 2 0 668188978 628666368 6787 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 514 18446604468477518080 0 0 -1 1 0 0 0
[pid=25275] ppid=25128 vsize=624188 CPUtime=1 cores=0-7
/proc/25275/stat : 25275 (python) S 25128 25128 25102 0 -1 4202560 999 0 0 0 53 47 0 0 18 0 1 0 668188984 639168512 6842 33554432000 4194304 5683644 140734980559984 18446744073709551615 218370977350 0 0 16781312 514 8799400353184317703 0 0 17 5 0 0 0
/proc/25275/statm: 156047 6842 372 364 0 91884 0
[pid=25277] ppid=25275 vsize=419776 CPUtime=369.45 cores=0-7
/proc/25277/stat : 25277 (NCVWr) R 25275 25277 25277 0 -1 4202496 1564 0 0 0 36945 0 0 0 25 0 1 0 668188984 429850624 1282 33554432000 134512640 135113558 4293922848 18446744073709551615 134534230 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/25277/statm: 104944 1282 58 147 0 103819 0
[pid=25280] ppid=25128 vsize=634444 CPUtime=0.28 cores=0-7
/proc/25280/stat : 25280 (python) S 25128 25128 25102 0 -1 4202560 1248 0 0 0 15 13 0 0 19 0 2 0 668197690 649670656 6887 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382783947 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/25280/statm: 158611 6887 406 364 0 94445 0
[pid=25282] ppid=25280 vsize=1019708 CPUtime=100 cores=0-7
/proc/25282/stat : 25282 (gnovelty+2) T 25280 25282 25282 0 -1 4202496 979 0 0 0 10000 0 0 0 25 0 1 0 668197691 1044180992 689 33554432000 134512640 135663704 4287681104 18446744073709551615 134525070 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/25282/statm: 254927 689 113 282 0 254643 0
[pid=25280/tid=25286] ppid=25128 vsize=634444 CPUtime=0 cores=0-7
/proc/25280/task/25286/stat : 25286 (python) S 25128 25128 25102 0 -1 4202560 92 0 0 0 0 0 0 0 19 0 2 0 668208124 649670656 6887 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 514 18446604468010721344 0 0 -1 0 0 0 0
[pid=25287] ppid=25128 vsize=644700 CPUtime=0.27 cores=0-7
/proc/25287/stat : 25287 (python) S 25128 25128 25102 0 -1 4202560 1249 0 0 0 13 14 0 0 19 0 2 0 668208130 660172800 6897 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382783947 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/25287/statm: 161175 6897 406 364 0 97006 0
[pid=25289] ppid=25287 vsize=335996 CPUtime=98.1 cores=0-7
/proc/25289/stat : 25289 (iPAWS) T 25287 25289 25289 0 -1 4202496 1826 0 0 0 9810 0 0 0 25 0 1 0 668208131 344059904 1537 33554432000 134512640 135114423 4289137072 18446744073709551615 134514777 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/25289/statm: 83999 1537 59 147 0 83851 0
[pid=25287/tid=25296] ppid=25128 vsize=644700 CPUtime=0 cores=0-7
/proc/25287/task/25296/stat : 25296 (python) S 25128 25128 25102 0 -1 4202560 92 0 0 0 0 0 0 0 19 0 2 0 668217959 660172800 6897 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 514 18446604471136814400 0 0 -1 2 0 0 0
[pid=25290] ppid=25128 vsize=654956 CPUtime=0.49 cores=0-7
/proc/25290/stat : 25290 (python) S 25128 25128 25102 0 -1 4202560 1053 0 0 0 27 22 0 0 18 0 1 0 668208508 670674944 6965 33554432000 4194304 5683644 140734980559984 18446744073709551615 218370977350 0 0 16781312 514 8799400353184317703 0 0 17 7 0 0 0
/proc/25290/statm: 163739 6965 379 364 0 99567 0
[pid=25292] ppid=25290 vsize=268420 CPUtime=175.72 cores=0-7
/proc/25292/stat : 25292 (hybridGM3) R 25290 25292 25292 0 -1 4202496 5415 0 0 0 17571 1 0 0 25 0 1 0 668208509 274862080 5125 33554432000 134512640 135204385 4288568800 18446744073709551615 134545437 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/25292/statm: 67105 5125 63 169 0 66935 0
[pid=25297] ppid=25128 vsize=665212 CPUtime=0.22 cores=0-7
/proc/25297/stat : 25297 (python) S 25128 25128 25102 0 -1 4202560 1059 0 0 0 12 10 0 0 18 0 1 0 668218075 681177088 6888 33554432000 4194304 5683644 140734980559984 18446744073709551615 218370977350 0 0 16781312 514 8799400353184317703 0 0 17 7 0 0 0
/proc/25297/statm: 166303 6888 379 364 0 102128 0
[pid=25299] ppid=25297 vsize=268420 CPUtime=79.7 cores=0-7
/proc/25299/stat : 25299 (hybridGM3) R 25297 25299 25299 0 -1 4202496 5415 0 0 0 7969 1 0 0 25 0 1 0 668218075 274862080 5126 33554432000 134512640 135204385 4291740368 18446744073709551615 134538295 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/25299/statm: 67105 5126 63 169 0 66935 0
[pid=25302] ppid=25128 vsize=675468 CPUtime=0 cores=0-7
/proc/25302/stat : 25302 (python) R 25128 25128 25102 0 -1 4202560 978 0 0 0 0 0 0 0 18 0 1 0 668226394 691679232 6965 33554432000 4194304 5683644 140734980559984 18446744073709551615 218370977350 0 0 16781312 514 0 0 0 17 7 0 0 0
/proc/25302/statm: 168867 6965 356 364 0 104689 0
[pid=25304] ppid=25302 vsize=268420 CPUtime=1.01 cores=0-7
/proc/25304/stat : 25304 (hybridGM3) R 25302 25304 25304 0 -1 4202496 5419 0 0 0 100 1 0 0 24 0 1 0 668226394 274862080 5126 33554432000 134512640 135204385 4292391536 18446744073709551615 134538264 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/25304/statm: 67105 5126 63 169 0 66935 0
[pid=25128/tid=25135] ppid=25126 vsize=685712 CPUtime=0 cores=0-7
/proc/25128/task/25135/stat : 25135 (python) S 25126 25128 25102 0 -1 4202560 24 934 0 0 0 0 0 0 18 0 34 0 668099781 702169088 8555 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 1 0 0 0
[pid=25128/tid=25138] ppid=25126 vsize=685712 CPUtime=0 cores=0-7
/proc/25128/task/25138/stat : 25138 (python) S 25126 25128 25102 0 -1 4202560 23 934 0 0 0 0 0 0 18 0 34 0 668099784 702169088 8555 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 6 0 0 0
[pid=25128/tid=25141] ppid=25126 vsize=685712 CPUtime=0 cores=0-7
/proc/25128/task/25141/stat : 25141 (python) S 25126 25128 25102 0 -1 4202560 32 934 0 0 0 0 0 0 18 0 34 0 668099788 702169088 8555 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 4 0 0 0
[pid=25128/tid=25144] ppid=25126 vsize=685712 CPUtime=0 cores=0-7
/proc/25128/task/25144/stat : 25144 (python) S 25126 25128 25102 0 -1 4202560 25 934 0 0 0 0 0 0 18 0 34 0 668099791 702169088 8555 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 6 0 0 0
[pid=25128/tid=25147] ppid=25126 vsize=685712 CPUtime=0 cores=0-7
/proc/25128/task/25147/stat : 25147 (python) S 25126 25128 25102 0 -1 4202560 29 934 0 0 0 0 0 0 18 0 34 0 668099794 702169088 8555 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 4 0 0 0
[pid=25128/tid=25150] ppid=25126 vsize=685712 CPUtime=0 cores=0-7
/proc/25128/task/25150/stat : 25150 (python) S 25126 25128 25102 0 -1 4202560 24 934 0 0 0 0 0 0 20 0 34 0 668099798 702169088 8555 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 0 0 0 0
[pid=25128/tid=25153] ppid=25126 vsize=685712 CPUtime=0 cores=0-7
/proc/25128/task/25153/stat : 25153 (python) S 25126 25128 25102 0 -1 4202560 32 934 0 0 0 0 0 0 18 0 34 0 668099801 702169088 8555 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 1 0 0 0
[pid=25128/tid=25156] ppid=25126 vsize=685712 CPUtime=0 cores=0-7
/proc/25128/task/25156/stat : 25156 (python) S 25126 25128 25102 0 -1 4202560 44 934 0 0 0 0 0 0 18 0 34 0 668099804 702169088 8555 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 1 0 0 0
[pid=25128/tid=25165] ppid=25126 vsize=685712 CPUtime=0 cores=0-7
/proc/25128/task/25165/stat : 25165 (python) S 25126 25128 25102 0 -1 4202560 32 934 0 0 0 0 0 0 18 0 34 0 668109811 702169088 8555 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 6 0 0 0
[pid=25128/tid=25169] ppid=25126 vsize=685712 CPUtime=0 cores=0-7
/proc/25128/task/25169/stat : 25169 (python) S 25126 25128 25102 0 -1 4202560 39 934 0 0 0 0 0 0 18 0 34 0 668109815 702169088 8555 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 7 0 0 0
[pid=25128/tid=25173] ppid=25126 vsize=685712 CPUtime=0 cores=0-7
/proc/25128/task/25173/stat : 25173 (python) S 25126 25128 25102 0 -1 4202560 35 934 0 0 0 0 0 0 18 0 34 0 668109819 702169088 8555 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 6 0 0 0
[pid=25128/tid=25176] ppid=25126 vsize=685712 CPUtime=0 cores=0-7
/proc/25128/task/25176/stat : 25176 (python) S 25126 25128 25102 0 -1 4202560 23 934 0 0 0 0 0 0 25 0 34 0 668109823 702169088 8555 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 0 0 0 0
[pid=25128/tid=25181] ppid=25126 vsize=685712 CPUtime=0 cores=0-7
/proc/25128/task/25181/stat : 25181 (python) S 25126 25128 25102 0 -1 4202560 31 934 0 0 0 0 0 0 18 0 34 0 668109827 702169088 8555 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 7 0 0 0
[pid=25128/tid=25184] ppid=25126 vsize=685712 CPUtime=0 cores=0-7
/proc/25128/task/25184/stat : 25184 (python) S 25126 25128 25102 0 -1 4202560 105 934 0 0 0 0 0 0 17 0 34 0 668109832 702169088 8555 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 4 0 0 0
[pid=25128/tid=25188] ppid=25126 vsize=685712 CPUtime=0 cores=0-7
/proc/25128/task/25188/stat : 25188 (python) S 25126 25128 25102 0 -1 4202560 47 934 0 0 0 0 0 0 18 0 34 0 668109836 702169088 8555 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 1 0 0 0
[pid=25128/tid=25191] ppid=25126 vsize=685712 CPUtime=0 cores=0-7
/proc/25128/task/25191/stat : 25191 (python) S 25126 25128 25102 0 -1 4202560 37 934 0 0 0 0 0 0 18 0 34 0 668109841 702169088 8555 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 0 0 0 0
[pid=25128/tid=25199] ppid=25126 vsize=685712 CPUtime=0 cores=0-7
/proc/25128/task/25199/stat : 25199 (python) S 25126 25128 25102 0 -1 4202560 23 934 0 0 0 0 0 0 19 0 34 0 668119839 702169088 8555 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 0 0 0 0
[pid=25128/tid=25202] ppid=25126 vsize=685712 CPUtime=0 cores=0-7
/proc/25128/task/25202/stat : 25202 (python) S 25126 25128 25102 0 -1 4202560 29 934 0 0 0 0 0 0 21 0 34 0 668119845 702169088 8555 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 0 0 0 0
[pid=25128/tid=25223] ppid=25126 vsize=685712 CPUtime=0 cores=0-7
/proc/25128/task/25223/stat : 25223 (python) S 25126 25128 25102 0 -1 4202560 47 934 0 0 0 0 0 0 18 0 34 0 668130186 702169088 8555 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 0 0 0 0
[pid=25128/tid=25229] ppid=25126 vsize=685712 CPUtime=0 cores=0-7
/proc/25128/task/25229/stat : 25229 (python) S 25126 25128 25102 0 -1 4202560 34 934 0 0 0 0 0 0 18 0 34 0 668139223 702169088 8555 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 0 0 0 0
[pid=25128/tid=25232] ppid=25126 vsize=685712 CPUtime=0 cores=0-7
/proc/25128/task/25232/stat : 25232 (python) S 25126 25128 25102 0 -1 4202560 33 934 0 0 0 0 0 0 20 0 34 0 668139603 702169088 8555 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 0 0 0 0
[pid=25128/tid=25240] ppid=25126 vsize=685712 CPUtime=0 cores=0-7
/proc/25128/task/25240/stat : 25240 (python) S 25126 25128 25102 0 -1 4202560 29 934 0 0 0 0 0 0 20 0 34 0 668148849 702169088 8555 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 7 0 0 0
[pid=25128/tid=25245] ppid=25126 vsize=685712 CPUtime=0 cores=0-7
/proc/25128/task/25245/stat : 25245 (python) S 25126 25128 25102 0 -1 4202560 27 934 0 0 0 0 0 0 20 0 34 0 668149846 702169088 8555 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 7 0 0 0
[pid=25128/tid=25256] ppid=25126 vsize=685712 CPUtime=0 cores=0-7
/proc/25128/task/25256/stat : 25256 (python) S 25126 25128 25102 0 -1 4202560 29 934 0 0 0 0 0 0 20 0 34 0 668168419 702169088 8555 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 7 0 0 0
[pid=25128/tid=25259] ppid=25126 vsize=685712 CPUtime=0 cores=0-7
/proc/25128/task/25259/stat : 25259 (python) S 25126 25128 25102 0 -1 4202560 40 934 0 0 0 0 0 0 18 0 34 0 668169013 702169088 8555 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 2 0 0 0
[pid=25128/tid=25262] ppid=25126 vsize=685712 CPUtime=0 cores=0-7
/proc/25128/task/25262/stat : 25262 (python) S 25126 25128 25102 0 -1 4202560 152 934 0 0 0 0 0 0 20 0 34 0 668169710 702169088 8555 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 7 0 0 0
[pid=25128/tid=25268] ppid=25126 vsize=685712 CPUtime=0 cores=0-7
/proc/25128/task/25268/stat : 25268 (python) S 25126 25128 25102 0 -1 4202560 43 934 0 0 0 0 0 0 18 0 34 0 668178351 702169088 8555 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 0 0 0 0
[pid=25128/tid=25276] ppid=25126 vsize=685712 CPUtime=0 cores=0-7
/proc/25128/task/25276/stat : 25276 (python) S 25126 25128 25102 0 -1 4202560 46 934 0 0 0 0 0 0 20 0 34 0 668188984 702169088 8555 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 0 0 0 0
[pid=25128/tid=25281] ppid=25126 vsize=685712 CPUtime=0 cores=0-7
/proc/25128/task/25281/stat : 25281 (python) S 25126 25128 25102 0 -1 4202560 35 934 0 0 0 0 0 0 20 0 34 0 668197690 702169088 8555 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 2 0 0 0
[pid=25128/tid=25288] ppid=25126 vsize=685712 CPUtime=0 cores=0-7
/proc/25128/task/25288/stat : 25288 (python) S 25126 25128 25102 0 -1 4202560 38 934 0 0 0 0 0 0 20 0 34 0 668208130 702169088 8555 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 7 0 0 0
[pid=25128/tid=25291] ppid=25126 vsize=685712 CPUtime=0 cores=0-7
/proc/25128/task/25291/stat : 25291 (python) S 25126 25128 25102 0 -1 4202560 34 934 0 0 0 0 0 0 20 0 34 0 668208509 702169088 8555 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 7 0 0 0
[pid=25128/tid=25298] ppid=25126 vsize=685712 CPUtime=0 cores=0-7
/proc/25128/task/25298/stat : 25298 (python) S 25126 25128 25102 0 -1 4202560 37 934 0 0 0 0 0 0 20 0 34 0 668218075 702169088 8555 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 0 0 0 0
[pid=25128/tid=25303] ppid=25126 vsize=685712 CPUtime=0 cores=0-7
/proc/25128/task/25303/stat : 25303 (python) S 25126 25128 25102 0 -1 4202560 36 934 0 0 0 0 0 0 20 0 34 0 668226394 702169088 8555 33554432000 4194304 5683644 140734980559984 18446744073709551615 218382780849 0 0 16781312 2 18446604453985335104 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 10093.6
Current children cumulated vsize (KiB) 32619688

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 25128 and gives
#  childrusage.ru_utime.tv_sec=292
#  childrusage.ru_utime.tv_usec=787489
#  childrusage.ru_stime.tv_sec=3
#  childrusage.ru_stime.tv_usec=696438
# CPU time returned by wait4() is 296.484
# while last known CPU time is 10093.6
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1269.61
CPU time (s): 10093.6
CPU user time (s): 10081
CPU system time (s): 12.64
CPU usage (%): 795.018
Max. virtual memory (cumulated for all children) (KiB): 32619688

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 292.787
system time used= 3.69644
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 77866
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 7081
involuntary context switches= 3052

runsolver used 13.32 second user time and 29.5725 second system time

The end

Launcher Data 

Begin job on node101 at 2011-05-16 18:56:08
IDJOB=3385286
IDBENCH=82551
IDSOLVER=1870
FILE ID=node101/3385286-1305564967
RUNJOBID= node101-1305564967-25116
PBS_JOBID= 13350149
Free space on /tmp= 73748 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/large/unif-k7-r85-v300-c25500-S966156467-066-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3385286-1305564967/watcher-3385286-1305564967 -o /tmp/evaluation-result-3385286-1305564967/solver-3385286-1305564967 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385286-1305564967.cnf -seed 592305601 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 5297bd1d2a420892cfb1fb6d32ccacad
RANDOM SEED=592305601

node101.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      15548032 kB
Buffers:        352216 kB
Cached:        6523960 kB
SwapCached:      14844 kB
Active:       11699408 kB
Inactive:      5413044 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      15548032 kB
SwapTotal:    67111528 kB
SwapFree:     52307772 kB
Dirty:           32628 kB
Writeback:           0 kB
AnonPages:    10233148 kB
Mapped:          15436 kB
Slab:           160644 kB
PageTables:      63624 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 42582144 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73716 MiB
End job on node101 at 2011-05-16 19:17:20