Trace number 3385278

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)? (MO) 10310.7 1298.12

General information on the benchmark

Namerandom/large/
unif-k5-r20-v2000-c40000-S319509287-085-UNKNOWN.cnf
MD5SUMbb548a1e884feefbb349d3449e849f5e
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark3906.44
Satisfiable
(Un)Satisfiability was proved
Number of variables2000
Number of clauses40000
Sum of the clauses size200000
Maximum clause length5
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 540000
Number of clauses of size over 50

Solver Data 

0.41/0.82	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.41/0.82	c INFO: solving HOME/instance-3385278-1305564966.cnf
0.41/0.82	c INFO: computing task features
0.62/1.03	c INFO: computed variable-clause graph statistics
0.62/1.03	c INFO: computed clause balance statistics
1.60/2.01	c INFO: computed variable balance statistics
2.10/2.56	c INFO: computed small-clause counts
2.10/2.57	c INFO: computed Horn-clause counts
2.20/2.61	c INFO: computed variable graph statistics
2.20/2.61	c INFO: collected features for HOME/instance-3385278-1305564966.cnf in 1.78 s
2.20/2.64	c INFO: running gnovelty+2@0 for 97 with 4998 remaining (b = 0.38)
2.20/2.67	c INFO: running gnovelty+2@0 for 97 with 4998 remaining (b = 0.13)
2.30/2.70	c INFO: running adaptg2wsat2009++@0 for 97 with 4998 remaining (b = 0.08)
2.30/2.73	c INFO: running adaptg2wsat2009++@0 for 97 with 4998 remaining (b = 0.07)
2.30/2.77	c INFO: running adaptg2wsat2009++@0 for 97 with 4998 remaining (b = 0.05)
2.40/2.80	c INFO: running gnovelty+2@0 for 97 with 4998 remaining (b = 0.05)
2.40/2.83	c INFO: running FH@0 for 97 with 4998 remaining (b = 0.03)
2.40/2.86	c INFO: running adaptg2wsat2009++@0 for 97 with 4997 remaining (b = 0.03)
802.64/103.04	c INFO: running adaptg2wsat2009++@0 for 97 with 4897 remaining (b = 0.02)
802.64/103.07	c INFO: running FH@0 for 97 with 4897 remaining (b = 0.02)
803.26/103.11	c INFO: running hybridGM3@0 for 194 with 4897 remaining (b = 0.04)
803.26/103.16	c INFO: running gnovelty+2@0 for 97 with 4897 remaining (b = 0.02)
803.65/103.20	c INFO: running hybridGM3@0 for 194 with 4897 remaining (b = 0.03)
803.65/103.30	c INFO: running adaptg2wsat2009++@0 for 97 with 4897 remaining (b = 0.01)
804.74/103.35	c INFO: running hybridGM3@0 for 194 with 4897 remaining (b = 0.03)
804.74/103.40	c INFO: running hybridGM3@0 for 194 with 4897 remaining (b = 0.02)
1602.66/203.42	c INFO: running TNM@0 for 1552 with 4797 remaining (b = 0.19)
1602.66/203.47	c INFO: running FH@0 for 97 with 4797 remaining (b = 0.01)
1603.81/203.65	c INFO: running adaptg2wsat2009++@0 for 97 with 4797 remaining (b = 0.01)
1603.81/203.70	c INFO: running adaptg2wsat2009++@0 for 97 with 4797 remaining (b = 0.01)
2370.39/299.89	c INFO: running hybridGM3@99 for 97 with 4700 remaining (b = 0.01)
2371.91/300.03	c INFO: running hybridGM3@99 for 97 with 4700 remaining (b = 0.01)
2371.91/300.08	c INFO: running adaptg2wsat2009++@0 for 97 with 4700 remaining (b = 0.01)
2372.66/300.13	c INFO: running hybridGM3@99 for 97 with 4700 remaining (b = 0.01)
2372.66/300.20	c INFO: running hybridGM3@100 for 97 with 4700 remaining (b = 0.01)
2402.09/303.86	c INFO: running adaptg2wsat2009++@100 for 97 with 4696 remaining (b = 0.01)
2420.41/306.13	c INFO: running hybridGM3@0 for 194 with 4694 remaining (b = 0.02)
3133.38/395.61	c INFO: running hybridGM3@0 for 194 with 4605 remaining (b = 0.02)
3147.72/397.46	c INFO: running TNM@0 for 1455 with 4603 remaining (b = 0.14)
3177.98/401.24	c INFO: running adaptg2wsat2009++@98 for 97 with 4599 remaining (b = 0.01)
3177.98/401.30	c INFO: running hybridGM3@295 for 97 with 4599 remaining (b = 0.01)
3178.73/401.41	c INFO: running iPAWS@0 for 97 with 4599 remaining (b = 0.01)
3180.30/401.53	c INFO: running hybridGM3@295 for 97 with 4599 remaining (b = 0.01)
3937.71/496.66	c INFO: running gnovelty+2@0 for 97 with 4504 remaining (b = 0.01)
3939.15/496.89	c INFO: running hybridGM3@387 for 97 with 4503 remaining (b = 0.01)
3953.57/498.64	c INFO: running hybridGM3@392 for 97 with 4502 remaining (b = 0.01)
3978.23/501.80	c INFO: running FH@0 for 291 with 4499 remaining (b = 0.02)
3987.78/502.91	c INFO: running hybridGM3@95 for 97 with 4497 remaining (b = 0.01)
4699.53/592.39	c INFO: running hybridGM3@97 for 97 with 4408 remaining (b = 0.01)
4736.95/597.10	c INFO: running adaptg2wsat2009++@100 for 97 with 4403 remaining (b = 0.01)
4746.48/598.27	c INFO: running hybridGM3@484 for 97 with 4402 remaining (b = 0.01)
4792.61/604.04	c INFO: running hybridGM3@0 for 194 with 4396 remaining (b = 0.02)
4794.08/604.29	c INFO: running hybridGM3@292 for 97 with 4396 remaining (b = 0.01)
5458.53/687.75	c INFO: running TNM@0 for 1552 with 4313 remaining (b = 0.11)
5505.41/693.65	c INFO: running FH@100 for 291 with 4307 remaining (b = 0.02)
5512.56/694.51	c INFO: running iPAWS@0 for 97 with 4306 remaining (b = 0.01)
5553.24/699.67	c INFO: running hybridGM3@387 for 194 with 4301 remaining (b = 0.02)
6286.68/791.83	c INFO: running adaptg2wsat2009++@97 for 97 with 4209 remaining (b = 0.01)
6343.15/798.93	c INFO: running hybridGM3@100 for 97 with 4201 remaining (b = 0.01)
6359.07/800.97	c INFO: running FH@195 for 194 with 4199 remaining (b = 0.01)
7061.59/889.25	c INFO: running adaptg2wsat2009++@292 for 291 with 4111 remaining (b = 0.02)
7126.03/897.38	c INFO: running FH@0 for 97 with 4103 remaining (b = 0.01)
7149.90/900.37	c INFO: running iPAWS@0 for 97 with 4100 remaining (b = 0.01)
7812.52/983.69	c INFO: running hybridGM3@0 for 194 with 4017 remaining (b = 0.01)
7899.97/994.69	c INFO: running FH@292 for 97 with 4006 remaining (b = 0.01)
7940.57/999.81	c INFO: running NCVWr@0 for 485 with 4001 remaining (b = 0.03)
7965.19/1002.83	c INFO: running gnovelty+2@0 for 97 with 3998 remaining (b = 0.01)
8706.23/1096.09	c INFO: running hybridGM3@0 for 194 with 3904 remaining (b = 0.01)
8763.51/1103.30	c INFO: running FH@197 for 194 with 3897 remaining (b = 0.01)
9377.65/1180.51	c INFO: running hybridGM3@193 for 97 with 3820 remaining (b = 0.01)
9399.09/1183.28	c INFO: running adaptg2wsat2009++@486 for 97 with 3817 remaining (b = 0.01)
10157.18/1278.66	c INFO: running adaptg2wsat2009++@581 for 97 with 3722 remaining (b = 0.01)
10183.42/1281.91	c INFO: running hybridGM3@484 for 97 with 3718 remaining (b = 0.01)
10262.87/1291.93	c INFO: running hybridGM3@100 for 97 with 3708 remaining (b = 0.01)
10303.36/1297.03	c INFO: running hybridGM3@0 for 194 with 3703 remaining (b = 0.01)
10310.75/1298.09	c INFO: running TNM@0 for 1552 with 3702 remaining (b = 0.08)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3385278-1305564966/watcher-3385278-1305564966 -o /tmp/evaluation-result-3385278-1305564966/solver-3385278-1305564966 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385278-1305564966.cnf -seed 1960073158 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.00 0.00 0.00 1/412 31197
/proc/meminfo: memFree=18025388/32951124 swapFree=57414336/67111528
[pid=31197] ppid=31195 vsize=15780 CPUtime=0 cores=0-7
/proc/31197/stat : 31197 (python) S 31195 31197 31171 0 -1 4202496 894 0 0 0 0 0 0 0 17 0 1 0 746727164 16158720 777 33554432000 4194304 5683644 140736118916944 18446744073709551615 252458063872 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 0
/proc/31197/statm: 3945 777 343 364 0 432 0

[startup+0.0424899 s]
/proc/loadavg: 0.00 0.00 0.00 1/412 31197
/proc/meminfo: memFree=18025388/32951124 swapFree=57414336/67111528
[pid=31197] ppid=31195 vsize=16816 CPUtime=0.01 cores=0-7
/proc/31197/stat : 31197 (python) R 31195 31197 31171 0 -1 4202496 1226 0 0 0 1 0 0 0 17 0 1 0 746727164 17219584 1085 33554432000 4194304 5683644 140736118916944 18446744073709551615 4870108 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/31197/statm: 4204 1086 417 364 0 691 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 16816

[startup+0.100464 s]
/proc/loadavg: 0.00 0.00 0.00 1/412 31197
/proc/meminfo: memFree=18025388/32951124 swapFree=57414336/67111528
[pid=31197] ppid=31195 vsize=56784 CPUtime=0.05 cores=0-7
/proc/31197/stat : 31197 (python) R 31195 31197 31171 0 -1 4202496 2101 0 0 0 4 1 0 0 17 0 1 0 746727164 58146816 1857 33554432000 4194304 5683644 140736118916944 18446744073709551615 252457759710 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/31197/statm: 14196 1857 499 364 0 1319 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 56784

[startup+0.300455 s]
/proc/loadavg: 0.00 0.00 0.00 1/412 31197
/proc/meminfo: memFree=18025388/32951124 swapFree=57414336/67111528
[pid=31197] ppid=31195 vsize=135588 CPUtime=0.16 cores=0-7
/proc/31197/stat : 31197 (python) S 31195 31197 31171 0 -1 4202496 5890 934 1 0 12 4 0 0 17 0 1 0 746727164 138842112 4132 33554432000 4194304 5683644 140736118916944 18446744073709551615 252458063872 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 0
/proc/31197/statm: 33897 4132 1031 364 0 3083 0
Current children cumulated CPU time (s) 0.16
Current children cumulated vsize (KiB) 135588

[startup+0.700357 s]
/proc/loadavg: 0.00 0.00 0.00 1/412 31197
/proc/meminfo: memFree=18025388/32951124 swapFree=57414336/67111528
[pid=31197] ppid=31195 vsize=234164 CPUtime=0.34 cores=0-7
/proc/31197/stat : 31197 (python) S 31195 31197 31171 0 -1 4202496 7859 934 1 0 23 11 0 0 15 0 1 0 746727164 239783936 5771 33554432000 4194304 5683644 140736118916944 18446744073709551615 252458063872 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 0
/proc/31197/statm: 58541 5771 1590 364 0 4340 0
Current children cumulated CPU time (s) 0.34
Current children cumulated vsize (KiB) 234164

[startup+1.5002 s]
/proc/loadavg: 0.00 0.00 0.00 2/413 31202
/proc/meminfo: memFree=17992520/32951124 swapFree=57414336/67111528
[pid=31197] ppid=31195 vsize=290808 CPUtime=1.1 cores=0-7
/proc/31197/stat : 31197 (python) R 31195 31197 31171 0 -1 4202496 14007 934 1 0 97 13 0 0 16 0 1 0 746727164 297787392 10086 33554432000 4194304 5683644 140736118916944 18446744073709551615 47196347330246 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/31197/statm: 72702 10086 1826 364 0 8626 0
Current children cumulated CPU time (s) 1.1
Current children cumulated vsize (KiB) 290808

[startup+3.10086 s]
/proc/loadavg: 0.00 0.00 0.00 2/413 31202
/proc/meminfo: memFree=17990288/32951124 swapFree=57414336/67111528
[pid=31197] ppid=31195 vsize=372084 CPUtime=2.47 cores=0-7
/proc/31197/stat : 31197 (python) S 31195 31197 31171 0 -1 4202496 20354 934 1 0 231 16 0 0 20 0 9 0 746727164 381014016 9987 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469892555 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31197/statm: 93021 9987 1891 364 0 28918 0
Current children cumulated CPU time (s) 2.47
Current children cumulated vsize (KiB) 372084
heavy processes:

[startup+6.30127 s]
/proc/loadavg: 0.08 0.02 0.01 9/437 31226
/proc/meminfo: memFree=17899208/32951124 swapFree=57414336/67111528
[pid=31197] ppid=31195 vsize=372084 CPUtime=2.47 cores=0-7
/proc/31197/stat : 31197 (python) S 31195 31197 31171 0 -1 4202496 20354 934 1 0 231 16 0 0 20 0 9 0 746727164 381014016 9987 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469892555 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31197/statm: 93021 9987 1891 364 0 28918 0
[pid=31203] ppid=31197 vsize=290048 CPUtime=0 cores=0-7
/proc/31203/stat : 31203 (python) S 31197 31197 31171 0 -1 4202560 734 0 0 0 0 0 0 0 18 0 1 0 746727429 297009152 8417 33554432000 4194304 5683644 140736118916944 18446744073709551615 252458085903 0 0 16781312 514 978021369637732611 0 0 17 1 0 0 0
/proc/31203/statm: 72512 8417 353 364 0 8430 0
[pid=31205] ppid=31203 vsize=1020308 CPUtime=3.64 cores=0-7
/proc/31205/stat : 31205 (gnovelty+2) R 31203 31205 31205 0 -1 4202496 1154 0 0 0 364 0 0 0 25 0 1 0 746727429 1044795392 892 33554432000 134512640 135663704 4290839888 18446744073709551615 134525224 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/31205/statm: 255077 892 113 282 0 254793 0
[pid=31206] ppid=31197 vsize=300304 CPUtime=0 cores=0-7
/proc/31206/stat : 31206 (python) S 31197 31197 31171 0 -1 4202560 739 0 0 0 0 0 0 0 18 0 1 0 746727432 307511296 8415 33554432000 4194304 5683644 140736118916944 18446744073709551615 252458085958 0 0 16781312 514 978025217928429826 0 0 17 0 0 0 0
/proc/31206/statm: 75076 8415 347 364 0 10991 0
[pid=31208] ppid=31206 vsize=1020308 CPUtime=3.61 cores=0-7
/proc/31208/stat : 31208 (gnovelty+2) R 31206 31208 31208 0 -1 4202496 1155 0 0 0 361 0 0 0 25 0 1 0 746727432 1044795392 892 33554432000 134512640 135663704 4293791632 18446744073709551615 134525255 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/31208/statm: 255077 892 113 282 0 254793 0
[pid=31209] ppid=31197 vsize=310560 CPUtime=0 cores=0-7
/proc/31209/stat : 31209 (python) S 31197 31197 31171 0 -1 4202560 742 0 0 0 0 0 0 0 18 0 1 0 746727435 318013440 8419 33554432000 4194304 5683644 140736118916944 18446744073709551615 252458085958 0 0 16781312 514 11048640233217163521 0 0 17 2 0 0 0
/proc/31209/statm: 77640 8419 347 364 0 13552 0
[pid=31211] ppid=31209 vsize=423400 CPUtime=3.51 cores=0-7
/proc/31211/stat : 31211 (adaptg2wsat2009) R 31209 31211 31211 0 -1 4202496 2623 0 0 0 351 0 0 0 25 0 1 0 746727436 433561600 2357 33554432000 134512640 135107411 4288752832 18446744073709551615 134532074 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/31211/statm: 105850 2357 59 146 0 104726 0
[pid=31212] ppid=31197 vsize=320816 CPUtime=0 cores=0-7
/proc/31212/stat : 31212 (python) S 31197 31197 31171 0 -1 4202560 749 0 0 0 0 0 0 0 18 0 1 0 746727438 328515584 8423 33554432000 4194304 5683644 140736118916944 18446744073709551615 252458085958 0 0 16781312 514 16608528311886250243 0 0 17 2 0 0 0
/proc/31212/statm: 80204 8423 347 364 0 16113 0
[pid=31214] ppid=31212 vsize=423400 CPUtime=3.52 cores=0-7
/proc/31214/stat : 31214 (adaptg2wsat2009) R 31212 31214 31214 0 -1 4202496 2623 0 0 0 352 0 0 0 25 0 1 0 746727439 433561600 2357 33554432000 134512640 135107411 4294854864 18446744073709551615 134531600 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/31214/statm: 105850 2357 59 146 0 104726 0
[pid=31215] ppid=31197 vsize=331072 CPUtime=0 cores=0-7
/proc/31215/stat : 31215 (python) S 31197 31197 31171 0 -1 4202560 757 0 0 0 0 0 0 0 18 0 1 0 746727441 339017728 8427 33554432000 4194304 5683644 140736118916944 18446744073709551615 252458085958 0 0 16781312 514 978029066219127042 0 0 17 0 0 0 0
/proc/31215/statm: 82768 8427 347 364 0 18674 0
[pid=31217] ppid=31215 vsize=423400 CPUtime=3.51 cores=0-7
/proc/31217/stat : 31217 (adaptg2wsat2009) R 31215 31217 31217 0 -1 4202496 2630 0 0 0 351 0 0 0 25 0 1 0 746727442 433561600 2359 33554432000 134512640 135107411 4292223216 18446744073709551615 134532371 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/31217/statm: 105850 2359 59 146 0 104726 0
[pid=31218] ppid=31197 vsize=341328 CPUtime=0 cores=0-7
/proc/31218/stat : 31218 (python) S 31197 31197 31171 0 -1 4202560 753 0 0 0 0 0 0 0 18 0 1 0 746727445 349519872 8431 33554432000 4194304 5683644 140736118916944 18446744073709551615 252458085958 0 0 16781312 514 978032914509824257 0 0 17 6 0 0 0
/proc/31218/statm: 85332 8431 347 364 0 21235 0
[pid=31220] ppid=31218 vsize=1020308 CPUtime=3.48 cores=0-7
/proc/31220/stat : 31220 (gnovelty+2) R 31218 31220 31220 0 -1 4202496 1157 0 0 0 348 0 0 0 25 0 1 0 746727445 1044795392 892 33554432000 134512640 135663704 4292944896 18446744073709551615 134545644 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/31220/statm: 255077 892 113 282 0 254793 0
[pid=31221] ppid=31197 vsize=351584 CPUtime=0 cores=0-7
/proc/31221/stat : 31221 (python) S 31197 31197 31171 0 -1 4202560 765 0 0 0 0 0 0 0 18 0 1 0 746727448 360022016 8435 33554432000 4194304 5683644 140736118916944 18446744073709551615 252458085958 0 0 16781312 514 5588464939925733633 0 0 17 6 0 0 0
/proc/31221/statm: 87896 8435 347 364 0 23796 0
[pid=31223] ppid=31221 vsize=253028 CPUtime=3.42 cores=0-7
/proc/31223/stat : 31223 (FH) R 31221 31223 31223 0 -1 4202496 2500 0 0 0 342 0 0 0 25 0 1 0 746727448 259100672 2232 33554432000 134512640 134936988 4289700192 18446744073709551615 134552493 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/31223/statm: 63257 2232 41 104 0 62761 0
[pid=31224] ppid=31197 vsize=361840 CPUtime=0 cores=0-7
/proc/31224/stat : 31224 (python) S 31197 31197 31171 0 -1 4202560 798 0 0 0 0 0 0 0 18 0 1 0 746727451 370524160 8445 33554432000 4194304 5683644 140736118916944 18446744073709551615 252458085958 0 0 16781312 514 16608520615304855811 0 0 17 4 0 0 0
/proc/31224/statm: 90460 8445 353 364 0 26357 0
[pid=31226] ppid=31224 vsize=423400 CPUtime=3.42 cores=0-7
/proc/31226/stat : 31226 (adaptg2wsat2009) R 31224 31226 31226 0 -1 4202496 2632 0 0 0 342 0 0 0 25 0 1 0 746727452 433561600 2358 33554432000 134512640 135107411 4288554432 18446744073709551615 134531942 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/31226/statm: 105850 2358 59 146 0 104726 0
[pid=31197/tid=31204] ppid=31195 vsize=372084 CPUtime=0 cores=0-7
/proc/31197/task/31204/stat : 31204 (python) S 31195 31197 31171 0 -1 4202560 25 934 0 0 0 0 0 0 19 0 9 0 746727429 381014016 9987 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 0 0 0 0
[pid=31197/tid=31207] ppid=31195 vsize=372084 CPUtime=0 cores=0-7
/proc/31197/task/31207/stat : 31207 (python) S 31195 31197 31171 0 -1 4202560 25 934 0 0 0 0 0 0 21 0 9 0 746727432 381014016 9987 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 0 0 0 0
[pid=31197/tid=31210] ppid=31195 vsize=372084 CPUtime=0 cores=0-7
/proc/31197/task/31210/stat : 31210 (python) S 31195 31197 31171 0 -1 4202560 20 934 0 0 0 0 0 0 23 0 9 0 746727435 381014016 9987 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 0 0 0 0
[pid=31197/tid=31213] ppid=31195 vsize=372084 CPUtime=0 cores=0-7
/proc/31197/task/31213/stat : 31213 (python) S 31195 31197 31171 0 -1 4202560 22 934 0 0 0 0 0 0 20 0 9 0 746727439 381014016 9987 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 2 0 0 0
[pid=31197/tid=31216] ppid=31195 vsize=372084 CPUtime=0 cores=0-7
/proc/31197/task/31216/stat : 31216 (python) S 31195 31197 31171 0 -1 4202560 25 934 0 0 0 0 0 0 20 0 9 0 746727442 381014016 9987 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 0 0 0 0
[pid=31197/tid=31219] ppid=31195 vsize=372084 CPUtime=0 cores=0-7
/proc/31197/task/31219/stat : 31219 (python) S 31195 31197 31171 0 -1 4202560 25 934 0 0 0 0 0 0 20 0 9 0 746727445 381014016 9987 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 4 0 0 0
[pid=31197/tid=31222] ppid=31195 vsize=372084 CPUtime=0 cores=0-7
/proc/31197/task/31222/stat : 31222 (python) S 31195 31197 31171 0 -1 4202560 30 934 0 0 0 0 0 0 22 0 9 0 746727448 381014016 9987 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 6 0 0 0
[pid=31197/tid=31225] ppid=31195 vsize=372084 CPUtime=0 cores=0-7
/proc/31197/task/31225/stat : 31225 (python) S 31195 31197 31171 0 -1 4202560 26 934 0 0 0 0 0 0 20 0 9 0 746727452 381014016 9987 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 30.58
Current children cumulated vsize (KiB) 7987188

[startup+12.7011 s]
/proc/loadavg: 0.71 0.15 0.05 9/437 31226
/proc/meminfo: memFree=17897496/32951124 swapFree=57414336/67111528
[pid=31197] ppid=31195 vsize=372084 CPUtime=2.47 cores=0-7
/proc/31197/stat : 31197 (python) S 31195 31197 31171 0 -1 4202496 20354 934 1 0 231 16 0 0 20 0 9 0 746727164 381014016 9987 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469892555 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31197/statm: 93021 9987 1891 364 0 28918 0
[pid=31203] ppid=31197 vsize=290048 CPUtime=0.01 cores=0-7
/proc/31203/stat : 31203 (python) S 31197 31197 31171 0 -1 4202560 901 0 0 0 0 1 0 0 18 0 1 0 746727429 297009152 8440 33554432000 4194304 5683644 140736118916944 18446744073709551615 252458085903 0 0 16781312 514 978021369637732611 0 0 17 1 0 0 0
/proc/31203/statm: 72512 8440 376 364 0 8430 0
[pid=31205] ppid=31203 vsize=1020308 CPUtime=10.04 cores=0-7
/proc/31205/stat : 31205 (gnovelty+2) R 31203 31205 31205 0 -1 4202496 1154 0 0 0 1004 0 0 0 25 0 1 0 746727429 1044795392 892 33554432000 134512640 135663704 4290839888 18446744073709551615 134524918 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/31205/statm: 255077 892 113 282 0 254793 0
[pid=31206] ppid=31197 vsize=300304 CPUtime=0.01 cores=0-7
/proc/31206/stat : 31206 (python) S 31197 31197 31171 0 -1 4202560 907 0 0 0 0 1 0 0 18 0 1 0 746727432 307511296 8438 33554432000 4194304 5683644 140736118916944 18446744073709551615 252458085958 0 0 16781312 514 978025217928429826 0 0 17 0 0 0 0
/proc/31206/statm: 75076 8438 370 364 0 10991 0
[pid=31208] ppid=31206 vsize=1020308 CPUtime=10.01 cores=0-7
/proc/31208/stat : 31208 (gnovelty+2) R 31206 31208 31208 0 -1 4202496 1155 0 0 0 1001 0 0 0 25 0 1 0 746727432 1044795392 892 33554432000 134512640 135663704 4293791632 18446744073709551615 134524498 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/31208/statm: 255077 892 113 282 0 254793 0
[pid=31209] ppid=31197 vsize=310560 CPUtime=0.02 cores=0-7
/proc/31209/stat : 31209 (python) S 31197 31197 31171 0 -1 4202560 904 0 0 0 1 1 0 0 18 0 1 0 746727435 318013440 8442 33554432000 4194304 5683644 140736118916944 18446744073709551615 252458085958 0 0 16781312 514 11048640233217163521 0 0 17 2 0 0 0
/proc/31209/statm: 77640 8442 370 364 0 13552 0
[pid=31211] ppid=31209 vsize=423400 CPUtime=9.81 cores=0-7
/proc/31211/stat : 31211 (adaptg2wsat2009) R 31209 31211 31211 0 -1 4202496 2623 0 0 0 981 0 0 0 25 0 1 0 746727436 433561600 2357 33554432000 134512640 135107411 4288752832 18446744073709551615 134532277 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/31211/statm: 105850 2357 59 146 0 104726 0
[pid=31212] ppid=31197 vsize=320816 CPUtime=0.01 cores=0-7
/proc/31212/stat : 31212 (python) S 31197 31197 31171 0 -1 4202560 908 0 0 0 0 1 0 0 18 0 1 0 746727438 328515584 8446 33554432000 4194304 5683644 140736118916944 18446744073709551615 252458085958 0 0 16781312 514 16608528311886250243 0 0 17 2 0 0 0
/proc/31212/statm: 80204 8446 370 364 0 16113 0
[pid=31214] ppid=31212 vsize=423400 CPUtime=9.91 cores=0-7
/proc/31214/stat : 31214 (adaptg2wsat2009) R 31212 31214 31214 0 -1 4202496 2623 0 0 0 991 0 0 0 25 0 1 0 746727439 433561600 2357 33554432000 134512640 135107411 4294854864 18446744073709551615 134531600 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/31214/statm: 105850 2357 59 146 0 104726 0
[pid=31215] ppid=31197 vsize=331072 CPUtime=0.01 cores=0-7
/proc/31215/stat : 31215 (python) S 31197 31197 31171 0 -1 4202560 937 0 0 0 0 1 0 0 18 0 1 0 746727441 339017728 8450 33554432000 4194304 5683644 140736118916944 18446744073709551615 252458085958 0 0 16781312 514 978029066219127042 0 0 17 0 0 0 0
/proc/31215/statm: 82768 8450 370 364 0 18674 0
[pid=31217] ppid=31215 vsize=423400 CPUtime=9.87 cores=0-7
/proc/31217/stat : 31217 (adaptg2wsat2009) R 31215 31217 31217 0 -1 4202496 2630 0 0 0 987 0 0 0 25 0 1 0 746727442 433561600 2359 33554432000 134512640 135107411 4292223216 18446744073709551615 134532128 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/31217/statm: 105850 2359 59 146 0 104726 0
[pid=31218] ppid=31197 vsize=341328 CPUtime=0.02 cores=0-7
/proc/31218/stat : 31218 (python) S 31197 31197 31171 0 -1 4202560 936 0 0 0 1 1 0 0 18 0 1 0 746727445 349519872 8454 33554432000 4194304 5683644 140736118916944 18446744073709551615 252458085958 0 0 16781312 514 978032914509824257 0 0 17 6 0 0 0
/proc/31218/statm: 85332 8454 370 364 0 21235 0
[pid=31220] ppid=31218 vsize=1020308 CPUtime=9.85 cores=0-7
/proc/31220/stat : 31220 (gnovelty+2) R 31218 31220 31220 0 -1 4202496 1157 0 0 0 985 0 0 0 25 0 1 0 746727445 1044795392 892 33554432000 134512640 135663704 4292944896 18446744073709551615 134545572 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/31220/statm: 255077 892 113 282 0 254793 0
[pid=31221] ppid=31197 vsize=351584 CPUtime=0.01 cores=0-7
/proc/31221/stat : 31221 (python) S 31197 31197 31171 0 -1 4202560 946 0 0 0 0 1 0 0 18 0 1 0 746727448 360022016 8458 33554432000 4194304 5683644 140736118916944 18446744073709551615 252458085958 0 0 16781312 514 5588464939925733633 0 0 17 6 0 0 0

################
# More data... #
################

/proc/31302/statm: 142740 8875 407 364 0 78595 0
[pid=31304] ppid=31302 vsize=335996 CPUtime=100.32 cores=0-7
/proc/31304/stat : 31304 (iPAWS) T 31302 31304 31304 0 -1 4202496 2058 0 0 0 10032 0 0 0 25 0 1 0 746767306 344059904 1776 33554432000 134512640 135114423 4287092320 18446744073709551615 134515806 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/31304/statm: 83999 1776 59 147 0 83851 0
[pid=31302/tid=31311] ppid=31197 vsize=570960 CPUtime=0 cores=0-7
/proc/31302/task/31311/stat : 31311 (python) S 31197 31197 31171 0 -1 4202560 94 0 0 0 0 0 0 0 19 0 2 0 746777339 584663040 8875 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 514 18446604442125238528 0 0 -1 1 0 0 0
[pid=31308] ppid=31197 vsize=581216 CPUtime=0.33 cores=0-7
/proc/31308/stat : 31308 (python) S 31197 31197 31171 0 -1 4202560 1196 0 0 0 18 15 0 0 19 0 2 0 746776831 595165184 8872 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469892555 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/31308/statm: 145304 8872 407 364 0 81156 0
[pid=31310] ppid=31308 vsize=1020308 CPUtime=100.23 cores=0-7
/proc/31310/stat : 31310 (gnovelty+2) T 31308 31310 31310 0 -1 4202496 1175 0 0 0 10023 0 0 0 25 0 1 0 746776831 1044795392 891 33554432000 134512640 135663704 4286905216 18446744073709551615 134524918 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/31310/statm: 255077 891 113 282 0 254793 0
[pid=31308/tid=31317] ppid=31197 vsize=581216 CPUtime=0 cores=0-7
/proc/31308/task/31317/stat : 31317 (python) S 31197 31197 31171 0 -1 4202560 89 0 0 0 0 0 0 0 19 0 2 0 746786869 595165184 8872 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 514 18446604444334524608 0 0 -1 1 0 0 0
[pid=31318] ppid=31197 vsize=591472 CPUtime=1.03 cores=0-7
/proc/31318/stat : 31318 (python) S 31197 31197 31171 0 -1 4202560 1227 0 0 0 55 48 0 0 18 0 2 0 746787569 605667328 8941 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469892555 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/31318/statm: 147868 8941 406 364 0 83717 0
[pid=31320] ppid=31318 vsize=268424 CPUtime=291.57 cores=0-7
/proc/31320/stat : 31320 (hybridGM3) T 31318 31320 31320 0 -1 4202496 6206 0 0 0 29157 0 0 0 25 0 1 0 746787569 274866176 5920 33554432000 134512640 135204385 4292829904 18446744073709551615 134538395 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/31320/statm: 67106 5920 63 169 0 66936 0
[pid=31318/tid=31332] ppid=31197 vsize=591472 CPUtime=0 cores=0-7
/proc/31318/task/31332/stat : 31332 (python) S 31197 31197 31171 0 -1 4202560 97 0 0 0 0 0 0 0 18 0 2 0 746807052 605667328 8941 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 514 18446604450066981248 0 0 -1 0 0 0 0
[pid=31323] ppid=31197 vsize=601728 CPUtime=2.1 cores=0-7
/proc/31323/stat : 31323 (python) S 31197 31197 31171 0 -1 4202560 1117 0 0 0 108 102 0 0 18 0 1 0 746795940 616169472 8800 33554432000 4194304 5683644 140736118916944 18446744073709551615 252458085958 0 0 16781312 514 13713284943482683651 0 0 17 2 0 0 0
/proc/31323/statm: 150432 8800 371 364 0 86278 0
[pid=31325] ppid=31323 vsize=461508 CPUtime=593.07 cores=0-7
/proc/31325/stat : 31325 (TNM) R 31323 31325 31325 0 -1 4202496 2486 0 0 0 59305 2 0 0 25 0 1 0 746795941 472584192 2195 33554432000 134512640 135008748 4292287984 18446744073709551615 134527524 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/31325/statm: 115377 2195 46 122 0 114277 0
[pid=31326] ppid=31197 vsize=611984 CPUtime=0.36 cores=0-7
/proc/31326/stat : 31326 (python) S 31197 31197 31171 0 -1 4202560 1227 0 0 0 22 14 0 0 19 0 2 0 746796616 626671616 8956 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469892555 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/31326/statm: 152996 8956 400 364 0 88839 0
[pid=31328] ppid=31326 vsize=335996 CPUtime=101.5 cores=0-7
/proc/31328/stat : 31328 (iPAWS) T 31326 31328 31328 0 -1 4202496 2054 0 0 0 10150 0 0 0 25 0 1 0 746796616 344059904 1774 33554432000 134512640 135114423 4287716752 18446744073709551615 134517442 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/31328/statm: 83999 1774 59 147 0 83851 0
[pid=31326/tid=31333] ppid=31197 vsize=611984 CPUtime=0 cores=0-7
/proc/31326/task/31333/stat : 31333 (python) S 31197 31197 31171 0 -1 4202560 151 0 0 0 0 0 0 0 19 0 2 0 746807256 626671616 8956 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 514 18446604453985334208 0 0 -1 7 0 0 0
[pid=31337] ppid=31197 vsize=622240 CPUtime=0.37 cores=0-7
/proc/31337/stat : 31337 (python) S 31197 31197 31171 0 -1 4202560 1194 0 0 0 20 17 0 0 19 0 2 0 746816903 637173760 8968 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469892555 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/31337/statm: 155560 8968 400 364 0 91400 0
[pid=31339] ppid=31337 vsize=253028 CPUtime=100.8 cores=0-7
/proc/31339/stat : 31339 (FH) T 31337 31339 31339 0 -1 4202496 2516 0 0 0 10080 0 0 0 25 0 1 0 746816904 259100672 2232 33554432000 134512640 134936988 4287672192 18446744073709551615 134553238 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/31339/statm: 63257 2232 41 104 0 62761 0
[pid=31337/tid=31352] ppid=31197 vsize=622240 CPUtime=0 cores=0-7
/proc/31337/task/31352/stat : 31352 (python) S 31197 31197 31171 0 -1 4202560 106 0 0 0 0 0 0 0 19 0 2 0 746827442 637173760 8968 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 514 18446604444306371520 0 0 -1 7 0 0 0
[pid=31340] ppid=31197 vsize=632496 CPUtime=0.35 cores=0-7
/proc/31340/stat : 31340 (python) S 31197 31197 31171 0 -1 4202560 1203 0 0 0 18 17 0 0 19 0 2 0 746817202 647675904 9060 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469892555 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/31340/statm: 158124 9060 401 364 0 93961 0
[pid=31342] ppid=31340 vsize=335996 CPUtime=98.72 cores=0-7
/proc/31342/stat : 31342 (iPAWS) T 31340 31342 31342 0 -1 4202496 2055 0 0 0 9871 1 0 0 25 0 1 0 746817202 344059904 1774 33554432000 134512640 135114423 4288754816 18446744073709551615 134546166 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/31342/statm: 83999 1774 59 147 0 83851 0
[pid=31340/tid=31348] ppid=31197 vsize=632496 CPUtime=0 cores=0-7
/proc/31340/task/31348/stat : 31348 (python) S 31197 31197 31171 0 -1 4202560 102 0 0 0 0 0 0 0 19 0 2 0 746827140 647675904 9060 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 514 18446604453861255296 0 0 -1 5 0 0 0
[pid=31345] ppid=31197 vsize=642752 CPUtime=1.1 cores=0-7
/proc/31345/stat : 31345 (python) S 31197 31197 31171 0 -1 4202560 1251 0 0 0 55 55 0 0 18 0 2 0 746825534 658178048 9025 33554432000 4194304 5683644 140736118916944 18446744073709551615 252458085958 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/31345/statm: 160688 9025 406 364 0 96522 0
[pid=31347] ppid=31345 vsize=268424 CPUtime=311.91 cores=0-7
/proc/31347/stat : 31347 (hybridGM3) R 31345 31347 31347 0 -1 4202496 6218 0 0 0 31190 1 0 0 25 0 1 0 746825534 274866176 5920 33554432000 134512640 135204385 4292904320 18446744073709551615 134538541 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/31347/statm: 67106 5920 63 169 0 66936 0
[pid=31345/tid=31365] ppid=31197 vsize=642752 CPUtime=0 cores=0-7
/proc/31345/task/31365/stat : 31365 (python) S 31197 31197 31171 0 -1 4202560 103 0 0 0 0 0 0 0 18 0 2 0 746845209 658178048 9025 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 514 18446604440389657792 0 0 -1 1 0 0 0
[pid=31349] ppid=31197 vsize=653008 CPUtime=1.02 cores=0-7
/proc/31349/stat : 31349 (python) S 31197 31197 31171 0 -1 4202560 1040 0 0 0 53 49 0 0 18 0 1 0 746827146 668680192 8978 33554432000 4194304 5683644 140736118916944 18446744073709551615 252458085958 0 0 16781312 514 13713284943482683649 0 0 17 0 0 0 0
/proc/31349/statm: 163252 8978 379 364 0 99083 0
[pid=31351] ppid=31349 vsize=423308 CPUtime=295.95 cores=0-7
/proc/31351/stat : 31351 (NCVWr) R 31349 31351 31351 0 -1 4202496 2553 0 0 0 29594 1 0 0 25 0 1 0 746827146 433467392 2265 33554432000 134512640 135113558 4289564336 18446744073709551615 134533845 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/31351/statm: 105827 2265 58 147 0 104702 0
[pid=31353] ppid=31197 vsize=663264 CPUtime=0.35 cores=0-7
/proc/31353/stat : 31353 (python) S 31197 31197 31171 0 -1 4202560 1251 0 0 0 18 17 0 0 18 0 2 0 746827448 679182336 9104 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469892555 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/31353/statm: 165816 9104 406 364 0 101644 0
[pid=31355] ppid=31353 vsize=1020308 CPUtime=100.05 cores=0-7
/proc/31355/stat : 31355 (gnovelty+2) T 31353 31355 31355 0 -1 4202496 1181 0 0 0 10005 0 0 0 25 0 1 0 746827449 1044795392 891 33554432000 134512640 135663704 4287393600 18446744073709551615 134544196 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/31355/statm: 255077 891 113 282 0 254793 0
[pid=31353/tid=31362] ppid=31197 vsize=663264 CPUtime=0 cores=0-7
/proc/31353/task/31362/stat : 31362 (python) S 31197 31197 31171 0 -1 4202560 102 0 0 0 0 0 0 0 19 0 2 0 746837488 679182336 9104 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 514 18446604448385599488 0 0 -1 3 0 0 0
[pid=31359] ppid=31197 vsize=673520 CPUtime=0.7 cores=0-7
/proc/31359/stat : 31359 (python) S 31197 31197 31171 0 -1 4202560 1273 0 0 0 37 33 0 0 18 0 2 0 746836774 689684480 9059 33554432000 4194304 5683644 140736118916944 18446744073709551615 252458085958 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/31359/statm: 168380 9059 407 364 0 104205 0
[pid=31361] ppid=31359 vsize=268420 CPUtime=201.68 cores=0-7
/proc/31361/stat : 31361 (hybridGM3) R 31359 31361 31361 0 -1 4202496 6210 0 0 0 20167 1 0 0 25 0 1 0 746836775 274862080 5918 33554432000 134512640 135204385 4293987360 18446744073709551615 134538375 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/31361/statm: 67105 5918 63 169 0 66935 0
[pid=31359/tid=31369] ppid=31197 vsize=673520 CPUtime=0 cores=0-7
/proc/31359/task/31369/stat : 31369 (python) S 31197 31197 31171 0 -1 4202560 103 0 0 0 0 0 0 0 19 0 2 0 746856352 689684480 9059 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 514 18446604445741736000 0 0 -1 3 0 0 0
[pid=31370] ppid=31197 vsize=683776 CPUtime=0 cores=0-7
/proc/31370/stat : 31370 (python) S 31197 31197 31171 0 -1 4202560 965 0 0 0 0 0 0 0 19 0 1 0 746856868 700186624 9015 33554432000 4194304 5683644 140736118916944 18446744073709551615 252458085958 0 0 16781312 514 1077763292073558015 0 0 17 0 0 0 0
/proc/31370/statm: 170944 9015 355 364 0 106766 0
[pid=31372] ppid=31370 vsize=268420 CPUtime=0.88 cores=0-7
/proc/31372/stat : 31372 (hybridGM3) R 31370 31372 31372 0 -1 4202496 6213 0 0 0 88 0 0 0 23 0 1 0 746856869 274862080 5919 33554432000 134512640 135204385 4287505504 18446744073709551615 134538284 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/31372/statm: 67105 5919 63 169 0 66935 0
[pid=31197/tid=31204] ppid=31195 vsize=694020 CPUtime=0 cores=0-7
/proc/31197/task/31204/stat : 31204 (python) S 31195 31197 31171 0 -1 4202560 25 934 0 0 0 0 0 0 19 0 34 0 746727429 710676480 10599 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 0 0 0 0
[pid=31197/tid=31207] ppid=31195 vsize=694020 CPUtime=0 cores=0-7
/proc/31197/task/31207/stat : 31207 (python) S 31195 31197 31171 0 -1 4202560 25 934 0 0 0 0 0 0 21 0 34 0 746727432 710676480 10599 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 0 0 0 0
[pid=31197/tid=31210] ppid=31195 vsize=694020 CPUtime=0 cores=0-7
/proc/31197/task/31210/stat : 31210 (python) S 31195 31197 31171 0 -1 4202560 34 934 0 0 0 0 0 0 18 0 34 0 746727435 710676480 10599 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 0 0 0 0
[pid=31197/tid=31213] ppid=31195 vsize=694020 CPUtime=0 cores=0-7
/proc/31197/task/31213/stat : 31213 (python) S 31195 31197 31171 0 -1 4202560 27 934 0 0 0 0 0 0 18 0 34 0 746727439 710676480 10599 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 2 0 0 0
[pid=31197/tid=31216] ppid=31195 vsize=694020 CPUtime=0 cores=0-7
/proc/31197/task/31216/stat : 31216 (python) S 31195 31197 31171 0 -1 4202560 29 934 0 0 0 0 0 0 18 0 34 0 746727442 710676480 10599 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 0 0 0 0
[pid=31197/tid=31219] ppid=31195 vsize=694020 CPUtime=0 cores=0-7
/proc/31197/task/31219/stat : 31219 (python) S 31195 31197 31171 0 -1 4202560 25 934 0 0 0 0 0 0 20 0 34 0 746727445 710676480 10599 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 4 0 0 0
[pid=31197/tid=31222] ppid=31195 vsize=694020 CPUtime=0 cores=0-7
/proc/31197/task/31222/stat : 31222 (python) S 31195 31197 31171 0 -1 4202560 46 934 0 0 0 0 0 0 18 0 34 0 746727448 710676480 10599 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 6 0 0 0
[pid=31197/tid=31225] ppid=31195 vsize=694020 CPUtime=0 cores=0-7
/proc/31197/task/31225/stat : 31225 (python) S 31195 31197 31171 0 -1 4202560 30 934 0 0 0 0 0 0 18 0 34 0 746727452 710676480 10599 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 6 0 0 0
[pid=31197/tid=31233] ppid=31195 vsize=694020 CPUtime=0 cores=0-7
/proc/31197/task/31233/stat : 31233 (python) S 31195 31197 31171 0 -1 4202560 33 934 0 0 0 0 0 0 17 0 34 0 746737469 710676480 10599 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 6 0 0 0
[pid=31197/tid=31237] ppid=31195 vsize=694020 CPUtime=0 cores=0-7
/proc/31197/task/31237/stat : 31237 (python) S 31195 31197 31171 0 -1 4202560 38 934 0 0 0 0 0 0 18 0 34 0 746737473 710676480 10599 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 0 0 0 0
[pid=31197/tid=31241] ppid=31195 vsize=694020 CPUtime=0 cores=0-7
/proc/31197/task/31241/stat : 31241 (python) S 31195 31197 31171 0 -1 4202560 37 934 0 0 0 0 0 0 18 0 34 0 746737477 710676480 10599 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 6 0 0 0
[pid=31197/tid=31244] ppid=31195 vsize=694020 CPUtime=0 cores=0-7
/proc/31197/task/31244/stat : 31244 (python) S 31195 31197 31171 0 -1 4202560 26 934 0 0 0 0 0 0 24 0 34 0 746737481 710676480 10599 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 0 0 0 0
[pid=31197/tid=31247] ppid=31195 vsize=694020 CPUtime=0 cores=0-7
/proc/31197/task/31247/stat : 31247 (python) S 31195 31197 31171 0 -1 4202560 33 934 0 0 0 0 0 0 18 0 34 0 746737485 710676480 10599 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 6 0 0 0
[pid=31197/tid=31253] ppid=31195 vsize=694020 CPUtime=0 cores=0-7
/proc/31197/task/31253/stat : 31253 (python) S 31195 31197 31171 0 -1 4202560 31 934 0 0 0 0 0 0 18 0 34 0 746737495 710676480 10599 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 2 0 0 0
[pid=31197/tid=31256] ppid=31195 vsize=694020 CPUtime=0 cores=0-7
/proc/31197/task/31256/stat : 31256 (python) S 31195 31197 31171 0 -1 4202560 38 934 0 0 0 0 0 0 18 0 34 0 746737500 710676480 10599 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 6 0 0 0
[pid=31197/tid=31259] ppid=31195 vsize=694020 CPUtime=0 cores=0-7
/proc/31197/task/31259/stat : 31259 (python) S 31195 31197 31171 0 -1 4202560 30 934 0 0 0 0 0 0 18 0 34 0 746737505 710676480 10599 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 2 0 0 0
[pid=31197/tid=31267] ppid=31195 vsize=694020 CPUtime=0 cores=0-7
/proc/31197/task/31267/stat : 31267 (python) S 31195 31197 31171 0 -1 4202560 23 934 0 0 0 0 0 0 19 0 34 0 746747507 710676480 10599 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 2 0 0 0
[pid=31197/tid=31270] ppid=31195 vsize=694020 CPUtime=0 cores=0-7
/proc/31197/task/31270/stat : 31270 (python) S 31195 31197 31171 0 -1 4202560 27 934 0 0 0 0 0 0 21 0 34 0 746747513 710676480 10599 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 2 0 0 0
[pid=31197/tid=31292] ppid=31195 vsize=694020 CPUtime=0 cores=0-7
/proc/31197/task/31292/stat : 31292 (python) S 31195 31197 31171 0 -1 4202560 52 934 0 0 0 0 0 0 18 0 34 0 746757778 710676480 10599 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 1 0 0 0
[pid=31197/tid=31297] ppid=31195 vsize=694020 CPUtime=0 cores=0-7
/proc/31197/task/31297/stat : 31297 (python) S 31195 31197 31171 0 -1 4202560 139 934 0 0 0 0 0 0 18 0 34 0 746766726 710676480 10599 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 1 0 0 0
[pid=31197/tid=31300] ppid=31195 vsize=694020 CPUtime=0 cores=0-7
/proc/31197/task/31300/stat : 31300 (python) S 31195 31197 31171 0 -1 4202560 37 934 0 0 0 0 0 0 20 0 34 0 746766911 710676480 10599 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 2 0 0 0
[pid=31197/tid=31303] ppid=31195 vsize=694020 CPUtime=0 cores=0-7
/proc/31197/task/31303/stat : 31303 (python) S 31195 31197 31171 0 -1 4202560 40 934 0 0 0 0 0 0 20 0 34 0 746767306 710676480 10599 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 2 0 0 0
[pid=31197/tid=31309] ppid=31195 vsize=694020 CPUtime=0 cores=0-7
/proc/31197/task/31309/stat : 31309 (python) S 31195 31197 31171 0 -1 4202560 34 934 0 0 0 0 0 0 20 0 34 0 746776831 710676480 10599 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 1 0 0 0
[pid=31197/tid=31319] ppid=31195 vsize=694020 CPUtime=0 cores=0-7
/proc/31197/task/31319/stat : 31319 (python) S 31195 31197 31171 0 -1 4202560 47 934 0 0 0 0 0 0 18 0 34 0 746787569 710676480 10599 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 2 0 0 0
[pid=31197/tid=31324] ppid=31195 vsize=694020 CPUtime=0 cores=0-7
/proc/31197/task/31324/stat : 31324 (python) S 31195 31197 31171 0 -1 4202560 33 934 0 0 0 0 0 0 20 0 34 0 746795941 710676480 10599 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 1 0 0 0
[pid=31197/tid=31327] ppid=31195 vsize=694020 CPUtime=0 cores=0-7
/proc/31197/task/31327/stat : 31327 (python) S 31195 31197 31171 0 -1 4202560 97 934 0 0 0 0 0 0 20 0 34 0 746796616 710676480 10599 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 2 0 0 0
[pid=31197/tid=31338] ppid=31195 vsize=694020 CPUtime=0 cores=0-7
/proc/31197/task/31338/stat : 31338 (python) S 31195 31197 31171 0 -1 4202560 39 934 0 0 0 0 0 0 20 0 34 0 746816903 710676480 10599 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 1 0 0 0
[pid=31197/tid=31341] ppid=31195 vsize=694020 CPUtime=0 cores=0-7
/proc/31197/task/31341/stat : 31341 (python) S 31195 31197 31171 0 -1 4202560 37 934 0 0 0 0 0 0 20 0 34 0 746817202 710676480 10599 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 0 0 0 0
[pid=31197/tid=31346] ppid=31195 vsize=694020 CPUtime=0 cores=0-7
/proc/31197/task/31346/stat : 31346 (python) S 31195 31197 31171 0 -1 4202560 45 934 0 0 0 0 0 0 18 0 34 0 746825534 710676480 10599 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 3 0 0 0
[pid=31197/tid=31350] ppid=31195 vsize=694020 CPUtime=0 cores=0-7
/proc/31197/task/31350/stat : 31350 (python) S 31195 31197 31171 0 -1 4202560 39 934 0 0 0 0 0 0 20 0 34 0 746827146 710676480 10599 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 0 0 0 0
[pid=31197/tid=31354] ppid=31195 vsize=694020 CPUtime=0 cores=0-7
/proc/31197/task/31354/stat : 31354 (python) S 31195 31197 31171 0 -1 4202560 37 934 0 0 0 0 0 0 19 0 34 0 746827448 710676480 10599 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 0 0 0 0
[pid=31197/tid=31360] ppid=31195 vsize=694020 CPUtime=0 cores=0-7
/proc/31197/task/31360/stat : 31360 (python) S 31195 31197 31171 0 -1 4202560 36 934 0 0 0 0 0 0 18 0 34 0 746836774 710676480 10599 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 1 0 0 0
[pid=31197/tid=31371] ppid=31195 vsize=694020 CPUtime=0 cores=0-7
/proc/31197/task/31371/stat : 31371 (python) S 31195 31197 31171 0 -1 4202560 42 934 0 0 0 0 0 0 20 0 34 0 746856869 710676480 10599 33554432000 4194304 5683644 140736118916944 18446744073709551615 252469889457 0 0 16781312 2 18446604448385600384 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 10310.7
Current children cumulated vsize (KiB) 31914024

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 31197 and gives
#  childrusage.ru_utime.tv_sec=2606
#  childrusage.ru_utime.tv_usec=417764
#  childrusage.ru_stime.tv_sec=16
#  childrusage.ru_stime.tv_usec=706460
# CPU time returned by wait4() is 2623.12
# while last known CPU time is 10310.7
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1298.12
CPU time (s): 10310.7
CPU user time (s): 10293.1
CPU system time (s): 17.63
CPU usage (%): 794.285
Max. virtual memory (cumulated for all children) (KiB): 31914024

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2606.42
system time used= 16.7065
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 97317
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 20858
involuntary context switches= 15350

runsolver used 15.5746 second user time and 35.9075 second system time

The end

Launcher Data 

Begin job on node111 at 2011-05-16 18:56:07
IDJOB=3385278
IDBENCH=82543
IDSOLVER=1870
FILE ID=node111/3385278-1305564966
RUNJOBID= node111-1305564966-31185
PBS_JOBID= 13350139
Free space on /tmp= 73580 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/large/unif-k5-r20-v2000-c40000-S319509287-085-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3385278-1305564966/watcher-3385278-1305564966 -o /tmp/evaluation-result-3385278-1305564966/solver-3385278-1305564966 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385278-1305564966.cnf -seed 1960073158 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= bb548a1e884feefbb349d3449e849f5e
RANDOM SEED=1960073158

node111.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5428.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      18025940 kB
Buffers:        309496 kB
Cached:        6401016 kB
SwapCached:      19200 kB
Active:        7296772 kB
Inactive:      7367484 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      18025940 kB
SwapTotal:    67111528 kB
SwapFree:     57414336 kB
Dirty:           32620 kB
Writeback:           0 kB
AnonPages:     7943696 kB
Mapped:          24104 kB
Slab:           133356 kB
PageTables:      59148 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 124885228 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73548 MiB
End job on node111 at 2011-05-16 19:17:47