Trace number 3385264

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)? (MO) 8735.33 1100.11

General information on the benchmark

Namerandom/large/
unif-k5-r20-v2000-c40000-S656892101-075-UNKNOWN.cnf
MD5SUM187184c44e9c5a07f5a8f762e04588fa
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark379.26
Satisfiable
(Un)Satisfiability was proved
Number of variables2000
Number of clauses40000
Sum of the clauses size200000
Maximum clause length5
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 540000
Number of clauses of size over 50

Solver Data 

0.39/1.44	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.39/1.44	c INFO: solving HOME/instance-3385264-1305564964.cnf
0.39/1.44	c INFO: computing task features
0.57/1.65	c INFO: computed variable-clause graph statistics
0.57/1.66	c INFO: computed clause balance statistics
1.57/2.66	c INFO: computed variable balance statistics
2.19/3.22	c INFO: computed small-clause counts
2.19/3.22	c INFO: computed Horn-clause counts
2.19/3.27	c INFO: computed variable graph statistics
2.19/3.27	c INFO: collected features for HOME/instance-3385264-1305564964.cnf in 1.82 s
2.19/3.30	c INFO: running gnovelty+2@0 for 97 with 4998 remaining (b = 0.38)
2.28/3.33	c INFO: running gnovelty+2@0 for 97 with 4998 remaining (b = 0.13)
2.28/3.36	c INFO: running adaptg2wsat2009++@0 for 97 with 4998 remaining (b = 0.08)
2.28/3.39	c INFO: running adaptg2wsat2009++@0 for 97 with 4998 remaining (b = 0.07)
2.36/3.42	c INFO: running adaptg2wsat2009++@0 for 97 with 4998 remaining (b = 0.05)
2.36/3.46	c INFO: running gnovelty+2@0 for 97 with 4998 remaining (b = 0.05)
2.36/3.49	c INFO: running FH@0 for 97 with 4997 remaining (b = 0.03)
2.46/3.52	c INFO: running adaptg2wsat2009++@0 for 97 with 4997 remaining (b = 0.03)
802.55/103.63	c INFO: running adaptg2wsat2009++@0 for 97 with 4897 remaining (b = 0.02)
802.55/103.66	c INFO: running FH@0 for 97 with 4897 remaining (b = 0.02)
803.15/103.71	c INFO: running hybridGM3@0 for 194 with 4897 remaining (b = 0.04)
803.15/103.75	c INFO: running gnovelty+2@0 for 97 with 4897 remaining (b = 0.02)
803.15/103.79	c INFO: running hybridGM3@0 for 194 with 4897 remaining (b = 0.03)
803.46/103.84	c INFO: running adaptg2wsat2009++@0 for 97 with 4897 remaining (b = 0.01)
803.46/103.89	c INFO: running hybridGM3@0 for 194 with 4897 remaining (b = 0.03)
803.61/103.94	c INFO: running hybridGM3@0 for 194 with 4897 remaining (b = 0.02)
1578.83/201.01	c INFO: running TNM@0 for 1552 with 4800 remaining (b = 0.19)
1595.54/203.11	c INFO: running FH@0 for 97 with 4798 remaining (b = 0.01)
1595.54/203.19	c INFO: running adaptg2wsat2009++@0 for 97 with 4798 remaining (b = 0.01)
1601.88/203.94	c INFO: running adaptg2wsat2009++@0 for 97 with 4797 remaining (b = 0.01)
2363.67/299.57	c INFO: running hybridGM3@99 for 97 with 4701 remaining (b = 0.01)
2370.14/300.34	c INFO: running hybridGM3@99 for 97 with 4701 remaining (b = 0.01)
2370.82/300.43	c INFO: running hybridGM3@99 for 97 with 4701 remaining (b = 0.01)
2370.82/300.49	c INFO: running adaptg2wsat2009++@0 for 97 with 4700 remaining (b = 0.01)
2394.72/303.42	c INFO: running adaptg2wsat2009++@100 for 97 with 4698 remaining (b = 0.01)
2405.16/304.78	c INFO: running hybridGM3@97 for 97 with 4696 remaining (b = 0.01)
2425.76/307.32	c INFO: running hybridGM3@0 for 194 with 4694 remaining (b = 0.02)
3130.19/395.72	c INFO: running hybridGM3@0 for 194 with 4605 remaining (b = 0.02)
3146.92/397.81	c INFO: running TNM@0 for 1455 with 4603 remaining (b = 0.14)
3169.97/400.73	c INFO: running adaptg2wsat2009++@100 for 97 with 4600 remaining (b = 0.01)
3171.55/400.96	c INFO: running hybridGM3@295 for 97 with 4600 remaining (b = 0.01)
3177.08/401.65	c INFO: running hybridGM3@295 for 97 with 4599 remaining (b = 0.01)
3212.13/406.09	c INFO: running FH@0 for 291 with 4595 remaining (b = 0.03)
3930.35/496.28	c INFO: running gnovelty+2@0 for 97 with 4505 remaining (b = 0.01)
3935.28/496.94	c INFO: running hybridGM3@396 for 97 with 4504 remaining (b = 0.01)
3944.49/498.08	c INFO: running hybridGM3@393 for 97 with 4503 remaining (b = 0.01)
3984.32/503.03	c INFO: running iPAWS@0 for 97 with 4498 remaining (b = 0.01)
4687.79/591.38	c INFO: running hybridGM3@94 for 97 with 4410 remaining (b = 0.01)
4694.91/592.24	c INFO: running hybridGM3@97 for 97 with 4409 remaining (b = 0.01)
4729.96/596.65	c INFO: running adaptg2wsat2009++@298 for 291 with 4404 remaining (b = 0.02)
4743.59/598.40	c INFO: running FH@100 for 291 with 4403 remaining (b = 0.02)
4783.30/603.35	c INFO: running iPAWS@0 for 97 with 4398 remaining (b = 0.01)
5462.30/688.70	c INFO: running hybridGM3@189 for 97 with 4312 remaining (b = 0.01)
5501.19/693.55	c INFO: running hybridGM3@0 for 194 with 4307 remaining (b = 0.02)
5519.49/695.88	c INFO: running TNM@0 for 1552 with 4305 remaining (b = 0.11)
5581.52/703.68	c INFO: running gnovelty+2@0 for 97 with 4297 remaining (b = 0.01)
6269.17/790.03	c INFO: running hybridGM3@0 for 194 with 4211 remaining (b = 0.01)
6380.64/804.09	c INFO: running adaptg2wsat2009++@100 for 97 with 4197 remaining (b = 0.01)
7036.38/886.47	c INFO: running adaptg2wsat2009++@397 for 194 with 4115 remaining (b = 0.01)
7066.61/890.25	c INFO: running FH@0 for 97 with 4111 remaining (b = 0.01)
7099.32/894.34	c INFO: running hybridGM3@0 for 194 with 4107 remaining (b = 0.01)
7155.71/901.43	c INFO: running adaptg2wsat2009++@389 for 194 with 4100 remaining (b = 0.01)
7859.22/989.83	c INFO: running hybridGM3@99 for 97 with 4011 remaining (b = 0.01)
7864.78/990.60	c INFO: running FH@295 for 97 with 4010 remaining (b = 0.01)
8608.82/1084.06	c INFO: running NCVWr@0 for 485 with 3917 remaining (b = 0.03)
8623.91/1085.92	c INFO: running iPAWS@0 for 97 with 3915 remaining (b = 0.01)
8657.38/1090.18	c INFO: running hybridGM3@0 for 194 with 3911 remaining (b = 0.01)
8663.63/1091.04	c INFO: running hybridGM3@96 for 97 with 3910 remaining (b = 0.01)
8727.98/1099.02	c INFO: running hybridGM3@0 for 194 with 3902 remaining (b = 0.01)
8735.33/1100.08	c INFO: running hybridGM3@0 for 194 with 3901 remaining (b = 0.01)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3385264-1305564964/watcher-3385264-1305564964 -o /tmp/evaluation-result-3385264-1305564964/solver-3385264-1305564964 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385264-1305564964.cnf -seed 437939708 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.00 0.00 0.00 1/321 25337
/proc/meminfo: memFree=8513000/32950928 swapFree=64721852/67111528
[pid=25337] ppid=25335 vsize=11796 CPUtime=0 cores=0-7
/proc/25337/stat : 25337 (runsolver) S 25335 25337 25311 0 -1 4202560 45 0 0 0 0 0 0 0 20 0 1 0 668096322 12079104 52 33554432000 4194304 5432360 140735679822576 18446744073709551615 5055511 0 2147483391 4096 24578 18446744071699199166 0 0 17 6 0 0 0
/proc/25337/statm: 2949 52 33 303 0 2643 0

[startup+0.0869559 s]
/proc/loadavg: 0.00 0.00 0.00 1/321 25337
/proc/meminfo: memFree=8513000/32950928 swapFree=64721852/67111528
[pid=25337] ppid=25335 vsize=15184 CPUtime=0 cores=0-7
/proc/25337/stat : 25337 (python) S 25335 25337 25311 0 -1 4202496 723 0 0 0 0 0 0 0 20 0 1 0 668096322 15548416 614 33554432000 4194304 5683644 140734465662608 18446744073709551615 248089696256 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 0
/proc/25337/statm: 3796 614 321 364 0 283 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 15184

[startup+0.100961 s]
/proc/loadavg: 0.00 0.00 0.00 1/321 25337
/proc/meminfo: memFree=8513000/32950928 swapFree=64721852/67111528
[pid=25337] ppid=25335 vsize=15184 CPUtime=0 cores=0-7
/proc/25337/stat : 25337 (python) S 25335 25337 25311 0 -1 4202496 723 0 0 0 0 0 0 0 20 0 1 0 668096322 15548416 614 33554432000 4194304 5683644 140734465662608 18446744073709551615 248089696256 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 0
/proc/25337/statm: 3796 614 321 364 0 283 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 15184

[startup+0.30092 s]
/proc/loadavg: 0.00 0.00 0.00 1/321 25337
/proc/meminfo: memFree=8513000/32950928 swapFree=64721852/67111528
[pid=25337] ppid=25335 vsize=16396 CPUtime=0.01 cores=0-7
/proc/25337/stat : 25337 (python) S 25335 25337 25311 0 -1 4202496 1110 0 0 0 1 0 0 0 18 0 1 0 668096322 16789504 979 33554432000 4194304 5683644 140734465662608 18446744073709551615 248089694053 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 0
/proc/25337/statm: 4099 979 407 364 0 586 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 16396

[startup+0.700857 s]
/proc/loadavg: 0.00 0.00 0.00 1/321 25337
/proc/meminfo: memFree=8513000/32950928 swapFree=64721852/67111528
[pid=25337] ppid=25335 vsize=113344 CPUtime=0.09 cores=0-7
/proc/25337/stat : 25337 (python) S 25335 25337 25311 0 -1 4202496 3582 0 0 0 6 3 0 0 15 0 1 0 668096322 116064256 3226 33554432000 4194304 5683644 140734465662608 18446744073709551615 248089696256 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 0
/proc/25337/statm: 28337 3227 886 364 0 2359 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 113344

[startup+1.50071 s]
/proc/loadavg: 0.08 0.02 0.01 2/322 25342
/proc/meminfo: memFree=8499604/32950928 swapFree=64721852/67111528
[pid=25337] ppid=25335 vsize=282228 CPUtime=0.47 cores=0-7
/proc/25337/stat : 25337 (python) R 25335 25337 25311 0 -1 4202496 10191 935 1 0 36 11 0 0 15 0 1 0 668096322 289001472 7871 33554432000 4194304 5683644 140734465662608 18446744073709551615 47839212025141 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/25337/statm: 70557 7871 1761 364 0 6481 0
Current children cumulated CPU time (s) 0.47
Current children cumulated vsize (KiB) 282228

[startup+3.10042 s]
/proc/loadavg: 0.08 0.02 0.01 2/322 25342
/proc/meminfo: memFree=8478028/32950928 swapFree=64721852/67111528
[pid=25337] ppid=25335 vsize=290804 CPUtime=2.07 cores=0-7
/proc/25337/stat : 25337 (python) R 25335 25337 25311 0 -1 4202496 14013 935 1 0 195 12 0 0 17 0 1 0 668096322 297783296 10087 33554432000 4194304 5683644 140734465662608 18446744073709551615 4510895 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/25337/statm: 72701 10087 1827 364 0 8625 0
Current children cumulated CPU time (s) 2.07
Current children cumulated vsize (KiB) 290804
heavy processes:

[startup+6.30179 s]
/proc/loadavg: 0.08 0.02 0.01 9/346 25366
/proc/meminfo: memFree=8393748/32950928 swapFree=64721852/67111528
[pid=25337] ppid=25335 vsize=366808 CPUtime=2.48 cores=0-7
/proc/25337/stat : 25337 (python) S 25335 25337 25311 0 -1 4202496 20419 935 1 0 234 14 0 0 25 0 9 0 668096322 375611392 8669 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101524939 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/25337/statm: 91702 8669 1891 364 0 27599 0
[pid=25343] ppid=25337 vsize=284988 CPUtime=0 cores=0-7
/proc/25343/stat : 25343 (python) S 25337 25337 25311 0 -1 4202560 728 0 0 0 0 0 0 0 18 0 1 0 668096652 291827712 6900 33554432000 4194304 5683644 140734465662608 18446744073709551615 248089718287 0 0 16781312 514 3551695515773010180 0 0 17 1 0 0 0
/proc/25343/statm: 71247 6900 355 364 0 7165 0
[pid=25345] ppid=25343 vsize=1020308 CPUtime=2.99 cores=0-7
/proc/25345/stat : 25345 (gnovelty+2) R 25343 25345 25345 0 -1 4202496 1156 0 0 0 299 0 0 0 25 0 1 0 668096652 1044795392 892 33554432000 134512640 135663704 4291075968 18446744073709551615 134544204 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/25345/statm: 255077 892 113 282 0 254793 0
[pid=25346] ppid=25337 vsize=295008 CPUtime=0 cores=0-7
/proc/25346/stat : 25346 (python) S 25337 25337 25311 0 -1 4202560 741 0 0 0 0 0 0 0 18 0 1 0 668096655 302088192 7033 33554432000 4194304 5683644 140734465662608 18446744073709551615 248089718342 0 0 16781312 514 10848137965700350210 0 0 17 6 0 0 0
/proc/25346/statm: 73752 7033 349 364 0 9667 0
[pid=25348] ppid=25346 vsize=1020308 CPUtime=2.95 cores=0-7
/proc/25348/stat : 25348 (gnovelty+2) R 25346 25348 25348 0 -1 4202496 1154 0 0 0 295 0 0 0 25 0 1 0 668096655 1044795392 891 33554432000 134512640 135663704 4288314032 18446744073709551615 134545619 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/25348/statm: 255077 891 113 282 0 254793 0
[pid=25349] ppid=25337 vsize=305016 CPUtime=0 cores=0-7
/proc/25349/stat : 25349 (python) S 25337 25337 25311 0 -1 4202560 736 0 0 0 0 0 0 0 18 0 1 0 668096658 312336384 7035 33554432000 4194304 5683644 140734465662608 18446744073709551615 248089718342 0 0 16781312 514 3551691667482312961 0 0 17 2 0 0 0
/proc/25349/statm: 76254 7035 347 364 0 12166 0
[pid=25351] ppid=25349 vsize=423400 CPUtime=2.92 cores=0-7
/proc/25351/stat : 25351 (adaptg2wsat2009) R 25349 25351 25351 0 -1 4202496 2626 0 0 0 292 0 0 0 25 0 1 0 668096659 433561600 2356 33554432000 134512640 135107411 4289183072 18446744073709551615 134531792 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/25351/statm: 105850 2356 59 146 0 104726 0
[pid=25352] ppid=25337 vsize=316024 CPUtime=0 cores=0-7
/proc/25352/stat : 25352 (python) S 25337 25337 25311 0 -1 4202560 753 0 0 0 0 0 0 0 18 0 1 0 668096661 323608576 7041 33554432000 4194304 5683644 140734465662608 18446744073709551615 248089718342 0 0 16781312 514 10848134117409652993 0 0 17 6 0 0 0
/proc/25352/statm: 79006 7041 349 364 0 14915 0
[pid=25354] ppid=25352 vsize=423400 CPUtime=2.87 cores=0-7
/proc/25354/stat : 25354 (adaptg2wsat2009) R 25352 25354 25354 0 -1 4202496 2622 0 0 0 287 0 0 0 25 0 1 0 668096662 433561600 2357 33554432000 134512640 135107411 4292022992 18446744073709551615 134539318 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/25354/statm: 105850 2357 59 146 0 104726 0
[pid=25355] ppid=25337 vsize=325796 CPUtime=0 cores=0-7
/proc/25355/stat : 25355 (python) S 25337 25337 25311 0 -1 4202560 751 0 0 0 0 0 0 0 18 0 1 0 668096665 333615104 7110 33554432000 4194304 5683644 140734465662608 18446744073709551615 248089718342 0 0 16781312 514 11367179197144334596 0 0 17 0 0 0 0
/proc/25355/statm: 81449 7110 347 364 0 17355 0
[pid=25357] ppid=25355 vsize=423400 CPUtime=2.86 cores=0-7
/proc/25357/stat : 25357 (adaptg2wsat2009) R 25355 25357 25357 0 -1 4202496 2625 0 0 0 286 0 0 0 25 0 1 0 668096665 433561600 2357 33554432000 134512640 135107411 4291687232 18446744073709551615 134531239 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/25357/statm: 105850 2357 59 146 0 104726 0
[pid=25358] ppid=25337 vsize=336052 CPUtime=0 cores=0-7
/proc/25358/stat : 25358 (python) S 25337 25337 25311 0 -1 4202560 751 0 0 0 0 0 0 0 18 0 1 0 668096668 344117248 7113 33554432000 4194304 5683644 140734465662608 18446744073709551615 248089718342 0 0 16781312 514 2778650880516260098 0 0 17 2 0 0 0
/proc/25358/statm: 84013 7113 347 364 0 19916 0
[pid=25360] ppid=25358 vsize=1020308 CPUtime=2.83 cores=0-7
/proc/25360/stat : 25360 (gnovelty+2) R 25358 25360 25360 0 -1 4202496 1156 0 0 0 283 0 0 0 25 0 1 0 668096668 1044795392 892 33554432000 134512640 135663704 4288558800 18446744073709551615 134524897 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/25360/statm: 255077 892 113 282 0 254793 0
[pid=25361] ppid=25337 vsize=346308 CPUtime=0 cores=0-7
/proc/25361/stat : 25361 (python) S 25337 25337 25311 0 -1 4202560 758 0 0 0 0 0 0 0 18 0 1 0 668096671 354619392 7118 33554432000 4194304 5683644 140734465662608 18446744073709551615 248089718342 0 0 16781312 514 15993012356808409345 0 0 17 0 0 0 0
/proc/25361/statm: 86577 7118 347 364 0 22477 0
[pid=25363] ppid=25361 vsize=253032 CPUtime=2.8 cores=0-7
/proc/25363/stat : 25363 (FH) R 25361 25363 25363 0 -1 4202496 2499 0 0 0 280 0 0 0 25 0 1 0 668096671 259104768 2234 33554432000 134512640 134936988 4288667296 18446744073709551615 134553267 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/25363/statm: 63258 2234 41 104 0 62762 0
[pid=25364] ppid=25337 vsize=356564 CPUtime=0 cores=0-7
/proc/25364/stat : 25364 (python) S 25337 25337 25311 0 -1 4202560 797 0 0 0 0 0 0 0 18 0 1 0 668096674 365121536 7128 33554432000 4194304 5683644 140734465662608 18446744073709551615 248089718342 0 0 16781312 514 13803168648958804225 0 0 17 2 0 0 0
/proc/25364/statm: 89141 7128 353 364 0 25038 0
[pid=25366] ppid=25364 vsize=423400 CPUtime=2.72 cores=0-7
/proc/25366/stat : 25366 (adaptg2wsat2009) R 25364 25366 25366 0 -1 4202496 2628 0 0 0 272 0 0 0 25 0 1 0 668096677 433561600 2356 33554432000 134512640 135107411 4294513920 18446744073709551615 134531561 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/25366/statm: 105850 2356 59 146 0 104726 0
[pid=25337/tid=25344] ppid=25335 vsize=366808 CPUtime=0 cores=0-7
/proc/25337/task/25344/stat : 25344 (python) S 25335 25337 25311 0 -1 4202560 25 935 0 0 0 0 0 0 19 0 9 0 668096652 375611392 8669 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 2 0 0 0
[pid=25337/tid=25347] ppid=25335 vsize=366808 CPUtime=0 cores=0-7
/proc/25337/task/25347/stat : 25347 (python) S 25335 25337 25311 0 -1 4202560 25 935 0 0 0 0 0 0 21 0 9 0 668096655 375611392 8669 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 2 0 0 0
[pid=25337/tid=25350] ppid=25335 vsize=366808 CPUtime=0 cores=0-7
/proc/25337/task/25350/stat : 25350 (python) S 25335 25337 25311 0 -1 4202560 18 935 0 0 0 0 0 0 23 0 9 0 668096658 375611392 8669 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 2 0 0 0
[pid=25337/tid=25353] ppid=25335 vsize=366808 CPUtime=0 cores=0-7
/proc/25337/task/25353/stat : 25353 (python) S 25335 25337 25311 0 -1 4202560 23 935 0 0 0 0 0 0 25 0 9 0 668096662 375611392 8669 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 6 0 0 0
[pid=25337/tid=25356] ppid=25335 vsize=366808 CPUtime=0 cores=0-7
/proc/25337/task/25356/stat : 25356 (python) S 25335 25337 25311 0 -1 4202560 25 935 0 0 0 0 0 0 25 0 9 0 668096665 375611392 8669 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 6 0 0 0
[pid=25337/tid=25359] ppid=25335 vsize=366808 CPUtime=0 cores=0-7
/proc/25337/task/25359/stat : 25359 (python) S 25335 25337 25311 0 -1 4202560 26 935 0 0 0 0 0 0 25 0 9 0 668096668 375611392 8669 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 2 0 0 0
[pid=25337/tid=25362] ppid=25335 vsize=366808 CPUtime=0 cores=0-7
/proc/25337/task/25362/stat : 25362 (python) S 25335 25337 25311 0 -1 4202560 27 935 0 0 0 0 0 0 25 0 9 0 668096671 375611392 8669 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 0 0 0 0
[pid=25337/tid=25365] ppid=25335 vsize=366808 CPUtime=0 cores=0-7
/proc/25337/task/25365/stat : 25365 (python) S 25335 25337 25311 0 -1 4202560 27 935 0 0 0 0 0 0 25 0 9 0 668096674 375611392 8669 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 25.42
Current children cumulated vsize (KiB) 7940120

[startup+12.7017 s]
/proc/loadavg: 1.30 0.28 0.09 9/346 25366
/proc/meminfo: memFree=8391624/32950928 swapFree=64721852/67111528
[pid=25337] ppid=25335 vsize=366808 CPUtime=2.48 cores=0-7
/proc/25337/stat : 25337 (python) S 25335 25337 25311 0 -1 4202496 20419 935 1 0 234 14 0 0 25 0 9 0 668096322 375611392 8669 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101524939 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/25337/statm: 91702 8669 1891 364 0 27599 0
[pid=25343] ppid=25337 vsize=284988 CPUtime=0.01 cores=0-7
/proc/25343/stat : 25343 (python) S 25337 25337 25311 0 -1 4202560 884 0 0 0 1 0 0 0 18 0 1 0 668096652 291827712 6923 33554432000 4194304 5683644 140734465662608 18446744073709551615 248089718287 0 0 16781312 514 3551695515773010180 0 0 17 1 0 0 0
/proc/25343/statm: 71247 6923 378 364 0 7165 0
[pid=25345] ppid=25343 vsize=1020308 CPUtime=9.39 cores=0-7
/proc/25345/stat : 25345 (gnovelty+2) R 25343 25345 25345 0 -1 4202496 1156 0 0 0 939 0 0 0 25 0 1 0 668096652 1044795392 892 33554432000 134512640 135663704 4291075968 18446744073709551615 134545619 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/25345/statm: 255077 892 113 282 0 254793 0
[pid=25346] ppid=25337 vsize=295008 CPUtime=0 cores=0-7
/proc/25346/stat : 25346 (python) S 25337 25337 25311 0 -1 4202560 905 0 0 0 0 0 0 0 18 0 1 0 668096655 302088192 7056 33554432000 4194304 5683644 140734465662608 18446744073709551615 248089718342 0 0 16781312 514 10848137965700350210 0 0 17 2 0 0 0
/proc/25346/statm: 73752 7056 372 364 0 9667 0
[pid=25348] ppid=25346 vsize=1020308 CPUtime=9.35 cores=0-7
/proc/25348/stat : 25348 (gnovelty+2) R 25346 25348 25348 0 -1 4202496 1154 0 0 0 935 0 0 0 25 0 1 0 668096655 1044795392 891 33554432000 134512640 135663704 4288314032 18446744073709551615 134524897 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/25348/statm: 255077 891 113 282 0 254793 0
[pid=25349] ppid=25337 vsize=305016 CPUtime=0 cores=0-7
/proc/25349/stat : 25349 (python) S 25337 25337 25311 0 -1 4202560 887 0 0 0 0 0 0 0 18 0 1 0 668096658 312336384 7057 33554432000 4194304 5683644 140734465662608 18446744073709551615 248089718342 0 0 16781312 514 3551691667482312961 0 0 17 2 0 0 0
/proc/25349/statm: 76254 7057 369 364 0 12166 0
[pid=25351] ppid=25349 vsize=423400 CPUtime=9.32 cores=0-7
/proc/25351/stat : 25351 (adaptg2wsat2009) R 25349 25351 25351 0 -1 4202496 2626 0 0 0 932 0 0 0 25 0 1 0 668096659 433561600 2356 33554432000 134512640 135107411 4289183072 18446744073709551615 134531928 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/25351/statm: 105850 2356 59 146 0 104726 0
[pid=25352] ppid=25337 vsize=316024 CPUtime=0 cores=0-7
/proc/25352/stat : 25352 (python) S 25337 25337 25311 0 -1 4202560 932 0 0 0 0 0 0 0 18 0 1 0 668096661 323608576 7064 33554432000 4194304 5683644 140734465662608 18446744073709551615 248089718342 0 0 16781312 514 10848134117409652993 0 0 17 6 0 0 0
/proc/25352/statm: 79006 7064 372 364 0 14915 0
[pid=25354] ppid=25352 vsize=423400 CPUtime=9.19 cores=0-7
/proc/25354/stat : 25354 (adaptg2wsat2009) R 25352 25354 25354 0 -1 4202496 2622 0 0 0 919 0 0 0 25 0 1 0 668096662 433561600 2357 33554432000 134512640 135107411 4292022992 18446744073709551615 134532076 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/25354/statm: 105850 2357 59 146 0 104726 0
[pid=25355] ppid=25337 vsize=325796 CPUtime=0.01 cores=0-7
/proc/25355/stat : 25355 (python) S 25337 25337 25311 0 -1 4202560 925 0 0 0 1 0 0 0 18 0 1 0 668096665 333615104 7133 33554432000 4194304 5683644 140734465662608 18446744073709551615 248089718342 0 0 16781312 514 11367179197144334596 0 0 17 0 0 0 0
/proc/25355/statm: 81449 7133 370 364 0 17355 0
[pid=25357] ppid=25355 vsize=423400 CPUtime=9.23 cores=0-7
/proc/25357/stat : 25357 (adaptg2wsat2009) R 25355 25357 25357 0 -1 4202496 2625 0 0 0 923 0 0 0 25 0 1 0 668096665 433561600 2357 33554432000 134512640 135107411 4291687232 18446744073709551615 134531584 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/25357/statm: 105850 2357 59 146 0 104726 0
[pid=25358] ppid=25337 vsize=336052 CPUtime=0.01 cores=0-7
/proc/25358/stat : 25358 (python) S 25337 25337 25311 0 -1 4202560 923 0 0 0 0 1 0 0 18 0 1 0 668096668 344117248 7136 33554432000 4194304 5683644 140734465662608 18446744073709551615 248089718342 0 0 16781312 514 2778650880516260098 0 0 17 2 0 0 0
/proc/25358/statm: 84013 7136 370 364 0 19916 0
[pid=25360] ppid=25358 vsize=1020308 CPUtime=9.21 cores=0-7
/proc/25360/stat : 25360 (gnovelty+2) R 25358 25360 25360 0 -1 4202496 1156 0 0 0 921 0 0 0 25 0 1 0 668096668 1044795392 892 33554432000 134512640 135663704 4288558800 18446744073709551615 134525302 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/25360/statm: 255077 892 113 282 0 254793 0
[pid=25361] ppid=25337 vsize=346308 CPUtime=0 cores=0-7
/proc/25361/stat : 25361 (python) S 25337 25337 25311 0 -1 4202560 940 0 0 0 0 0 0 0 18 0 1 0 668096671 354619392 7140 33554432000 4194304 5683644 140734465662608 18446744073709551615 248089718342 0 0 16781312 514 15993012356808409345 0 0 17 0 0 0 0

################
# More data... #
################

[pid=25452] ppid=25337 vsize=576668 CPUtime=0.26 cores=0-7
/proc/25452/stat : 25452 (python) S 25337 25337 25311 0 -1 4202560 1285 0 0 0 12 14 0 0 19 0 2 0 668146625 590508032 7651 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/25452/statm: 144167 7651 406 364 0 80019 0
[pid=25454] ppid=25452 vsize=335996 CPUtime=99.18 cores=0-7
/proc/25454/stat : 25454 (iPAWS) T 25452 25454 25454 0 -1 4202496 2054 0 0 0 9918 0 0 0 25 0 1 0 668146626 344059904 1774 33554432000 134512640 135114423 4288744832 18446744073709551615 134516669 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/25454/statm: 83999 1774 59 147 0 83851 0
[pid=25452/tid=25460] ppid=25337 vsize=576668 CPUtime=0 cores=0-7
/proc/25452/task/25460/stat : 25460 (python) S 25337 25337 25311 0 -1 4202560 214 0 0 0 0 0 0 0 19 0 2 0 668156651 590508032 7651 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 514 18446604471134454848 0 0 -1 3 0 0 0
[pid=25461] ppid=25337 vsize=586924 CPUtime=0.27 cores=0-7
/proc/25461/stat : 25461 (python) S 25337 25337 25311 0 -1 4202560 1184 0 0 0 14 13 0 0 19 0 2 0 668156657 601010176 7660 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/25461/statm: 146731 7660 407 364 0 82580 0
[pid=25463] ppid=25461 vsize=335996 CPUtime=99.2 cores=0-7
/proc/25463/stat : 25463 (iPAWS) T 25461 25463 25463 0 -1 4202496 2054 0 0 0 9920 0 0 0 25 0 1 0 668156658 344059904 1774 33554432000 134512640 135114423 4289375696 18446744073709551615 134516698 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/25463/statm: 83999 1774 59 147 0 83851 0
[pid=25461/tid=25472] ppid=25337 vsize=586924 CPUtime=0 cores=0-7
/proc/25461/task/25472/stat : 25472 (python) S 25337 25337 25311 0 -1 4202560 97 0 0 0 0 0 0 0 19 0 2 0 668166684 601010176 7660 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 514 18446604455982418112 0 0 -1 0 0 0 0
[pid=25466] ppid=25337 vsize=597572 CPUtime=0.55 cores=0-7
/proc/25466/stat : 25466 (python) S 25337 25337 25311 0 -1 4202560 1282 0 0 0 29 26 0 0 19 0 2 0 668165677 611913728 7659 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/25466/statm: 149393 7659 401 364 0 85239 0
[pid=25468] ppid=25466 vsize=268424 CPUtime=195.28 cores=0-7
/proc/25468/stat : 25468 (hybridGM3) T 25466 25468 25468 0 -1 4202496 6209 0 0 0 19528 0 0 0 25 0 1 0 668165678 274866176 5918 33554432000 134512640 135204385 4290858384 18446744073709551615 134538013 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/25468/statm: 67106 5918 63 169 0 66936 0
[pid=25466/tid=25485] ppid=25337 vsize=597572 CPUtime=0 cores=0-7
/proc/25466/task/25485/stat : 25485 (python) S 25337 25337 25311 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 668185341 611913728 7659 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 514 18446604459632371008 0 0 -1 1 0 0 0
[pid=25469] ppid=25337 vsize=608032 CPUtime=1.13 cores=0-7
/proc/25469/stat : 25469 (python) S 25337 25337 25311 0 -1 4202560 1097 0 0 0 58 55 0 0 18 0 1 0 668165910 622624768 7634 33554432000 4194304 5683644 140734465662608 18446744073709551615 248089718342 0 0 16781312 514 7062981221856280837 0 0 17 5 0 0 0
/proc/25469/statm: 152008 7634 373 364 0 87851 0
[pid=25471] ppid=25469 vsize=461504 CPUtime=403.59 cores=0-7
/proc/25471/stat : 25471 (TNM) R 25469 25471 25471 0 -1 4202496 2479 0 0 0 40358 1 0 0 25 0 1 0 668165911 472580096 2194 33554432000 134512640 135008748 4290104384 18446744073709551615 134527750 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/25471/statm: 115376 2194 46 122 0 114276 0
[pid=25473] ppid=25337 vsize=618176 CPUtime=0.29 cores=0-7
/proc/25473/stat : 25473 (python) S 25337 25337 25311 0 -1 4202560 1252 0 0 0 17 12 0 0 19 0 2 0 668166690 633012224 7686 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/25473/statm: 154544 7686 401 364 0 90384 0
[pid=25475] ppid=25473 vsize=1020308 CPUtime=99.84 cores=0-7
/proc/25475/stat : 25475 (gnovelty+2) T 25473 25475 25475 0 -1 4202496 1176 0 0 0 9984 0 0 0 25 0 1 0 668166692 1044795392 892 33554432000 134512640 135663704 4291954112 18446744073709551615 134544183 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/25475/statm: 255077 892 113 282 0 254793 0
[pid=25473/tid=25482] ppid=25337 vsize=618176 CPUtime=0 cores=0-7
/proc/25473/task/25482/stat : 25482 (python) S 25337 25337 25311 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 668176725 633012224 7686 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 514 18446604467646007488 0 0 -1 3 0 0 0
[pid=25479] ppid=25337 vsize=628648 CPUtime=0.86 cores=0-7
/proc/25479/stat : 25479 (python) S 25337 25337 25311 0 -1 4202560 1270 0 0 0 45 41 0 0 18 0 2 0 668175325 643735552 7691 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/25479/statm: 157162 7691 402 364 0 92999 0
[pid=25481] ppid=25479 vsize=268424 CPUtime=295.72 cores=0-7
/proc/25481/stat : 25481 (hybridGM3) T 25479 25481 25481 0 -1 4202496 6209 0 0 0 29571 1 0 0 25 0 1 0 668175326 274866176 5920 33554432000 134512640 135204385 4294807792 18446744073709551615 134537724 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/25481/statm: 67106 5920 63 169 0 66936 0
[pid=25479/tid=25495] ppid=25337 vsize=628648 CPUtime=0 cores=0-7
/proc/25479/task/25495/stat : 25495 (python) S 25337 25337 25311 0 -1 4202560 46 0 0 0 0 0 0 0 18 0 2 0 668195299 643735552 7691 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 514 18446604469792289792 0 0 -1 7 0 0 0
[pid=25486] ppid=25337 vsize=638204 CPUtime=0.56 cores=0-7
/proc/25486/stat : 25486 (python) S 25337 25337 25311 0 -1 4202560 1282 0 0 0 29 27 0 0 18 0 2 0 668185347 653520896 7780 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101524939 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/25486/statm: 159551 7780 406 364 0 95385 0
[pid=25488] ppid=25486 vsize=253036 CPUtime=194.3 cores=0-7
/proc/25488/stat : 25488 (FH) T 25486 25488 25488 0 -1 4202496 2525 0 0 0 19430 0 0 0 25 0 1 0 668185348 259108864 2233 33554432000 134512640 134936988 4289837616 18446744073709551615 134552455 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/25488/statm: 63259 2233 41 104 0 62763 0
[pid=25486/tid=25496] ppid=25337 vsize=638204 CPUtime=0 cores=0-7
/proc/25486/task/25496/stat : 25496 (python) S 25337 25337 25311 0 -1 4202560 146 0 0 0 0 0 0 0 18 0 2 0 668195375 653520896 7780 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 514 18446604456659077056 0 0 -1 0 0 0 0
[pid=25489] ppid=25337 vsize=648460 CPUtime=0.57 cores=0-7
/proc/25489/stat : 25489 (python) S 25337 25337 25311 0 -1 4202560 1235 0 0 0 29 28 0 0 18 0 2 0 668185756 664023040 7874 33554432000 4194304 5683644 140734465662608 18446744073709551615 248089718342 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/25489/statm: 162115 7874 406 364 0 97946 0
[pid=25491] ppid=25489 vsize=268424 CPUtime=205.41 cores=0-7
/proc/25491/stat : 25491 (hybridGM3) R 25489 25491 25491 0 -1 4202496 6203 0 0 0 20540 1 0 0 25 0 1 0 668185757 274866176 5918 33554432000 134512640 135204385 4291816736 18446744073709551615 134538435 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/25491/statm: 67106 5918 63 169 0 66936 0
[pid=25489/tid=25509] ppid=25337 vsize=648460 CPUtime=0 cores=0-7
/proc/25489/task/25509/stat : 25509 (python) S 25337 25337 25311 0 -1 4202560 100 0 0 0 0 0 0 0 19 0 2 0 668205419 664023040 7874 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 514 18446604453966995840 0 0 -1 6 0 0 0
[pid=25500] ppid=25337 vsize=658716 CPUtime=0.03 cores=0-7
/proc/25500/stat : 25500 (python) S 25337 25337 25311 0 -1 4202560 1027 0 0 0 1 2 0 0 18 0 1 0 668204728 674525184 7876 33554432000 4194304 5683644 140734465662608 18446744073709551615 248089718342 0 0 16781312 514 7062981221856280837 0 0 17 6 0 0 0
/proc/25500/statm: 164679 7876 379 364 0 100507 0
[pid=25502] ppid=25500 vsize=423308 CPUtime=15.09 cores=0-7
/proc/25502/stat : 25502 (NCVWr) R 25500 25502 25502 0 -1 4202496 2550 0 0 0 1509 0 0 0 25 0 1 0 668204729 433467392 2265 33554432000 134512640 135113558 4290827376 18446744073709551615 134533743 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/25502/statm: 105827 2265 58 147 0 104702 0
[pid=25503] ppid=25337 vsize=668972 CPUtime=0.02 cores=0-7
/proc/25503/stat : 25503 (python) S 25337 25337 25311 0 -1 4202560 1035 0 0 0 1 1 0 0 18 0 1 0 668204914 685027328 7896 33554432000 4194304 5683644 140734465662608 18446744073709551615 248089718342 0 0 16781312 514 7062981221856280837 0 0 17 3 0 0 0
/proc/25503/statm: 167243 7896 378 364 0 103068 0
[pid=25505] ppid=25503 vsize=335996 CPUtime=13.91 cores=0-7
/proc/25505/stat : 25505 (iPAWS) R 25503 25505 25505 0 -1 4202496 2062 0 0 0 1391 0 0 0 25 0 1 0 668204914 344059904 1773 33554432000 134512640 135114423 4288559296 18446744073709551615 134546153 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/25505/statm: 83999 1773 59 147 0 83851 0
[pid=25506] ppid=25337 vsize=679228 CPUtime=0.01 cores=0-7
/proc/25506/stat : 25506 (python) S 25337 25337 25311 0 -1 4202560 1040 0 0 0 1 0 0 0 18 0 1 0 668205340 695529472 7979 33554432000 4194304 5683644 140734465662608 18446744073709551615 248089718342 0 0 16781312 514 2923242432004849925 0 0 17 6 0 0 0
/proc/25506/statm: 169807 7979 379 364 0 105629 0
[pid=25508] ppid=25506 vsize=268428 CPUtime=9.72 cores=0-7
/proc/25508/stat : 25508 (hybridGM3) R 25506 25508 25508 0 -1 4202496 6208 0 0 0 972 0 0 0 25 0 1 0 668205340 274870272 5919 33554432000 134512640 135204385 4293921072 18446744073709551615 134538013 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/25508/statm: 67107 5919 63 169 0 66937 0
[pid=25510] ppid=25337 vsize=690392 CPUtime=0 cores=0-7
/proc/25510/stat : 25510 (python) S 25337 25337 25311 0 -1 4202560 956 0 0 0 0 0 0 0 19 0 1 0 668206224 706961408 7860 33554432000 4194304 5683644 140734465662608 18446744073709551615 248089718342 0 0 16781312 514 15498752743064993791 0 0 17 6 0 0 0
/proc/25510/statm: 172598 7860 351 364 0 108417 0
[pid=25512] ppid=25510 vsize=268420 CPUtime=0.89 cores=0-7
/proc/25512/stat : 25512 (hybridGM3) R 25510 25512 25512 0 -1 4202496 6210 0 0 0 88 1 0 0 23 0 1 0 668206225 274862080 5918 33554432000 134512640 135204385 4286783536 18446744073709551615 134537785 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/25512/statm: 67105 5918 63 169 0 66935 0
[pid=25337/tid=25344] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25344/stat : 25344 (python) S 25335 25337 25311 0 -1 4202560 25 935 0 0 0 0 0 0 19 0 35 0 668096652 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 2 0 0 0
[pid=25337/tid=25347] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25347/stat : 25347 (python) S 25335 25337 25311 0 -1 4202560 25 935 0 0 0 0 0 0 21 0 35 0 668096655 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 2 0 0 0
[pid=25337/tid=25350] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25350/stat : 25350 (python) S 25335 25337 25311 0 -1 4202560 22 935 0 0 0 0 0 0 18 0 35 0 668096658 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 2 0 0 0
[pid=25337/tid=25353] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25353/stat : 25353 (python) S 25335 25337 25311 0 -1 4202560 28 935 0 0 0 0 0 0 18 0 35 0 668096662 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 6 0 0 0
[pid=25337/tid=25356] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25356/stat : 25356 (python) S 25335 25337 25311 0 -1 4202560 29 935 0 0 0 0 0 0 18 0 35 0 668096665 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 6 0 0 0
[pid=25337/tid=25359] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25359/stat : 25359 (python) S 25335 25337 25311 0 -1 4202560 26 935 0 0 0 0 0 0 25 0 35 0 668096668 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 2 0 0 0
[pid=25337/tid=25362] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25362/stat : 25362 (python) S 25335 25337 25311 0 -1 4202560 30 935 0 0 0 0 0 0 18 0 35 0 668096671 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 0 0 0 0
[pid=25337/tid=25365] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25365/stat : 25365 (python) S 25335 25337 25311 0 -1 4202560 39 935 0 0 0 0 0 0 18 0 35 0 668096674 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 2 0 0 0
[pid=25337/tid=25374] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25374/stat : 25374 (python) S 25335 25337 25311 0 -1 4202560 41 935 0 0 0 0 0 0 18 0 35 0 668106685 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 3 0 0 0
[pid=25337/tid=25377] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25377/stat : 25377 (python) S 25335 25337 25311 0 -1 4202560 30 935 0 0 0 0 0 0 18 0 35 0 668106689 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 3 0 0 0
[pid=25337/tid=25381] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25381/stat : 25381 (python) S 25335 25337 25311 0 -1 4202560 35 935 0 0 0 0 0 0 18 0 35 0 668106693 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 3 0 0 0
[pid=25337/tid=25384] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25384/stat : 25384 (python) S 25335 25337 25311 0 -1 4202560 23 935 0 0 0 0 0 0 20 0 35 0 668106697 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 1 0 0 0
[pid=25337/tid=25389] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25389/stat : 25389 (python) S 25335 25337 25311 0 -1 4202560 29 935 0 0 0 0 0 0 18 0 35 0 668106702 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 1 0 0 0
[pid=25337/tid=25393] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25393/stat : 25393 (python) S 25335 25337 25311 0 -1 4202560 33 935 0 0 0 0 0 0 18 0 35 0 668106707 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 7 0 0 0
[pid=25337/tid=25396] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25396/stat : 25396 (python) S 25335 25337 25311 0 -1 4202560 36 935 0 0 0 0 0 0 18 0 35 0 668106711 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 3 0 0 0
[pid=25337/tid=25399] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25399/stat : 25399 (python) S 25335 25337 25311 0 -1 4202560 31 935 0 0 0 0 0 0 18 0 35 0 668106716 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 4 0 0 0
[pid=25337/tid=25406] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25406/stat : 25406 (python) S 25335 25337 25311 0 -1 4202560 28 935 0 0 0 0 0 0 20 0 35 0 668116423 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 1 0 0 0
[pid=25337/tid=25410] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25410/stat : 25410 (python) S 25335 25337 25311 0 -1 4202560 28 935 0 0 0 0 0 0 20 0 35 0 668116633 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 1 0 0 0
[pid=25337/tid=25432] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25432/stat : 25432 (python) S 25335 25337 25311 0 -1 4202560 39 935 0 0 0 0 0 0 18 0 35 0 668127054 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 1 0 0 0
[pid=25337/tid=25437] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25437/stat : 25437 (python) S 25335 25337 25311 0 -1 4202560 34 935 0 0 0 0 0 0 18 0 35 0 668135895 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 1 0 0 0
[pid=25337/tid=25440] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25440/stat : 25440 (python) S 25335 25337 25311 0 -1 4202560 32 935 0 0 0 0 0 0 20 0 35 0 668136103 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 1 0 0 0
[pid=25337/tid=25449] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25449/stat : 25449 (python) S 25335 25337 25311 0 -1 4202560 31 935 0 0 0 0 0 0 20 0 35 0 668145950 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 0 0 0 0
[pid=25337/tid=25453] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25453/stat : 25453 (python) S 25335 25337 25311 0 -1 4202560 176 935 0 0 0 0 0 0 19 0 35 0 668146625 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 1 0 0 0
[pid=25337/tid=25462] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25462/stat : 25462 (python) S 25335 25337 25311 0 -1 4202560 43 935 0 0 0 0 0 0 19 0 35 0 668156657 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 1 0 0 0
[pid=25337/tid=25467] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25467/stat : 25467 (python) S 25335 25337 25311 0 -1 4202560 36 935 0 0 0 0 0 0 20 0 35 0 668165678 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 0 0 0 0
[pid=25337/tid=25470] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25470/stat : 25470 (python) S 25335 25337 25311 0 -1 4202560 25 935 0 0 0 0 0 0 20 0 35 0 668165911 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 5 0 0 0
[pid=25337/tid=25474] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25474/stat : 25474 (python) S 25335 25337 25311 0 -1 4202560 25 935 0 0 0 0 0 0 19 0 35 0 668166690 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 5 0 0 0
[pid=25337/tid=25480] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25480/stat : 25480 (python) S 25335 25337 25311 0 -1 4202560 32 935 0 0 0 0 0 0 18 0 35 0 668175326 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 0 0 0 0
[pid=25337/tid=25487] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25487/stat : 25487 (python) S 25335 25337 25311 0 -1 4202560 92 935 0 0 0 0 0 0 18 0 35 0 668185348 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 0 0 0 0
[pid=25337/tid=25490] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25490/stat : 25490 (python) S 25335 25337 25311 0 -1 4202560 37 935 0 0 0 0 0 0 18 0 35 0 668185757 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 6 0 0 0
[pid=25337/tid=25501] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25501/stat : 25501 (python) S 25335 25337 25311 0 -1 4202560 41 935 0 0 0 0 0 0 20 0 35 0 668204729 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 1 0 0 0
[pid=25337/tid=25504] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25504/stat : 25504 (python) S 25335 25337 25311 0 -1 4202560 37 935 0 0 0 0 0 0 20 0 35 0 668204914 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 6 0 0 0
[pid=25337/tid=25507] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25507/stat : 25507 (python) S 25335 25337 25311 0 -1 4202560 36 935 0 0 0 0 0 0 20 0 35 0 668205340 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 6 0 0 0
[pid=25337/tid=25511] ppid=25335 vsize=699728 CPUtime=0 cores=0-7
/proc/25337/task/25511/stat : 25511 (python) S 25335 25337 25311 0 -1 4202560 29 935 0 0 0 0 0 0 20 0 35 0 668206225 716521472 9456 33554432000 4194304 5683644 140734465662608 18446744073709551615 248101521841 0 0 16781312 2 18446604471018560256 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 8735.33
Current children cumulated vsize (KiB) 32592612

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 25337 and gives
#  childrusage.ru_utime.tv_sec=1795
#  childrusage.ru_utime.tv_usec=985968
#  childrusage.ru_stime.tv_sec=11
#  childrusage.ru_stime.tv_usec=957182
# CPU time returned by wait4() is 1807.94
# while last known CPU time is 8735.33
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1100.11
CPU time (s): 8735.33
CPU user time (s): 8723.45
CPU system time (s): 11.88
CPU usage (%): 794.038
Max. virtual memory (cumulated for all children) (KiB): 32592612

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1795.99
system time used= 11.9572
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 102936
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 15878
involuntary context switches= 16209

runsolver used 10.9553 second user time and 25.9041 second system time

The end

Launcher Data 

Begin job on node123 at 2011-05-16 18:56:05
IDJOB=3385264
IDBENCH=82529
IDSOLVER=1870
FILE ID=node123/3385264-1305564964
RUNJOBID= node123-1305564964-25325
PBS_JOBID= 13350127
Free space on /tmp= 73288 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/large/unif-k5-r20-v2000-c40000-S656892101-075-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3385264-1305564964/watcher-3385264-1305564964 -o /tmp/evaluation-result-3385264-1305564964/solver-3385264-1305564964 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385264-1305564964.cnf -seed 437939708 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 187184c44e9c5a07f5a8f762e04588fa
RANDOM SEED=437939708

node123.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:       8513404 kB
Buffers:        338332 kB
Cached:        4832540 kB
SwapCached:      10524 kB
Active:       20734672 kB
Inactive:      3398300 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:       8513404 kB
SwapTotal:    67111528 kB
SwapFree:     64721852 kB
Dirty:           32620 kB
Writeback:           0 kB
AnonPages:    18959616 kB
Mapped:          24656 kB
Slab:           180716 kB
PageTables:      59776 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 69836308 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73256 MiB
End job on node123 at 2011-05-16 19:14:27