Trace number 3385398

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)? (MO) 11056.7 1391.62

General information on the benchmark

Namerandom/large/
unif-k7-r85-v400-c34000-S536077088-015-UNKNOWN.cnf
MD5SUM7eb6d5b5c1c99a6b88954b61ac97b4f7
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables400
Number of clauses34000
Sum of the clauses size238000
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 534000

Solver Data 

0.32/0.47	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.32/0.47	c INFO: solving HOME/instance-3385398-1305566247.cnf
0.32/0.47	c INFO: computing task features
0.61/0.70	c INFO: computed variable-clause graph statistics
0.61/0.70	c INFO: computed clause balance statistics
1.81/1.91	c INFO: computed variable balance statistics
2.21/2.39	c INFO: computed small-clause counts
2.31/2.40	c INFO: computed Horn-clause counts
2.31/2.44	c INFO: computed variable graph statistics
2.31/2.44	c INFO: collected features for HOME/instance-3385398-1305566247.cnf in 1.95 s
2.31/2.47	c INFO: running gnovelty+2@0 for 97 with 4998 remaining (b = 0.38)
2.31/2.50	c INFO: running gnovelty+2@0 for 97 with 4997 remaining (b = 0.13)
2.41/2.53	c INFO: running adaptg2wsat2009++@0 for 97 with 4997 remaining (b = 0.08)
2.41/2.56	c INFO: running adaptg2wsat2009++@0 for 97 with 4997 remaining (b = 0.07)
2.41/2.59	c INFO: running adaptg2wsat2009++@0 for 97 with 4997 remaining (b = 0.05)
2.50/2.62	c INFO: running gnovelty+2@0 for 97 with 4997 remaining (b = 0.04)
2.50/2.65	c INFO: running FH@0 for 97 with 4997 remaining (b = 0.03)
2.50/2.68	c INFO: running adaptg2wsat2009++@0 for 97 with 4997 remaining (b = 0.03)
794.63/101.88	c INFO: running adaptg2wsat2009++@0 for 97 with 4898 remaining (b = 0.02)
802.59/102.87	c INFO: running FH@0 for 97 with 4897 remaining (b = 0.02)
803.35/102.92	c INFO: running hybridGM3@0 for 194 with 4897 remaining (b = 0.04)
803.35/102.96	c INFO: running gnovelty+2@0 for 97 with 4897 remaining (b = 0.02)
803.86/103.00	c INFO: running hybridGM3@0 for 194 with 4897 remaining (b = 0.03)
803.86/103.05	c INFO: running adaptg2wsat2009++@0 for 97 with 4897 remaining (b = 0.01)
803.86/103.10	c INFO: running hybridGM3@0 for 194 with 4897 remaining (b = 0.03)
804.11/103.15	c INFO: running hybridGM3@0 for 194 with 4897 remaining (b = 0.02)
1594.70/202.26	c INFO: running TNM@0 for 1552 with 4798 remaining (b = 0.18)
1602.63/203.25	c INFO: running FH@0 for 97 with 4797 remaining (b = 0.01)
1603.38/203.41	c INFO: running adaptg2wsat2009++@0 for 97 with 4797 remaining (b = 0.01)
1604.26/203.46	c INFO: running adaptg2wsat2009++@0 for 97 with 4797 remaining (b = 0.01)
2356.07/297.83	c INFO: running hybridGM3@99 for 97 with 4702 remaining (b = 0.01)
2370.42/299.68	c INFO: running hybridGM3@100 for 97 with 4700 remaining (b = 0.01)
2379.97/300.81	c INFO: running adaptg2wsat2009++@0 for 97 with 4699 remaining (b = 0.01)
2379.97/300.86	c INFO: running adaptg2wsat2009++@99 for 97 with 4699 remaining (b = 0.01)
2394.29/302.68	c INFO: running hybridGM3@0 for 194 with 4697 remaining (b = 0.02)
2396.42/303.00	c INFO: running hybridGM3@98 for 97 with 4697 remaining (b = 0.01)
2397.13/303.09	c INFO: running hybridGM3@100 for 97 with 4697 remaining (b = 0.01)
3155.04/398.20	c INFO: running hybridGM3@0 for 194 with 4602 remaining (b = 0.02)
3155.81/398.26	c INFO: running hybridGM3@295 for 97 with 4602 remaining (b = 0.01)
3177.99/401.06	c INFO: running TNM@0 for 1552 with 4599 remaining (b = 0.14)
3203.46/404.26	c INFO: running adaptg2wsat2009++@100 for 97 with 4596 remaining (b = 0.01)
3204.25/404.38	c INFO: running hybridGM3@293 for 97 with 4596 remaining (b = 0.01)
3205.03/404.47	c INFO: running iPAWS@0 for 97 with 4596 remaining (b = 0.01)
3907.50/492.68	c INFO: running hybridGM3@396 for 97 with 4507 remaining (b = 0.01)
3964.05/499.75	c INFO: running gnovelty+2@0 for 97 with 4500 remaining (b = 0.01)
3979.28/501.64	c INFO: running hybridGM3@394 for 97 with 4498 remaining (b = 0.01)
4004.74/504.88	c INFO: running FH@0 for 291 with 4495 remaining (b = 0.02)
4009.40/505.49	c INFO: running adaptg2wsat2009++@98 for 97 with 4495 remaining (b = 0.01)
4714.78/594.06	c INFO: running hybridGM3@100 for 97 with 4406 remaining (b = 0.01)
4730.56/596.02	c INFO: running FH@100 for 291 with 4404 remaining (b = 0.02)
4755.20/599.13	c INFO: running iPAWS@0 for 97 with 4401 remaining (b = 0.01)
4755.95/599.23	c INFO: running adaptg2wsat2009++@99 for 97 with 4401 remaining (b = 0.01)
4784.72/602.89	c INFO: running hybridGM3@0 for 194 with 4397 remaining (b = 0.01)
5521.29/695.46	c INFO: running TNM@0 for 1552 with 4305 remaining (b = 0.11)
5530.79/696.64	c INFO: running adaptg2wsat2009++@293 for 291 with 4303 remaining (b = 0.02)
5553.97/699.54	c INFO: running FH@0 for 97 with 4300 remaining (b = 0.01)
6312.76/794.91	c INFO: running hybridGM3@0 for 194 with 4205 remaining (b = 0.01)
6344.54/798.96	c INFO: running NCVWr@0 for 485 with 4201 remaining (b = 0.03)
6398.62/805.73	c INFO: running hybridGM3@101 for 97 with 4194 remaining (b = 0.01)
7085.25/892.07	c INFO: running FH@198 for 194 with 4108 remaining (b = 0.01)
7157.53/901.11	c INFO: running hybridGM3@395 for 97 with 4099 remaining (b = 0.01)
7837.82/986.67	c INFO: running adaptg2wsat2009++@393 for 194 with 4013 remaining (b = 0.01)
7964.14/1002.51	c INFO: running iPAWS@0 for 97 with 3998 remaining (b = 0.01)
7965.56/1002.80	c INFO: running hybridGM3@99 for 97 with 3997 remaining (b = 0.01)
8674.45/1091.81	c INFO: running FH@292 for 97 with 3908 remaining (b = 0.01)
8724.58/1098.18	c INFO: running hybridGM3@298 for 194 with 3902 remaining (b = 0.01)
8762.73/1102.92	c INFO: running FH@288 for 194 with 3897 remaining (b = 0.01)
9432.43/1187.20	c INFO: running FH@386 for 97 with 3813 remaining (b = 0.01)
9458.67/1190.41	c INFO: running adaptg2wsat2009++@493 for 97 with 3810 remaining (b = 0.01)
10238.95/1288.60	c INFO: running hybridGM3@0 for 194 with 3711 remaining (b = 0.01)
10247.65/1289.68	c INFO: running NCVWr@196 for 291 with 3710 remaining (b = 0.02)
10265.11/1291.81	c INFO: running TNM@0 for 1552 with 3708 remaining (b = 0.08)
10295.29/1295.65	c INFO: running FH@383 for 97 with 3704 remaining (b = 0.01)
10297.67/1295.92	c INFO: running hybridGM3@398 for 97 with 3704 remaining (b = 0.01)
11055.69/1391.31	c INFO: running NCVWr@0 for 485 with 3609 remaining (b = 0.03)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3385398-1305566247/watcher-3385398-1305566247 -o /tmp/evaluation-result-3385398-1305566247/solver-3385398-1305566247 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385398-1305566247.cnf -seed 930569595 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.93 8.22 6.14 2/417 30112
/proc/meminfo: memFree=11239892/32951124 swapFree=56316216/67111528
[pid=30112] ppid=30110 vsize=16240 CPUtime=0 cores=0-7
/proc/30112/stat : 30112 (python) R 30110 30112 29579 0 -1 4202496 1040 0 0 0 0 0 0 0 17 0 1 0 668227084 16629760 913 33554432000 4194304 5683644 140737147971136 18446744073709551615 4540219 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/30112/statm: 4060 913 380 364 0 547 0

[startup+0.058632 s]
/proc/loadavg: 7.93 8.22 6.14 2/417 30112
/proc/meminfo: memFree=11239892/32951124 swapFree=56316216/67111528
[pid=30112] ppid=30110 vsize=36508 CPUtime=0.04 cores=0-7
/proc/30112/stat : 30112 (python) R 30110 30112 29579 0 -1 4202496 1618 0 0 0 3 1 0 0 17 0 1 0 668227084 37384192 1450 33554432000 4194304 5683644 140737147971136 18446744073709551615 212436015104 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/30112/statm: 9127 1450 452 364 0 944 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 36508

[startup+0.100619 s]
/proc/loadavg: 7.93 8.22 6.14 2/417 30112
/proc/meminfo: memFree=11239892/32951124 swapFree=56316216/67111528
[pid=30112] ppid=30110 vsize=104224 CPUtime=0.07 cores=0-7
/proc/30112/stat : 30112 (python) R 30110 30112 29579 0 -1 4202496 3056 0 0 0 5 2 0 0 17 0 1 0 668227084 106725376 2741 33554432000 4194304 5683644 140737147971136 18446744073709551615 212436060295 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/30112/statm: 26055 2740 847 364 0 1803 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 104224

[startup+0.300575 s]
/proc/loadavg: 7.93 8.22 6.14 2/417 30112
/proc/meminfo: memFree=11239892/32951124 swapFree=56316216/67111528
[pid=30112] ppid=30110 vsize=170752 CPUtime=0.24 cores=0-7
/proc/30112/stat : 30112 (python) R 30110 30112 29579 0 -1 4202496 6730 939 1 0 17 7 0 0 17 0 1 0 668227084 174850048 4805 33554432000 4194304 5683644 140737147971136 18446744073709551615 5307659 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/30112/statm: 42688 4805 1336 364 0 3453 0
Current children cumulated CPU time (s) 0.24
Current children cumulated vsize (KiB) 170752

[startup+0.7005 s]
/proc/loadavg: 7.93 8.22 6.14 2/417 30112
/proc/meminfo: memFree=11239892/32951124 swapFree=56316216/67111528
[pid=30112] ppid=30110 vsize=289492 CPUtime=0.61 cores=0-7
/proc/30112/stat : 30112 (python) R 30110 30112 29579 0 -1 4202496 13868 939 1 0 51 10 0 0 17 0 1 0 668227084 296439808 9747 33554432000 4194304 5683644 140737147971136 18446744073709551615 47851113772839 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/30112/statm: 72373 9747 1815 364 0 8297 0
Current children cumulated CPU time (s) 0.61
Current children cumulated vsize (KiB) 289492

[startup+1.50035 s]
/proc/loadavg: 7.93 8.22 6.14 2/418 30117
/proc/meminfo: memFree=11207272/32951124 swapFree=56316216/67111528
[pid=30112] ppid=30110 vsize=289492 CPUtime=1.41 cores=0-7
/proc/30112/stat : 30112 (python) R 30110 30112 29579 0 -1 4202496 13880 939 1 0 131 10 0 0 19 0 1 0 668227084 296439808 9758 33554432000 4194304 5683644 140737147971136 18446744073709551615 212435675510 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/30112/statm: 72373 9758 1826 364 0 8297 0
Current children cumulated CPU time (s) 1.41
Current children cumulated vsize (KiB) 289492

[startup+3.10101 s]
/proc/loadavg: 7.46 8.12 6.11 2/418 30117
/proc/meminfo: memFree=11207644/32951124 swapFree=56316216/67111528
[pid=30112] ppid=30110 vsize=429952 CPUtime=2.59 cores=0-7
/proc/30112/stat : 30112 (python) S 30110 30112 29579 0 -1 4202496 19878 939 1 0 246 13 0 0 20 0 9 0 668227084 440270848 8320 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447843787 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/30112/statm: 107488 8320 1891 364 0 43385 0
Current children cumulated CPU time (s) 2.59
Current children cumulated vsize (KiB) 429952

[startup+6.30141 s]
/proc/loadavg: 7.46 8.12 6.11 9/442 30141
/proc/meminfo: memFree=11146676/32951124 swapFree=56316216/67111528
[pid=30112] ppid=30110 vsize=429952 CPUtime=2.59 cores=0-7
/proc/30112/stat : 30112 (python) S 30110 30112 29579 0 -1 4202496 19878 939 1 0 246 13 0 0 20 0 9 0 668227084 440270848 8320 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447843787 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/30112/statm: 107488 8320 1891 364 0 43385 0
[pid=30118] ppid=30112 vsize=282236 CPUtime=0 cores=0-7
/proc/30118/stat : 30118 (python) S 30112 30112 29579 0 -1 4202560 737 0 0 0 0 0 0 0 15 0 1 0 668227331 289009664 6330 33554432000 4194304 5683644 140737147971136 18446744073709551615 212436037135 0 0 16781312 514 17922787754706436354 0 0 17 7 0 0 0
/proc/30118/statm: 70559 6330 355 364 0 6477 0
[pid=30120] ppid=30118 vsize=1020188 CPUtime=3.82 cores=0-7
/proc/30120/stat : 30120 (gnovelty+2) R 30118 30120 30120 0 -1 4202496 1142 0 0 0 382 0 0 0 25 0 1 0 668227332 1044672512 876 33554432000 134512640 135663704 4290204160 18446744073709551615 134525276 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/30120/statm: 255047 876 113 282 0 254763 0
[pid=30121] ppid=30112 vsize=293120 CPUtime=0 cores=0-7
/proc/30121/stat : 30121 (python) S 30112 30112 29579 0 -1 4202560 740 0 0 0 0 0 0 0 18 0 1 0 668227334 300154880 6367 33554432000 4194304 5683644 140737147971136 18446744073709551615 212436037190 0 0 16781312 514 13012509287669268738 0 0 17 0 0 0 0
/proc/30121/statm: 73280 6367 349 364 0 9195 0
[pid=30123] ppid=30121 vsize=1020188 CPUtime=3.79 cores=0-7
/proc/30123/stat : 30123 (gnovelty+2) R 30121 30123 30123 0 -1 4202496 1139 0 0 0 379 0 0 0 25 0 1 0 668227335 1044672512 876 33554432000 134512640 135663704 4293823056 18446744073709551615 134525276 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/30123/statm: 255047 876 113 282 0 254763 0
[pid=30124] ppid=30112 vsize=302892 CPUtime=0 cores=0-7
/proc/30124/stat : 30124 (python) S 30112 30112 29579 0 -1 4202560 737 0 0 0 0 0 0 0 18 0 1 0 668227337 310161408 6504 33554432000 4194304 5683644 140737147971136 18446744073709551615 212436037190 0 0 16781312 514 17442587321269453057 0 0 17 1 0 0 0
/proc/30124/statm: 75723 6504 347 364 0 11635 0
[pid=30126] ppid=30124 vsize=421140 CPUtime=3.76 cores=0-7
/proc/30126/stat : 30126 (adaptg2wsat2009) R 30124 30126 30126 0 -1 4202496 1810 0 0 0 376 0 0 0 25 0 1 0 668227338 431247360 1540 33554432000 134512640 135107411 4291404112 18446744073709551615 134523261 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/30126/statm: 105285 1540 49 146 0 104161 0
[pid=30127] ppid=30112 vsize=313148 CPUtime=0 cores=0-7
/proc/30127/stat : 30127 (python) S 30112 30112 29579 0 -1 4202560 747 0 0 0 0 0 0 0 18 0 1 0 668227340 320663552 6508 33554432000 4194304 5683644 140737147971136 18446744073709551615 212436037190 0 0 16781312 514 935472743543963905 0 0 17 1 0 0 0
/proc/30127/statm: 78287 6508 347 364 0 14196 0
[pid=30129] ppid=30127 vsize=421140 CPUtime=3.73 cores=0-7
/proc/30129/stat : 30129 (adaptg2wsat2009) R 30127 30129 30129 0 -1 4202496 1801 0 0 0 373 0 0 0 25 0 1 0 668227341 431247360 1538 33554432000 134512640 135107411 4291021008 18446744073709551615 134523424 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/30129/statm: 105285 1538 49 146 0 104161 0
[pid=30130] ppid=30112 vsize=388940 CPUtime=0 cores=0-7
/proc/30130/stat : 30130 (python) S 30112 30112 29579 0 -1 4202560 724 0 0 0 0 0 0 0 18 0 1 0 668227344 398274560 6685 33554432000 4194304 5683644 140737147971136 18446744073709551615 212436037190 0 0 16781312 514 13012501591087874305 0 0 17 1 0 0 0
/proc/30130/statm: 97235 6685 348 364 0 33141 0
[pid=30132] ppid=30130 vsize=421140 CPUtime=3.68 cores=0-7
/proc/30132/stat : 30132 (adaptg2wsat2009) R 30130 30132 30132 0 -1 4202496 1798 0 0 0 368 0 0 0 25 0 1 0 668227344 431247360 1538 33554432000 134512640 135107411 4287224000 18446744073709551615 134523245 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/30132/statm: 105285 1538 49 146 0 104161 0
[pid=30133] ppid=30112 vsize=399196 CPUtime=0 cores=0-7
/proc/30133/stat : 30133 (python) S 30112 30112 29579 0 -1 4202560 729 0 0 0 0 0 0 0 18 0 1 0 668227347 408776704 6670 33554432000 4194304 5683644 140737147971136 18446744073709551615 212436037190 0 0 16781312 514 12058278330294567171 0 0 17 1 0 0 0
/proc/30133/statm: 99799 6670 348 364 0 35702 0
[pid=30135] ppid=30133 vsize=1020188 CPUtime=3.61 cores=0-7
/proc/30135/stat : 30135 (gnovelty+2) R 30133 30135 30135 0 -1 4202496 1131 0 0 0 361 0 0 0 25 0 1 0 668227347 1044672512 876 33554432000 134512640 135663704 4288118352 18446744073709551615 134545619 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/30135/statm: 255047 876 113 282 0 254763 0
[pid=30136] ppid=30112 vsize=409452 CPUtime=0 cores=0-7
/proc/30136/stat : 30136 (python) S 30112 30112 29579 0 -1 4202560 743 0 0 0 0 0 0 0 18 0 1 0 668227350 419278848 6641 33554432000 4194304 5683644 140737147971136 18446744073709551615 212436037190 0 0 16781312 514 12058274482003869954 0 0 17 1 0 0 0
/proc/30136/statm: 102363 6641 348 364 0 38263 0
[pid=30138] ppid=30136 vsize=249144 CPUtime=3.62 cores=0-7
/proc/30138/stat : 30138 (FH) R 30136 30138 30138 0 -1 4202496 1556 0 0 0 362 0 0 0 25 0 1 0 668227351 255123456 1296 33554432000 134512640 134936988 4287645680 18446744073709551615 134523503 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/30138/statm: 62286 1296 37 104 0 61790 0
[pid=30139] ppid=30112 vsize=419708 CPUtime=0 cores=0-7
/proc/30139/stat : 30139 (python) S 30112 30112 29579 0 -1 4202560 771 0 0 0 0 0 0 0 18 0 1 0 668227353 429780992 6779 33554432000 4194304 5683644 140737147971136 18446744073709551615 212436037190 0 0 16781312 514 12058270633713172738 0 0 17 1 0 0 0
/proc/30139/statm: 104927 6779 354 364 0 40824 0
[pid=30141] ppid=30139 vsize=421140 CPUtime=3.6 cores=0-7
/proc/30141/stat : 30141 (adaptg2wsat2009) R 30139 30141 30141 0 -1 4202496 1803 0 0 0 360 0 0 0 25 0 1 0 668227353 431247360 1538 33554432000 134512640 135107411 4293830928 18446744073709551615 134523215 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/30141/statm: 105285 1538 49 146 0 104161 0
[pid=30112/tid=30119] ppid=30110 vsize=429952 CPUtime=0 cores=0-7
/proc/30112/task/30119/stat : 30119 (python) S 30110 30112 29579 0 -1 4202560 27 939 0 0 0 0 0 0 22 0 9 0 668227331 440270848 8320 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 6 0 0 0
[pid=30112/tid=30122] ppid=30110 vsize=429952 CPUtime=0 cores=0-7
/proc/30112/task/30122/stat : 30122 (python) S 30110 30112 29579 0 -1 4202560 23 939 0 0 0 0 0 0 20 0 9 0 668227335 440270848 8320 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 6 0 0 0
[pid=30112/tid=30125] ppid=30110 vsize=429952 CPUtime=0 cores=0-7
/proc/30112/task/30125/stat : 30125 (python) S 30110 30112 29579 0 -1 4202560 22 939 0 0 0 0 0 0 19 0 9 0 668227338 440270848 8320 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 0 0 0 0
[pid=30112/tid=30128] ppid=30110 vsize=429952 CPUtime=0 cores=0-7
/proc/30112/task/30128/stat : 30128 (python) S 30110 30112 29579 0 -1 4202560 24 939 0 0 0 0 0 0 18 0 9 0 668227341 440270848 8320 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 5 0 0 0
[pid=30112/tid=30131] ppid=30110 vsize=429952 CPUtime=0 cores=0-7
/proc/30112/task/30131/stat : 30131 (python) S 30110 30112 29579 0 -1 4202560 89 939 0 0 0 0 0 0 19 0 9 0 668227344 440270848 8320 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 0 0 0 0
[pid=30112/tid=30134] ppid=30110 vsize=429952 CPUtime=0 cores=0-7
/proc/30112/task/30134/stat : 30134 (python) S 30110 30112 29579 0 -1 4202560 34 939 0 0 0 0 0 0 20 0 9 0 668227347 440270848 8320 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 7 0 0 0
[pid=30112/tid=30137] ppid=30110 vsize=429952 CPUtime=0 cores=0-7
/proc/30112/task/30137/stat : 30137 (python) S 30110 30112 29579 0 -1 4202560 33 939 0 0 0 0 0 0 19 0 9 0 668227350 440270848 8320 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 1 0 0 0
[pid=30112/tid=30140] ppid=30110 vsize=429952 CPUtime=0 cores=0-7
/proc/30112/task/30140/stat : 30140 (python) S 30110 30112 29579 0 -1 4202560 33 939 0 0 0 0 0 0 20 0 9 0 668227353 440270848 8320 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 32.2
Current children cumulated vsize (KiB) 8232912

[startup+12.7012 s]
/proc/loadavg: 7.61 8.13 6.14 9/442 30141
/proc/meminfo: memFree=11138888/32951124 swapFree=56316216/67111528
[pid=30112] ppid=30110 vsize=429952 CPUtime=2.59 cores=0-7
/proc/30112/stat : 30112 (python) S 30110 30112 29579 0 -1 4202496 19878 939 1 0 246 13 0 0 20 0 9 0 668227084 440270848 8320 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447843787 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/30112/statm: 107488 8320 1891 364 0 43385 0
[pid=30118] ppid=30112 vsize=282236 CPUtime=0.03 cores=0-7
/proc/30118/stat : 30118 (python) S 30112 30112 29579 0 -1 4202560 906 0 0 0 1 2 0 0 16 0 1 0 668227331 289009664 6353 33554432000 4194304 5683644 140737147971136 18446744073709551615 212436037135 0 0 16781312 514 3562347305250029826 0 0 17 7 0 0 0
/proc/30118/statm: 70559 6353 378 364 0 6477 0
[pid=30120] ppid=30118 vsize=1020188 CPUtime=10.22 cores=0-7
/proc/30120/stat : 30120 (gnovelty+2) R 30118 30120 30120 0 -1 4202496 1142 0 0 0 1022 0 0 0 25 0 1 0 668227332 1044672512 876 33554432000 134512640 135663704 4290204160 18446744073709551615 134524868 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/30120/statm: 255047 876 113 282 0 254763 0
[pid=30121] ppid=30112 vsize=293120 CPUtime=0.03 cores=0-7
/proc/30121/stat : 30121 (python) S 30112 30112 29579 0 -1 4202560 909 0 0 0 1 2 0 0 18 0 1 0 668227334 300154880 6390 33554432000 4194304 5683644 140737147971136 18446744073709551615 212436037190 0 0 16781312 514 3562347305250029826 0 0 17 0 0 0 0
/proc/30121/statm: 73280 6390 372 364 0 9195 0
[pid=30123] ppid=30121 vsize=1020188 CPUtime=10.19 cores=0-7
/proc/30123/stat : 30123 (gnovelty+2) R 30121 30123 30123 0 -1 4202496 1139 0 0 0 1019 0 0 0 25 0 1 0 668227335 1044672512 876 33554432000 134512640 135663704 4293823056 18446744073709551615 134524553 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/30123/statm: 255047 876 113 282 0 254763 0
[pid=30124] ppid=30112 vsize=302892 CPUtime=0.03 cores=0-7
/proc/30124/stat : 30124 (python) S 30112 30112 29579 0 -1 4202560 897 0 0 0 1 2 0 0 18 0 1 0 668227337 310161408 6527 33554432000 4194304 5683644 140737147971136 18446744073709551615 212436037190 0 0 16781312 514 3562347305250029825 0 0 17 1 0 0 0
/proc/30124/statm: 75723 6527 370 364 0 11635 0
[pid=30126] ppid=30124 vsize=421804 CPUtime=10.16 cores=0-7
/proc/30126/stat : 30126 (adaptg2wsat2009) R 30124 30126 30126 0 -1 4202496 2143 0 0 0 1016 0 0 0 25 0 1 0 668227338 431927296 1873 33554432000 134512640 135107411 4291404112 18446744073709551615 134532135 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/30126/statm: 105451 1873 59 146 0 104327 0
[pid=30127] ppid=30112 vsize=313148 CPUtime=0.04 cores=0-7
/proc/30127/stat : 30127 (python) S 30112 30112 29579 0 -1 4202560 937 0 0 0 2 2 0 0 18 0 1 0 668227340 320663552 6531 33554432000 4194304 5683644 140737147971136 18446744073709551615 212436037190 0 0 16781312 514 3562347305250029825 0 0 17 1 0 0 0
/proc/30127/statm: 78287 6531 370 364 0 14196 0
[pid=30129] ppid=30127 vsize=421804 CPUtime=10.13 cores=0-7
/proc/30129/stat : 30129 (adaptg2wsat2009) R 30127 30129 30129 0 -1 4202496 2134 0 0 0 1013 0 0 0 25 0 1 0 668227341 431927296 1871 33554432000 134512640 135107411 4291021008 18446744073709551615 134532224 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/30129/statm: 105451 1871 59 146 0 104327 0
[pid=30130] ppid=30112 vsize=388940 CPUtime=0.03 cores=0-7
/proc/30130/stat : 30130 (python) S 30112 30112 29579 0 -1 4202560 803 0 0 0 2 1 0 0 18 0 1 0 668227344 398274560 6708 33554432000 4194304 5683644 140737147971136 18446744073709551615 212436037190 0 0 16781312 514 3562347305250029825 0 0 17 1 0 0 0
/proc/30130/statm: 97235 6708 371 364 0 33141 0
[pid=30132] ppid=30130 vsize=421804 CPUtime=10.08 cores=0-7
/proc/30132/stat : 30132 (adaptg2wsat2009) R 30130 30132 30132 0 -1 4202496 2131 0 0 0 1008 0 0 0 25 0 1 0 668227344 431927296 1871 33554432000 134512640 135107411 4287224000 18446744073709551615 134532277 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/30132/statm: 105451 1871 59 146 0 104327 0
[pid=30133] ppid=30112 vsize=399196 CPUtime=0.02 cores=0-7
/proc/30133/stat : 30133 (python) S 30112 30112 29579 0 -1 4202560 808 0 0 0 1 1 0 0 18 0 1 0 668227347 408776704 6693 33554432000 4194304 5683644 140737147971136 18446744073709551615 212436037190 0 0 16781312 514 12058278330294567171 0 0 17 1 0 0 0
/proc/30133/statm: 99799 6693 371 364 0 35702 0
[pid=30135] ppid=30133 vsize=1020188 CPUtime=9.87 cores=0-7
/proc/30135/stat : 30135 (gnovelty+2) R 30133 30135 30135 0 -1 4202496 1131 0 0 0 987 0 0 0 25 0 1 0 668227347 1044672512 876 33554432000 134512640 135663704 4288118352 18446744073709551615 134524918 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/30135/statm: 255047 876 113 282 0 254763 0
[pid=30136] ppid=30112 vsize=409452 CPUtime=0.02 cores=0-7
/proc/30136/stat : 30136 (python) R 30112 30112 29579 0 -1 4202560 826 0 0 0 1 1 0 0 18 0 1 0 668227350 419278848 6664 33554432000 4194304 5683644 140737147971136 18446744073709551615 212436060295 0 0 16781312 514 0 0 0 17 1 0 0 0
/proc/30136/statm: 102363 6664 371 364 0 38263 0

################
# More data... #
################

/proc/30211/statm: 83999 1751 59 147 0 83851 0
[pid=30209/tid=30217] ppid=30112 vsize=563292 CPUtime=0 cores=0-7
/proc/30209/task/30217/stat : 30217 (python) S 30112 30112 29579 0 -1 4202560 84 0 0 0 0 0 0 0 19 0 2 0 668277567 576811008 7010 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 514 18446604444886320384 0 0 -1 0 0 0 0
[pid=30214] ppid=30112 vsize=573548 CPUtime=0.34 cores=0-7
/proc/30214/stat : 30214 (python) S 30112 30112 29579 0 -1 4202560 1189 0 0 0 18 16 0 0 19 0 2 0 668277060 587313152 7061 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447843787 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/30214/statm: 143387 7061 406 364 0 79239 0
[pid=30216] ppid=30214 vsize=1020188 CPUtime=99.38 cores=0-7
/proc/30216/stat : 30216 (gnovelty+2) T 30214 30216 30216 0 -1 4202496 1159 0 0 0 9938 0 0 0 25 0 1 0 668277061 1044672512 876 33554432000 134512640 135663704 4289258512 18446744073709551615 134525070 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/30216/statm: 255047 876 113 282 0 254763 0
[pid=30214/tid=30226] ppid=30112 vsize=573548 CPUtime=0 cores=0-7
/proc/30214/task/30226/stat : 30226 (python) S 30112 30112 29579 0 -1 4202560 84 0 0 0 0 0 0 0 19 0 2 0 668287002 587313152 7061 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 514 18446604450580246976 0 0 -1 0 0 0 0
[pid=30223] ppid=30112 vsize=583804 CPUtime=0.35 cores=0-7
/proc/30223/stat : 30223 (python) S 30112 30112 29579 0 -1 4202560 1196 0 0 0 18 17 0 0 19 0 2 0 668286998 597815296 6994 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447843787 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/30223/statm: 145951 6994 407 364 0 81800 0
[pid=30225] ppid=30223 vsize=335996 CPUtime=100.31 cores=0-7
/proc/30225/stat : 30225 (iPAWS) T 30223 30225 30225 0 -1 4202496 2031 0 0 0 10031 0 0 0 25 0 1 0 668286999 344059904 1750 33554432000 134512640 135114423 4288541120 18446744073709551615 134549846 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/30225/statm: 83999 1750 59 147 0 83851 0
[pid=30223/tid=30235] ppid=30112 vsize=583804 CPUtime=0 cores=0-7
/proc/30223/task/30235/stat : 30235 (python) S 30112 30112 29579 0 -1 4202560 89 0 0 0 0 0 0 0 19 0 2 0 668297033 597815296 6994 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 514 18446604451502144192 0 0 -1 1 0 0 0
[pid=30227] ppid=30112 vsize=594060 CPUtime=1.4 cores=0-7
/proc/30227/stat : 30227 (python) S 30112 30112 29579 0 -1 4202560 1195 0 0 0 75 65 0 0 18 0 2 0 668287374 608317440 7001 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447843787 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/30227/statm: 148515 7001 400 364 0 84361 0
[pid=30229] ppid=30227 vsize=268424 CPUtime=390.55 cores=0-7
/proc/30229/stat : 30229 (hybridGM3) T 30227 30229 30229 0 -1 4202496 5663 0 0 0 39051 4 0 0 25 0 1 0 668287374 274866176 5377 33554432000 134512640 135204385 4291456640 18446744073709551615 134537792 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/30229/statm: 67106 5377 63 169 0 66936 0
[pid=30227/tid=30249] ppid=30112 vsize=594060 CPUtime=0 cores=0-7
/proc/30227/task/30249/stat : 30249 (python) S 30112 30112 29579 0 -1 4202560 93 0 0 0 0 0 0 0 18 0 2 0 668307651 608317440 7001 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 514 18446604450236029632 0 0 -1 1 0 0 0
[pid=30232] ppid=30112 vsize=604316 CPUtime=2.41 cores=0-7
/proc/30232/stat : 30232 (python) S 30112 30112 29579 0 -1 4202560 995 0 0 0 122 119 0 0 18 0 1 0 668296630 618819584 7058 33554432000 4194304 5683644 140737147971136 18446744073709551615 212436037190 0 0 16781312 514 3562347305250029826 0 0 17 4 0 0 0
/proc/30232/statm: 151079 7058 372 364 0 86922 0
[pid=30234] ppid=30232 vsize=459772 CPUtime=695.81 cores=0-7
/proc/30234/stat : 30234 (TNM) R 30232 30234 30234 0 -1 4202496 1939 0 0 0 69580 1 0 0 25 0 1 0 668296631 470806528 1657 33554432000 134512640 135008748 4291946672 18446744073709551615 134529448 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/30234/statm: 114943 1657 46 122 0 113843 0
[pid=30236] ppid=30112 vsize=614572 CPUtime=1.4 cores=0-7
/proc/30236/stat : 30236 (python) S 30112 30112 29579 0 -1 4202560 1197 0 0 0 72 68 0 0 18 0 2 0 668297039 629321728 7118 33554432000 4194304 5683644 140737147971136 18446744073709551615 212436037190 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/30236/statm: 153643 7118 401 364 0 89483 0
[pid=30238] ppid=30236 vsize=251292 CPUtime=387.75 cores=0-7
/proc/30238/stat : 30238 (FH) R 30236 30238 30238 0 -1 4202496 1975 0 0 0 38775 0 0 0 25 0 1 0 668297039 257323008 1694 33554432000 134512640 134936988 4293500112 18446744073709551615 134554976 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/30238/statm: 62823 1694 41 104 0 62327 0
[pid=30236/tid=30245] ppid=30112 vsize=614572 CPUtime=0 cores=0-7
/proc/30236/task/30245/stat : 30245 (python) S 30112 30112 29579 0 -1 4202560 91 0 0 0 0 0 0 0 18 0 2 0 668306974 629321728 7118 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 514 18446604452273185792 0 0 -1 3 0 0 0
[pid=30242] ppid=30112 vsize=624828 CPUtime=2.13 cores=0-7
/proc/30242/stat : 30242 (python) S 30112 30112 29579 0 -1 4202560 1215 0 0 0 113 100 0 0 18 0 2 0 668306576 639823872 7048 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447843787 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/30242/statm: 156207 7048 400 364 0 92044 0
[pid=30244] ppid=30242 vsize=268424 CPUtime=583.92 cores=0-7
/proc/30244/stat : 30244 (hybridGM3) T 30242 30244 30244 0 -1 4202496 5664 0 0 0 58388 4 0 0 25 0 1 0 668306577 274866176 5378 33554432000 134512640 135204385 4290909616 18446744073709551615 134538589 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/30244/statm: 67106 5378 63 169 0 66936 0
[pid=30242/tid=30259] ppid=30112 vsize=624828 CPUtime=0 cores=0-7
/proc/30242/task/30259/stat : 30259 (python) S 30112 30112 29579 0 -1 4202560 89 0 0 0 0 0 0 0 18 0 2 0 668327358 639823872 7048 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 514 18446604448666916672 0 0 -1 1 0 0 0
[pid=30246] ppid=30112 vsize=635084 CPUtime=2.07 cores=0-7
/proc/30246/stat : 30246 (python) S 30112 30112 29579 0 -1 4202560 1243 0 0 0 107 100 0 0 18 0 2 0 668306981 650326016 7147 33554432000 4194304 5683644 140737147971136 18446744073709551615 212436037190 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/30246/statm: 158771 7147 407 364 0 94605 0
[pid=30248] ppid=30246 vsize=421344 CPUtime=590.4 cores=0-7
/proc/30248/stat : 30248 (NCVWr) R 30246 30248 30248 0 -1 4202496 2006 0 0 0 59038 2 0 0 25 0 1 0 668306981 431456256 1712 33554432000 134512640 135113558 4288085024 18446744073709551615 134534097 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/30248/statm: 105336 1712 58 147 0 104211 0
[pid=30246/tid=30271] ppid=30112 vsize=635084 CPUtime=0 cores=0-7
/proc/30246/task/30271/stat : 30271 (python) S 30112 30112 29579 0 -1 4202560 91 0 0 0 0 0 0 0 19 0 2 0 668356047 650326016 7147 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 514 18446604453903145792 0 0 -1 7 0 0 0
[pid=30256] ppid=30112 vsize=645340 CPUtime=0.36 cores=0-7
/proc/30256/stat : 30256 (python) S 30112 30112 29579 0 -1 4202560 1250 0 0 0 19 17 0 0 19 0 2 0 668327336 660828160 7136 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447843787 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/30256/statm: 161335 7136 407 364 0 97166 0
[pid=30258] ppid=30256 vsize=335996 CPUtime=99.95 cores=0-7
/proc/30258/stat : 30258 (iPAWS) T 30256 30258 30258 0 -1 4202496 2038 0 0 0 9995 0 0 0 25 0 1 0 668327337 344059904 1749 33554432000 134512640 135114423 4294076480 18446744073709551615 134517031 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/30258/statm: 83999 1749 59 147 0 83851 0
[pid=30256/tid=30262] ppid=30112 vsize=645340 CPUtime=0 cores=0-7
/proc/30256/task/30262/stat : 30262 (python) S 30112 30112 29579 0 -1 4202560 92 0 0 0 0 0 0 0 19 0 2 0 668337371 660828160 7136 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 514 18446604448374459072 0 0 -1 0 0 0 0
[pid=30268] ppid=30112 vsize=655596 CPUtime=0.35 cores=0-7
/proc/30268/stat : 30268 (python) S 30112 30112 29579 0 -1 4202560 1050 0 0 0 18 17 0 0 18 0 1 0 668355944 671330304 7056 33554432000 4194304 5683644 140737147971136 18446744073709551615 212436037190 0 0 16781312 514 3562347305250029825 0 0 17 0 0 0 0
/proc/30268/statm: 163899 7056 378 364 0 99727 0
[pid=30270] ppid=30268 vsize=268420 CPUtime=97.28 cores=0-7
/proc/30270/stat : 30270 (hybridGM3) R 30268 30270 30270 0 -1 4202496 5666 0 0 0 9728 0 0 0 25 0 1 0 668355945 274862080 5376 33554432000 134512640 135204385 4291894816 18446744073709551615 134538284 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/30270/statm: 67105 5376 63 169 0 66935 0
[pid=30272] ppid=30112 vsize=665852 CPUtime=0.34 cores=0-7
/proc/30272/stat : 30272 (python) S 30112 30112 29579 0 -1 4202560 1052 0 0 0 18 16 0 0 18 0 1 0 668356265 681832448 7155 33554432000 4194304 5683644 140737147971136 18446744073709551615 212436037190 0 0 16781312 514 3562347305250029827 0 0 17 0 0 0 0
/proc/30272/statm: 166463 7155 379 364 0 102288 0
[pid=30274] ppid=30272 vsize=459764 CPUtime=99.57 cores=0-7
/proc/30274/stat : 30274 (TNM) R 30272 30274 30274 0 -1 4202496 1946 0 0 0 9957 0 0 0 25 0 1 0 668356266 470798336 1657 33554432000 134512640 135008748 4290383184 18446744073709551615 134529398 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/30274/statm: 114941 1657 46 122 0 113841 0
[pid=30278] ppid=30112 vsize=676108 CPUtime=0 cores=0-7
/proc/30278/stat : 30278 (python) S 30112 30112 29579 0 -1 4202560 940 0 0 0 0 0 0 0 19 0 1 0 668366216 692334592 7165 33554432000 4194304 5683644 140737147971136 18446744073709551615 212436037190 0 0 16781312 514 10423009046041722879 0 0 17 0 0 0 0
/proc/30278/statm: 169027 7165 352 364 0 104849 0
[pid=30280] ppid=30278 vsize=416312 CPUtime=0.11 cores=0-7
/proc/30280/stat : 30280 (NCVWr) R 30278 30280 30280 0 -1 4202496 1605 0 0 0 11 0 0 0 21 0 1 0 668366216 426303488 1314 33554432000 134512640 135113558 4292365520 18446744073709551615 134526368 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/30280/statm: 104078 1314 55 147 0 102953 0
[pid=30112/tid=30119] ppid=30110 vsize=686352 CPUtime=0 cores=0-7
/proc/30112/task/30119/stat : 30119 (python) S 30110 30112 29579 0 -1 4202560 27 939 0 0 0 0 0 0 22 0 34 0 668227331 702824448 8759 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 6 0 0 0
[pid=30112/tid=30122] ppid=30110 vsize=686352 CPUtime=0 cores=0-7
/proc/30112/task/30122/stat : 30122 (python) S 30110 30112 29579 0 -1 4202560 23 939 0 0 0 0 0 0 20 0 34 0 668227335 702824448 8759 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 6 0 0 0
[pid=30112/tid=30125] ppid=30110 vsize=686352 CPUtime=0 cores=0-7
/proc/30112/task/30125/stat : 30125 (python) S 30110 30112 29579 0 -1 4202560 34 939 0 0 0 0 0 0 18 0 34 0 668227338 702824448 8759 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 7 0 0 0
[pid=30112/tid=30128] ppid=30110 vsize=686352 CPUtime=0 cores=0-7
/proc/30112/task/30128/stat : 30128 (python) S 30110 30112 29579 0 -1 4202560 29 939 0 0 0 0 0 0 15 0 34 0 668227341 702824448 8759 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 5 0 0 0
[pid=30112/tid=30131] ppid=30110 vsize=686352 CPUtime=0 cores=0-7
/proc/30112/task/30131/stat : 30131 (python) S 30110 30112 29579 0 -1 4202560 94 939 0 0 0 0 0 0 18 0 34 0 668227344 702824448 8759 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 7 0 0 0
[pid=30112/tid=30134] ppid=30110 vsize=686352 CPUtime=0 cores=0-7
/proc/30112/task/30134/stat : 30134 (python) S 30110 30112 29579 0 -1 4202560 34 939 0 0 0 0 0 0 20 0 34 0 668227347 702824448 8759 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 7 0 0 0
[pid=30112/tid=30137] ppid=30110 vsize=686352 CPUtime=0 cores=0-7
/proc/30112/task/30137/stat : 30137 (python) S 30110 30112 29579 0 -1 4202560 52 939 0 0 0 0 0 0 18 0 34 0 668227350 702824448 8759 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 1 0 0 0
[pid=30112/tid=30140] ppid=30110 vsize=686352 CPUtime=0 cores=0-7
/proc/30112/task/30140/stat : 30140 (python) S 30110 30112 29579 0 -1 4202560 39 939 0 0 0 0 0 0 18 0 34 0 668227353 702824448 8759 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 1 0 0 0
[pid=30112/tid=30147] ppid=30110 vsize=686352 CPUtime=0 cores=0-7
/proc/30112/task/30147/stat : 30147 (python) S 30110 30112 29579 0 -1 4202560 50 939 0 0 0 0 0 0 18 0 34 0 668237273 702824448 8759 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 1 0 0 0
[pid=30112/tid=30151] ppid=30110 vsize=686352 CPUtime=0 cores=0-7
/proc/30112/task/30151/stat : 30151 (python) S 30110 30112 29579 0 -1 4202560 39 939 0 0 0 0 0 0 18 0 34 0 668237372 702824448 8759 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 3 0 0 0
[pid=30112/tid=30157] ppid=30110 vsize=686352 CPUtime=0 cores=0-7
/proc/30112/task/30157/stat : 30157 (python) S 30110 30112 29579 0 -1 4202560 39 939 0 0 0 0 0 0 18 0 34 0 668237376 702824448 8759 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 4 0 0 0
[pid=30112/tid=30160] ppid=30110 vsize=686352 CPUtime=0 cores=0-7
/proc/30112/task/30160/stat : 30160 (python) S 30110 30112 29579 0 -1 4202560 32 939 0 0 0 0 0 0 24 0 34 0 668237381 702824448 8759 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 5 0 0 0
[pid=30112/tid=30165] ppid=30110 vsize=686352 CPUtime=0 cores=0-7
/proc/30112/task/30165/stat : 30165 (python) S 30110 30112 29579 0 -1 4202560 52 939 0 0 0 0 0 0 18 0 34 0 668237385 702824448 8759 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 7 0 0 0
[pid=30112/tid=30169] ppid=30110 vsize=686352 CPUtime=0 cores=0-7
/proc/30112/task/30169/stat : 30169 (python) S 30110 30112 29579 0 -1 4202560 61 939 0 0 0 0 0 0 18 0 34 0 668237390 702824448 8759 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 3 0 0 0
[pid=30112/tid=30172] ppid=30110 vsize=686352 CPUtime=0 cores=0-7
/proc/30112/task/30172/stat : 30172 (python) S 30110 30112 29579 0 -1 4202560 53 939 0 0 0 0 0 0 18 0 34 0 668237395 702824448 8759 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 0 0 0 0
[pid=30112/tid=30175] ppid=30110 vsize=686352 CPUtime=0 cores=0-7
/proc/30112/task/30175/stat : 30175 (python) S 30110 30112 29579 0 -1 4202560 40 939 0 0 0 0 0 0 18 0 34 0 668237400 702824448 8759 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 1 0 0 0
[pid=30112/tid=30181] ppid=30110 vsize=686352 CPUtime=0 cores=0-7
/proc/30112/task/30181/stat : 30181 (python) S 30110 30112 29579 0 -1 4202560 34 939 0 0 0 0 0 0 20 0 34 0 668247311 702824448 8759 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 0 0 0 0
[pid=30112/tid=30185] ppid=30110 vsize=686352 CPUtime=0 cores=0-7
/proc/30112/task/30185/stat : 30185 (python) S 30110 30112 29579 0 -1 4202560 33 939 0 0 0 0 0 0 20 0 34 0 668247410 702824448 8759 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 0 0 0 0
[pid=30112/tid=30196] ppid=30110 vsize=686352 CPUtime=0 cores=0-7
/proc/30112/task/30196/stat : 30196 (python) S 30110 30112 29579 0 -1 4202560 41 939 0 0 0 0 0 0 20 0 34 0 668257353 702824448 8759 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 0 0 0 0
[pid=30112/tid=30203] ppid=30110 vsize=686352 CPUtime=0 cores=0-7
/proc/30112/task/30203/stat : 30203 (python) S 30110 30112 29579 0 -1 4202560 41 939 0 0 0 0 0 0 18 0 34 0 668266905 702824448 8759 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 0 0 0 0
[pid=30112/tid=30207] ppid=30110 vsize=686352 CPUtime=0 cores=0-7
/proc/30112/task/30207/stat : 30207 (python) S 30110 30112 29579 0 -1 4202560 36 939 0 0 0 0 0 0 20 0 34 0 668267191 702824448 8759 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 0 0 0 0
[pid=30112/tid=30210] ppid=30110 vsize=686352 CPUtime=0 cores=0-7
/proc/30112/task/30210/stat : 30210 (python) S 30110 30112 29579 0 -1 4202560 38 939 0 0 0 0 0 0 20 0 34 0 668267532 702824448 8759 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 0 0 0 0
[pid=30112/tid=30215] ppid=30110 vsize=686352 CPUtime=0 cores=0-7
/proc/30112/task/30215/stat : 30215 (python) S 30110 30112 29579 0 -1 4202560 35 939 0 0 0 0 0 0 18 0 34 0 668277060 702824448 8759 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 1 0 0 0
[pid=30112/tid=30224] ppid=30110 vsize=686352 CPUtime=0 cores=0-7
/proc/30112/task/30224/stat : 30224 (python) S 30110 30112 29579 0 -1 4202560 38 939 0 0 0 0 0 0 20 0 34 0 668286998 702824448 8759 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 1 0 0 0
[pid=30112/tid=30228] ppid=30110 vsize=686352 CPUtime=0 cores=0-7
/proc/30112/task/30228/stat : 30228 (python) S 30110 30112 29579 0 -1 4202560 42 939 0 0 0 0 0 0 18 0 34 0 668287374 702824448 8759 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 0 0 0 0
[pid=30112/tid=30233] ppid=30110 vsize=686352 CPUtime=0 cores=0-7
/proc/30112/task/30233/stat : 30233 (python) S 30110 30112 29579 0 -1 4202560 35 939 0 0 0 0 0 0 20 0 34 0 668296631 702824448 8759 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 1 0 0 0
[pid=30112/tid=30237] ppid=30110 vsize=686352 CPUtime=0 cores=0-7
/proc/30112/task/30237/stat : 30237 (python) S 30110 30112 29579 0 -1 4202560 46 939 0 0 0 0 0 0 18 0 34 0 668297039 702824448 8759 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 0 0 0 0
[pid=30112/tid=30243] ppid=30110 vsize=686352 CPUtime=0 cores=0-7
/proc/30112/task/30243/stat : 30243 (python) S 30110 30112 29579 0 -1 4202560 43 939 0 0 0 0 0 0 18 0 34 0 668306576 702824448 8759 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 1 0 0 0
[pid=30112/tid=30247] ppid=30110 vsize=686352 CPUtime=0 cores=0-7
/proc/30112/task/30247/stat : 30247 (python) S 30110 30112 29579 0 -1 4202560 39 939 0 0 0 0 0 0 18 0 34 0 668306981 702824448 8759 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 0 0 0 0
[pid=30112/tid=30257] ppid=30110 vsize=686352 CPUtime=0 cores=0-7
/proc/30112/task/30257/stat : 30257 (python) S 30110 30112 29579 0 -1 4202560 37 939 0 0 0 0 0 0 20 0 34 0 668327336 702824448 8759 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 1 0 0 0
[pid=30112/tid=30269] ppid=30110 vsize=686352 CPUtime=0 cores=0-7
/proc/30112/task/30269/stat : 30269 (python) S 30110 30112 29579 0 -1 4202560 39 939 0 0 0 0 0 0 20 0 34 0 668355944 702824448 8759 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 0 0 0 0
[pid=30112/tid=30273] ppid=30110 vsize=686352 CPUtime=0 cores=0-7
/proc/30112/task/30273/stat : 30273 (python) S 30110 30112 29579 0 -1 4202560 36 939 0 0 0 0 0 0 20 0 34 0 668356266 702824448 8759 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 0 0 0 0
[pid=30112/tid=30279] ppid=30110 vsize=686352 CPUtime=0 cores=0-7
/proc/30112/task/30279/stat : 30279 (python) S 30110 30112 29579 0 -1 4202560 35 939 0 0 0 0 0 0 20 0 34 0 668366216 702824448 8759 33554432000 4194304 5683644 140737147971136 18446744073709551615 212447840689 0 0 16781312 2 18446604450488443136 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 11056.7
Current children cumulated vsize (KiB) 32134176

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 30112 and gives
#  childrusage.ru_utime.tv_sec=5
#  childrusage.ru_utime.tv_usec=608147
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=226965
# CPU time returned by wait4() is 5.83511
# while last known CPU time is 11056.7
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1391.62
CPU time (s): 11056.7
CPU user time (s): 11037.6
CPU system time (s): 19.12
CPU usage (%): 794.522
Max. virtual memory (cumulated for all children) (KiB): 32134176

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 5.60815
system time used= 0.226965
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 56359
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1380
involuntary context switches= 68

runsolver used 16.8524 second user time and 38.2962 second system time

The end

Launcher Data 

Begin job on node102 at 2011-05-16 19:17:27
IDJOB=3385398
IDBENCH=82663
IDSOLVER=1870
FILE ID=node102/3385398-1305566247
RUNJOBID= node102-1305564967-29593
PBS_JOBID= 13350148
Free space on /tmp= 73604 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/large/unif-k7-r85-v400-c34000-S536077088-015-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3385398-1305566247/watcher-3385398-1305566247 -o /tmp/evaluation-result-3385398-1305566247/solver-3385398-1305566247 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385398-1305566247.cnf -seed 930569595 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 7eb6d5b5c1c99a6b88954b61ac97b4f7
RANDOM SEED=930569595

node102.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      11240048 kB
Buffers:        122000 kB
Cached:        1030592 kB
SwapCached:      33496 kB
Active:       12860568 kB
Inactive:      8608316 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      11240048 kB
SwapTotal:    67111528 kB
SwapFree:     56316216 kB
Dirty:           32740 kB
Writeback:           0 kB
AnonPages:    20295188 kB
Mapped:          21168 kB
Slab:            79264 kB
PageTables:      91668 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 111823064 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73572 MiB
End job on node102 at 2011-05-16 19:40:41