Trace number 3321882

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-random? (MO) 9624.96 1371.81

General information on the benchmark

Namerandom/large/
unif-k7-r85-v300-c25500-S69399346-065-UNKNOWN.cnf
MD5SUM98486271f2f72889c040eabfd36ebd13
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables300
Number of clauses25500
Sum of the clauses size178500
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 525500

Solver Data 

0.38/0.67	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.38/0.67	c INFO: solving HOME/instance-3321882-1304369011.cnf
0.38/0.67	c INFO: computing task features
0.56/0.84	c INFO: computed variable-clause graph statistics
0.56/0.85	c INFO: computed clause balance statistics
1.45/1.75	c INFO: computed variable balance statistics
1.86/2.11	c INFO: computed small-clause counts
1.86/2.11	c INFO: computed Horn-clause counts
1.86/2.14	c INFO: computed variable graph statistics
1.86/2.14	c INFO: collected features for HOME/instance-3321882-1304369011.cnf in 1.47 s
1.86/2.18	c INFO: running gnovelty+2@0 for 97 with 39998 remaining (b = 0.37)
1.97/2.22	c INFO: running gnovelty+2@0 for 97 with 39998 remaining (b = 0.13)
1.97/2.26	c INFO: running adaptg2wsat2009++@0 for 97 with 39998 remaining (b = 0.08)
1.97/2.29	c INFO: running adaptg2wsat2009++@0 for 97 with 39998 remaining (b = 0.07)
2.05/2.33	c INFO: running adaptg2wsat2009++@0 for 97 with 39998 remaining (b = 0.05)
2.05/2.37	c INFO: running gnovelty+2@0 for 97 with 39998 remaining (b = 0.04)
2.15/2.41	c INFO: running FH@0 for 97 with 39998 remaining (b = 0.03)
2.15/2.46	c INFO: running adaptg2wsat2009++@0 for 97 with 39998 remaining (b = 0.03)
701.73/102.44	c INFO: running adaptg2wsat2009++@0 for 97 with 39898 remaining (b = 0.02)
702.36/102.52	c INFO: running FH@0 for 97 with 39898 remaining (b = 0.02)
702.36/102.57	c INFO: running hybridGM3@0 for 194 with 39898 remaining (b = 0.04)
703.45/102.70	c INFO: running gnovelty+2@0 for 97 with 39898 remaining (b = 0.02)
744.29/108.50	c INFO: running hybridGM3@0 for 194 with 39892 remaining (b = 0.03)
879.76/126.61	c INFO: running adaptg2wsat2009++@0 for 97 with 39874 remaining (b = 0.01)
923.02/132.88	c INFO: running hybridGM3@0 for 194 with 39867 remaining (b = 0.03)
1006.78/144.80	c INFO: running hybridGM3@0 for 194 with 39855 remaining (b = 0.02)
1410.20/202.70	c INFO: running TNM@0 for 1552 with 39798 remaining (b = 0.19)
1411.56/202.84	c INFO: running FH@0 for 97 with 39797 remaining (b = 0.01)
1538.49/221.07	c INFO: running adaptg2wsat2009++@0 for 97 with 39779 remaining (b = 0.01)
1788.71/256.93	c INFO: running adaptg2wsat2009++@0 for 97 with 39743 remaining (b = 0.01)
2083.02/299.19	c INFO: running hybridGM3@99 for 97 with 39701 remaining (b = 0.01)
2194.51/315.13	c INFO: running adaptg2wsat2009++@0 for 97 with 39685 remaining (b = 0.01)
2389.07/341.11	c INFO: running hybridGM3@102 for 97 with 39659 remaining (b = 0.01)
2397.46/342.37	c INFO: running adaptg2wsat2009++@100 for 97 with 39658 remaining (b = 0.01)
2516.82/359.47	c INFO: running hybridGM3@98 for 97 with 39641 remaining (b = 0.01)
2599.87/371.38	c INFO: running hybridGM3@99 for 97 with 39629 remaining (b = 0.01)
2648.03/378.25	c INFO: running hybridGM3@0 for 194 with 39622 remaining (b = 0.02)
2896.28/411.48	c INFO: running hybridGM3@0 for 194 with 39589 remaining (b = 0.02)
2993.37/424.41	c INFO: running TNM@0 for 1552 with 39576 remaining (b = 0.15)
3183.33/451.67	c INFO: running adaptg2wsat2009++@100 for 97 with 39549 remaining (b = 0.01)
3202.20/454.40	c INFO: running iPAWS@0 for 97 with 39546 remaining (b = 0.01)
3329.82/472.67	c INFO: running FH@0 for 291 with 39528 remaining (b = 0.03)
3456.81/490.82	c INFO: running hybridGM3@295 for 97 with 39509 remaining (b = 0.01)
3862.25/548.96	c INFO: running gnovelty+2@0 for 97 with 39451 remaining (b = 0.01)
4121.01/586.09	c INFO: running hybridGM3@391 for 97 with 39414 remaining (b = 0.01)
4210.31/598.85	c INFO: running hybridGM3@100 for 97 with 39401 remaining (b = 0.01)
4274.49/608.04	c INFO: running hybridGM3@96 for 97 with 39392 remaining (b = 0.01)
4561.87/649.29	c INFO: running hybridGM3@0 for 194 with 39351 remaining (b = 0.02)
4614.81/656.90	c INFO: running adaptg2wsat2009++@297 for 291 with 39343 remaining (b = 0.02)
4785.57/681.37	c INFO: running FH@100 for 291 with 39319 remaining (b = 0.02)
4980.78/709.33	c INFO: running iPAWS@0 for 97 with 39291 remaining (b = 0.01)
5083.90/724.19	c INFO: running hybridGM3@295 for 97 with 39276 remaining (b = 0.01)
5679.93/809.63	c INFO: running hybridGM3@0 for 194 with 39191 remaining (b = 0.01)
5888.03/837.51	c INFO: running hybridGM3@392 for 97 with 39163 remaining (b = 0.01)
6090.03/864.55	c INFO: running TNM@0 for 1552 with 39136 remaining (b = 0.11)
6288.62/893.01	c INFO: running hybridGM3@99 for 97 with 39107 remaining (b = 0.01)
6691.08/950.84	c INFO: running adaptg2wsat2009++@97 for 97 with 39049 remaining (b = 0.01)
6749.73/958.61	c INFO: running adaptg2wsat2009++@392 for 194 with 39042 remaining (b = 0.02)
7033.17/999.34	c INFO: running FH@0 for 97 with 39001 remaining (b = 0.01)
7296.28/1037.19	c INFO: running FH@200 for 194 with 38963 remaining (b = 0.01)
7416.01/1054.36	c INFO: running hybridGM3@100 for 97 with 38946 remaining (b = 0.01)
7540.59/1072.25	c INFO: running adaptg2wsat2009++@386 for 291 with 38928 remaining (b = 0.02)
7942.63/1129.73	c INFO: running FH@293 for 97 with 38871 remaining (b = 0.01)
8248.97/1173.71	c INFO: running gnovelty+2@0 for 97 with 38827 remaining (b = 0.01)
8251.49/1174.14	c INFO: running adaptg2wsat2009++@486 for 97 with 38826 remaining (b = 0.01)
8839.82/1258.73	c INFO: running hybridGM3@0 for 194 with 38742 remaining (b = 0.01)
8856.44/1261.11	c INFO: running iPAWS@0 for 97 with 38739 remaining (b = 0.01)
8955.88/1275.45	c INFO: running adaptg2wsat2009++@679 for 97 with 38725 remaining (b = 0.01)
9197.42/1310.18	c INFO: running hybridGM3@0 for 194 with 38690 remaining (b = 0.01)
9618.59/1370.77	c INFO: running NCVWr@0 for 485 with 38629 remaining (b = 0.03)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3321882-1304369011/watcher-3321882-1304369011 -o /tmp/evaluation-result-3321882-1304369011/solver-3321882-1304369011 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3321882-1304369011.cnf -seed 353230720 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.83 8.33 7.79 3/254 25692
/proc/meminfo: memFree=23346468/32950928 swapFree=67111464/67111528
[pid=25692] ppid=25690 vsize=15384 CPUtime=0 cores=0-7
/proc/25692/stat : 25692 (python) S 25690 25692 25198 0 -1 4202496 819 0 0 0 0 0 0 0 17 0 1 0 548499873 15753216 704 33554432000 4194304 5683644 140733287656320 18446744073709551615 268272687104 0 2147483391 16781312 2 18446744071699199166 0 0 17 2 0 0 0
/proc/25692/statm: 3846 704 334 364 0 333 0

[startup+0.0307461 s]
/proc/loadavg: 7.83 8.33 7.79 3/254 25692
/proc/meminfo: memFree=23346468/32950928 swapFree=67111464/67111528
[pid=25692] ppid=25690 vsize=16400 CPUtime=0.01 cores=0-7
/proc/25692/stat : 25692 (python) R 25690 25692 25198 0 -1 4202496 1153 0 0 0 1 0 0 0 17 0 1 0 548499873 16793600 1019 33554432000 4194304 5683644 140733287656320 18446744073709551615 268272688896 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/25692/statm: 4100 1020 413 364 0 587 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 16400

[startup+0.100733 s]
/proc/loadavg: 7.83 8.33 7.79 3/254 25692
/proc/meminfo: memFree=23346468/32950928 swapFree=67111464/67111528
[pid=25692] ppid=25690 vsize=87708 CPUtime=0.07 cores=0-7
/proc/25692/stat : 25692 (python) R 25690 25692 25198 0 -1 4202496 2638 0 0 0 5 2 0 0 17 0 1 0 548499873 89812992 2354 33554432000 4194304 5683644 140733287656320 18446744073709551615 268272381648 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/25692/statm: 22443 2356 718 364 0 1623 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 87708

[startup+0.300764 s]
/proc/loadavg: 7.83 8.33 7.79 3/254 25692
/proc/meminfo: memFree=23346468/32950928 swapFree=67111464/67111528
[pid=25692] ppid=25690 vsize=135752 CPUtime=0.19 cores=0-7
/proc/25692/stat : 25692 (python) S 25690 25692 25198 0 -1 4202496 5964 933 1 0 14 5 0 0 18 0 1 0 548499873 139010048 4151 33554432000 4194304 5683644 140733287656320 18446744073709551615 268272685061 0 2147483391 16781312 2 18446744071699199166 0 0 17 2 0 0 0
/proc/25692/statm: 33938 4151 1032 364 0 3124 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 135752

[startup+0.700689 s]
/proc/loadavg: 7.83 8.33 7.79 3/254 25692
/proc/meminfo: memFree=23346468/32950928 swapFree=67111464/67111528
[pid=25692] ppid=25690 vsize=279712 CPUtime=0.46 cores=0-7
/proc/25692/stat : 25692 (python) R 25690 25692 25198 0 -1 4202496 9592 933 1 0 33 13 0 0 17 0 1 0 548499873 286425088 7272 33554432000 4194304 5683644 140733287656320 18446744073709551615 4302585 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/25692/statm: 69928 7272 1761 364 0 5852 0
Current children cumulated CPU time (s) 0.46
Current children cumulated vsize (KiB) 279712

[startup+1.50048 s]
/proc/loadavg: 7.83 8.33 7.79 3/255 25697
/proc/meminfo: memFree=23318064/32950928 swapFree=67111464/67111528
[pid=25692] ppid=25690 vsize=285636 CPUtime=1.25 cores=0-7
/proc/25692/stat : 25692 (python) R 25690 25692 25198 0 -1 4202496 12503 933 1 0 112 13 0 0 18 0 1 0 548499873 292491264 8793 33554432000 4194304 5683644 140733287656320 18446744073709551615 47426772926499 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/25692/statm: 71409 8793 1826 364 0 7333 0
Current children cumulated CPU time (s) 1.25
Current children cumulated vsize (KiB) 285636

[startup+3.1002 s]
/proc/loadavg: 7.83 8.33 7.79 3/255 25697
/proc/meminfo: memFree=23317444/32950928 swapFree=67111464/67111528
[pid=25692] ppid=25690 vsize=362448 CPUtime=2.21 cores=0-7
/proc/25692/stat : 25692 (python) S 25690 25692 25198 0 -1 4202496 18882 933 1 0 206 15 0 0 25 0 9 0 548499873 371146752 7578 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284515787 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/25692/statm: 90612 7578 1891 364 0 26509 0
Current children cumulated CPU time (s) 2.21
Current children cumulated vsize (KiB) 362448
heavy processes:

[startup+6.30162 s]
/proc/loadavg: 7.93 8.34 7.79 10/279 25721
/proc/meminfo: memFree=23262072/32950928 swapFree=67111464/67111528
[pid=25692] ppid=25690 vsize=362448 CPUtime=2.21 cores=0-7
/proc/25692/stat : 25692 (python) S 25690 25692 25198 0 -1 4202496 18882 933 1 0 206 15 0 0 25 0 9 0 548499873 371146752 7578 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284515787 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/25692/statm: 90612 7578 1891 364 0 26509 0
[pid=25698] ppid=25692 vsize=280764 CPUtime=0 cores=0-7
/proc/25698/stat : 25698 (python) S 25692 25692 25198 0 -1 4202560 886 0 0 0 0 0 0 0 18 0 1 0 548500091 287502336 6001 33554432000 4194304 5683644 140733287656320 18446744073709551615 268272709135 0 0 16781312 514 4225401749361361154 0 0 17 2 0 0 0
/proc/25698/statm: 70191 6001 378 364 0 6109 0
[pid=25700] ppid=25698 vsize=1019708 CPUtime=4.11 cores=0-7
/proc/25700/stat : 25700 (gnovelty+2) R 25698 25700 25700 0 -1 4202496 953 0 0 0 411 0 0 0 25 0 1 0 548500092 1044180992 689 33554432000 134512640 135663704 4289355568 18446744073709551615 134524747 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/25700/statm: 254927 689 113 282 0 254643 0
[pid=25701] ppid=25692 vsize=291084 CPUtime=0 cores=0-7
/proc/25701/stat : 25701 (python) S 25692 25692 25198 0 -1 4202560 891 0 0 0 0 0 0 0 18 0 1 0 548500095 298070016 5999 33554432000 4194304 5683644 140733287656320 18446744073709551615 268272709190 0 0 16781312 514 4472857836308627713 0 0 17 2 0 0 0
/proc/25701/statm: 72771 5999 372 364 0 8686 0
[pid=25703] ppid=25701 vsize=1019708 CPUtime=4.07 cores=0-7
/proc/25703/stat : 25703 (gnovelty+2) R 25701 25703 25703 0 -1 4202496 951 0 0 0 407 0 0 0 25 0 1 0 548500095 1044180992 690 33554432000 134512640 135663704 4293239456 18446744073709551615 134524548 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/25703/statm: 254927 690 113 282 0 254643 0
[pid=25704] ppid=25692 vsize=301276 CPUtime=0 cores=0-7
/proc/25704/stat : 25704 (python) S 25692 25692 25198 0 -1 4202560 891 0 0 0 0 0 0 0 18 0 1 0 548500099 308506624 6019 33554432000 4194304 5683644 140733287656320 18446744073709551615 268272709190 0 0 16781312 514 12150589203394035970 0 0 17 0 0 0 0
/proc/25704/statm: 75319 6019 372 364 0 11231 0
[pid=25706] ppid=25704 vsize=419820 CPUtime=3.99 cores=0-7
/proc/25706/stat : 25706 (adaptg2wsat2009) R 25704 25706 25706 0 -1 4202496 1442 0 0 0 399 0 0 0 25 0 1 0 548500099 429895680 1174 33554432000 134512640 135107411 4292260608 18446744073709551615 134523457 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/25706/statm: 104955 1174 49 146 0 103831 0
[pid=25707] ppid=25692 vsize=311596 CPUtime=0 cores=0-7
/proc/25707/stat : 25707 (python) S 25692 25692 25198 0 -1 4202560 753 0 0 0 0 0 0 0 18 0 1 0 548500103 319074304 6000 33554432000 4194304 5683644 140733287656320 18446744073709551615 268272709190 0 0 16781312 514 10807377970147066116 0 0 17 2 0 0 0
/proc/25707/statm: 77899 6000 349 364 0 13808 0
[pid=25709] ppid=25707 vsize=420220 CPUtime=3.99 cores=0-7
/proc/25709/stat : 25709 (adaptg2wsat2009) R 25707 25709 25709 0 -1 4202496 1665 0 0 0 399 0 0 0 25 0 1 0 548500103 430305280 1401 33554432000 134512640 135107411 4286734336 18446744073709551615 134531246 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/25709/statm: 105055 1401 59 146 0 103931 0
[pid=25710] ppid=25692 vsize=321788 CPUtime=0 cores=0-7
/proc/25710/stat : 25710 (python) S 25692 25692 25198 0 -1 4202560 755 0 0 0 0 0 0 0 18 0 1 0 548500107 329510912 6004 33554432000 4194304 5683644 140733287656320 18446744073709551615 268272709190 0 0 16781312 514 4412670844682076419 0 0 17 0 0 0 0
/proc/25710/statm: 80447 6004 349 364 0 16353 0
[pid=25712] ppid=25710 vsize=419820 CPUtime=3.93 cores=0-7
/proc/25712/stat : 25712 (adaptg2wsat2009) R 25710 25712 25712 0 -1 4202496 1441 0 0 0 393 0 0 0 25 0 1 0 548500107 429895680 1174 33554432000 134512640 135107411 4288885792 18446744073709551615 134523424 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/25712/statm: 104955 1174 49 146 0 103831 0
[pid=25713] ppid=25692 vsize=332648 CPUtime=0 cores=0-7
/proc/25713/stat : 25713 (python) S 25692 25692 25198 0 -1 4202560 765 0 0 0 0 0 0 0 18 0 1 0 548500111 340631552 6008 33554432000 4194304 5683644 140733287656320 18446744073709551615 268272709190 0 0 16781312 514 1078409804910657795 0 0 17 1 0 0 0
/proc/25713/statm: 83162 6008 349 364 0 19065 0
[pid=25715] ppid=25713 vsize=1019708 CPUtime=3.86 cores=0-7
/proc/25715/stat : 25715 (gnovelty+2) R 25713 25715 25715 0 -1 4202496 957 0 0 0 386 0 0 0 25 0 1 0 548500113 1044180992 689 33554432000 134512640 135663704 4286860864 18446744073709551615 134525213 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/25715/statm: 254927 689 113 282 0 254643 0
[pid=25716] ppid=25692 vsize=342364 CPUtime=0 cores=0-7
/proc/25716/stat : 25716 (python) S 25692 25692 25198 0 -1 4202560 769 0 0 0 0 0 0 0 18 0 1 0 548500115 350580736 6012 33554432000 4194304 5683644 140733287656320 18446744073709551615 268272709190 0 0 16781312 514 4381065657819562244 0 0 17 1 0 0 0
/proc/25716/statm: 85591 6012 349 364 0 21491 0
[pid=25718] ppid=25716 vsize=247956 CPUtime=1.81 cores=0-7
/proc/25718/stat : 25718 (FH) R 25716 25718 25718 0 -1 4202496 1251 0 0 0 181 0 0 0 25 0 1 0 548500118 253906944 987 33554432000 134512640 134936988 4287770976 18446744073709551615 134523503 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/25718/statm: 61989 987 37 104 0 61493 0
[pid=25719] ppid=25692 vsize=353228 CPUtime=0 cores=0-7
/proc/25719/stat : 25719 (python) S 25692 25692 25198 0 -1 4202560 805 0 0 0 0 0 0 0 18 0 1 0 548500120 361705472 6039 33554432000 4194304 5683644 140733287656320 18446744073709551615 268272709190 0 0 16781312 514 4423309719192437000 0 0 17 1 0 0 0
/proc/25719/statm: 88307 6039 355 364 0 24204 0
[pid=25721] ppid=25719 vsize=419820 CPUtime=2 cores=0-7
/proc/25721/stat : 25721 (adaptg2wsat2009) R 25719 25721 25721 0 -1 4202496 1443 0 0 0 200 0 0 0 25 0 1 0 548500120 429895680 1172 33554432000 134512640 135107411 4291563168 18446744073709551615 134523314 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/25721/statm: 104955 1172 49 146 0 103831 0
[pid=25692/tid=25699] ppid=25690 vsize=362448 CPUtime=0 cores=0-7
/proc/25692/task/25699/stat : 25699 (python) S 25690 25692 25198 0 -1 4202560 23 933 0 0 0 0 0 0 21 0 9 0 548500091 371146752 7578 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 6 0 0 0
[pid=25692/tid=25702] ppid=25690 vsize=362448 CPUtime=0 cores=0-7
/proc/25692/task/25702/stat : 25702 (python) S 25690 25692 25198 0 -1 4202560 21 933 0 0 0 0 0 0 20 0 9 0 548500095 371146752 7578 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 4 0 0 0
[pid=25692/tid=25705] ppid=25690 vsize=362448 CPUtime=0 cores=0-7
/proc/25692/task/25705/stat : 25705 (python) S 25690 25692 25198 0 -1 4202560 22 933 0 0 0 0 0 0 20 0 9 0 548500099 371146752 7578 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 0 0 0 0
[pid=25692/tid=25708] ppid=25690 vsize=362448 CPUtime=0 cores=0-7
/proc/25692/task/25708/stat : 25708 (python) S 25690 25692 25198 0 -1 4202560 22 933 0 0 0 0 0 0 22 0 9 0 548500103 371146752 7578 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 0 0 0 0
[pid=25692/tid=25711] ppid=25690 vsize=362448 CPUtime=0 cores=0-7
/proc/25692/task/25711/stat : 25711 (python) S 25690 25692 25198 0 -1 4202560 26 933 0 0 0 0 0 0 24 0 9 0 548500107 371146752 7578 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 2 0 0 0
[pid=25692/tid=25714] ppid=25690 vsize=362448 CPUtime=0 cores=0-7
/proc/25692/task/25714/stat : 25714 (python) S 25690 25692 25198 0 -1 4202560 23 933 0 0 0 0 0 0 25 0 9 0 548500111 371146752 7578 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 1 0 0 0
[pid=25692/tid=25717] ppid=25690 vsize=362448 CPUtime=0 cores=0-7
/proc/25692/task/25717/stat : 25717 (python) S 25690 25692 25198 0 -1 4202560 28 933 0 0 0 0 0 0 25 0 9 0 548500115 371146752 7578 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 1 0 0 0
[pid=25692/tid=25720] ppid=25690 vsize=362448 CPUtime=0 cores=0-7
/proc/25692/task/25720/stat : 25720 (python) S 25690 25692 25198 0 -1 4202560 84 933 0 0 0 0 0 0 25 0 9 0 548500120 371146752 7578 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 29.97
Current children cumulated vsize (KiB) 7883956

[startup+12.7015 s]
/proc/loadavg: 8.01 8.35 7.80 10/279 25722
/proc/meminfo: memFree=23256772/32950928 swapFree=67111464/67111528
[pid=25692] ppid=25690 vsize=362448 CPUtime=2.21 cores=0-7
/proc/25692/stat : 25692 (python) S 25690 25692 25198 0 -1 4202496 18882 933 1 0 206 15 0 0 25 0 9 0 548499873 371146752 7578 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284515787 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/25692/statm: 90612 7578 1891 364 0 26509 0
[pid=25698] ppid=25692 vsize=280764 CPUtime=0.02 cores=0-7
/proc/25698/stat : 25698 (python) S 25692 25692 25198 0 -1 4202560 889 0 0 0 1 1 0 0 18 0 1 0 548500091 287502336 6001 33554432000 4194304 5683644 140733287656320 18446744073709551615 268272709135 0 0 16781312 514 7270709754668744962 0 0 17 2 0 0 0
/proc/25698/statm: 70191 6001 378 364 0 6109 0
[pid=25700] ppid=25698 vsize=1019708 CPUtime=10.51 cores=0-7
/proc/25700/stat : 25700 (gnovelty+2) R 25698 25700 25700 0 -1 4202496 953 0 0 0 1051 0 0 0 25 0 1 0 548500092 1044180992 689 33554432000 134512640 135663704 4289355568 18446744073709551615 134525678 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/25700/statm: 254927 689 113 282 0 254643 0
[pid=25701] ppid=25692 vsize=291084 CPUtime=0.01 cores=0-7
/proc/25701/stat : 25701 (python) S 25692 25692 25198 0 -1 4202560 895 0 0 0 0 1 0 0 18 0 1 0 548500095 298070016 5999 33554432000 4194304 5683644 140733287656320 18446744073709551615 268272709190 0 0 16781312 514 7270709754668744961 0 0 17 2 0 0 0
/proc/25701/statm: 72771 5999 372 364 0 8686 0
[pid=25703] ppid=25701 vsize=1019708 CPUtime=10.47 cores=0-7
/proc/25703/stat : 25703 (gnovelty+2) R 25701 25703 25703 0 -1 4202496 951 0 0 0 1047 0 0 0 25 0 1 0 548500095 1044180992 690 33554432000 134512640 135663704 4293239456 18446744073709551615 134525224 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/25703/statm: 254927 690 113 282 0 254643 0
[pid=25704] ppid=25692 vsize=301276 CPUtime=0.02 cores=0-7
/proc/25704/stat : 25704 (python) S 25692 25692 25198 0 -1 4202560 893 0 0 0 1 1 0 0 18 0 1 0 548500099 308506624 6019 33554432000 4194304 5683644 140733287656320 18446744073709551615 268272709190 0 0 16781312 514 7270709754668744962 0 0 17 0 0 0 0
/proc/25704/statm: 75319 6019 372 364 0 11231 0
[pid=25706] ppid=25704 vsize=420220 CPUtime=10.33 cores=0-7
/proc/25706/stat : 25706 (adaptg2wsat2009) R 25704 25706 25706 0 -1 4202496 1669 0 0 0 1033 0 0 0 25 0 1 0 548500099 430305280 1401 33554432000 134512640 135107411 4292260608 18446744073709551615 134531561 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/25706/statm: 105055 1401 59 146 0 103931 0
[pid=25707] ppid=25692 vsize=311596 CPUtime=0.02 cores=0-7
/proc/25707/stat : 25707 (python) S 25692 25692 25198 0 -1 4202560 934 0 0 0 1 1 0 0 18 0 1 0 548500103 319074304 6023 33554432000 4194304 5683644 140733287656320 18446744073709551615 268272709190 0 0 16781312 514 7270709754668744964 0 0 17 2 0 0 0
/proc/25707/statm: 77899 6023 372 364 0 13808 0
[pid=25709] ppid=25707 vsize=420220 CPUtime=10.39 cores=0-7
/proc/25709/stat : 25709 (adaptg2wsat2009) R 25707 25709 25709 0 -1 4202496 1665 0 0 0 1039 0 0 0 25 0 1 0 548500103 430305280 1401 33554432000 134512640 135107411 4286734336 18446744073709551615 134531709 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/25709/statm: 105055 1401 59 146 0 103931 0
[pid=25710] ppid=25692 vsize=321788 CPUtime=0.01 cores=0-7
/proc/25710/stat : 25710 (python) S 25692 25692 25198 0 -1 4202560 926 0 0 0 1 0 0 0 18 0 1 0 548500107 329510912 6027 33554432000 4194304 5683644 140733287656320 18446744073709551615 268272709190 0 0 16781312 514 7270709754668744963 0 0 17 0 0 0 0
/proc/25710/statm: 80447 6027 372 364 0 16353 0
[pid=25712] ppid=25710 vsize=420220 CPUtime=10.28 cores=0-7
/proc/25712/stat : 25712 (adaptg2wsat2009) R 25710 25712 25712 0 -1 4202496 1668 0 0 0 1028 0 0 0 25 0 1 0 548500107 430305280 1401 33554432000 134512640 135107411 4288885792 18446744073709551615 134532277 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/25712/statm: 105055 1401 59 146 0 103931 0
[pid=25713] ppid=25692 vsize=332648 CPUtime=0.02 cores=0-7
/proc/25713/stat : 25713 (python) S 25692 25692 25198 0 -1 4202560 942 0 0 0 1 1 0 0 18 0 1 0 548500111 340631552 6031 33554432000 4194304 5683644 140733287656320 18446744073709551615 268272709190 0 0 16781312 514 7270709754668744963 0 0 17 1 0 0 0
/proc/25713/statm: 83162 6031 372 364 0 19065 0
[pid=25715] ppid=25713 vsize=1019708 CPUtime=10.23 cores=0-7
/proc/25715/stat : 25715 (gnovelty+2) R 25713 25715 25715 0 -1 4202496 957 0 0 0 1023 0 0 0 25 0 1 0 548500113 1044180992 689 33554432000 134512640 135663704 4286860864 18446744073709551615 134524553 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/25715/statm: 254927 689 113 282 0 254643 0
[pid=25716] ppid=25692 vsize=342364 CPUtime=0.02 cores=0-7
/proc/25716/stat : 25716 (python) S 25692 25692 25198 0 -1 4202560 950 0 0 0 1 1 0 0 18 0 1 0 548500115 350580736 6035 33554432000 4194304 5683644 140733287656320 18446744073709551615 268272709190 0 0 16781312 514 7270709754668744964 0 0 17 1 0 0 0

################
# More data... #
################

[pid=25792] ppid=25790 vsize=335996 CPUtime=99.09 cores=0-7
/proc/25792/stat : 25792 (iPAWS) T 25790 25792 25792 0 -1 4202496 1815 0 0 0 9909 0 0 0 25 0 1 0 548545314 344059904 1536 33554432000 134512640 135114423 4292295376 18446744073709551615 134514774 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/25792/statm: 83999 1536 59 147 0 83851 0
[pid=25790/tid=25808] ppid=25692 vsize=562020 CPUtime=0 cores=0-7
/proc/25790/task/25808/stat : 25808 (python) S 25692 25692 25198 0 -1 4202560 98 0 0 0 0 0 0 0 19 0 2 0 548565555 575508480 6633 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 514 18446604440557478080 0 0 -1 2 0 0 0
[pid=25796] ppid=25692 vsize=572276 CPUtime=0.22 cores=0-7
/proc/25796/stat : 25796 (python) S 25692 25692 25198 0 -1 4202560 1189 0 0 0 12 10 0 0 19 0 2 0 548554769 586010624 6637 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/25796/statm: 143069 6637 407 364 0 78921 0
[pid=25798] ppid=25796 vsize=1019708 CPUtime=99.8 cores=0-7
/proc/25798/stat : 25798 (gnovelty+2) T 25796 25798 25798 0 -1 4202496 972 0 0 0 9980 0 0 0 25 0 1 0 548554770 1044180992 689 33554432000 134512640 135663704 4294691536 18446744073709551615 134545572 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/25798/statm: 254927 689 113 282 0 254643 0
[pid=25796/tid=25804] ppid=25692 vsize=572276 CPUtime=0 cores=0-7
/proc/25796/task/25804/stat : 25804 (python) S 25692 25692 25198 0 -1 4202560 97 0 0 0 0 0 0 0 19 0 2 0 548564794 586010624 6637 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 514 18446604447629284352 0 0 -1 4 0 0 0
[pid=25805] ppid=25692 vsize=582532 CPUtime=0.76 cores=0-7
/proc/25805/stat : 25805 (python) S 25692 25692 25198 0 -1 4202560 1208 0 0 0 42 34 0 0 18 0 2 0 548564803 596512768 6687 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/25805/statm: 145633 6687 406 364 0 81482 0
[pid=25807] ppid=25805 vsize=268420 CPUtime=291.42 cores=0-7
/proc/25807/stat : 25807 (hybridGM3) T 25805 25807 25807 0 -1 4202496 5408 0 0 0 29142 0 0 0 25 0 1 0 548564803 274862080 5125 33554432000 134512640 135204385 4287413792 18446744073709551615 134538520 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/25807/statm: 67105 5125 63 169 0 66935 0
[pid=25805/tid=25825] ppid=25692 vsize=582532 CPUtime=0 cores=0-7
/proc/25805/task/25825/stat : 25825 (python) S 25692 25692 25198 0 -1 4202560 105 0 0 0 0 0 0 0 18 0 2 0 548589164 596512768 6687 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 514 18446604447629283456 0 0 -1 6 0 0 0
[pid=25810] ppid=25692 vsize=593448 CPUtime=0.21 cores=0-7
/proc/25810/stat : 25810 (python) S 25692 25692 25198 0 -1 4202560 1263 0 0 0 11 10 0 0 19 0 2 0 548570806 607690752 6682 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/25810/statm: 148362 6682 401 364 0 84208 0
[pid=25812] ppid=25810 vsize=335996 CPUtime=97.46 cores=0-7
/proc/25812/stat : 25812 (iPAWS) T 25810 25812 25812 0 -1 4202496 1824 0 0 0 9746 0 0 0 25 0 1 0 548570807 344059904 1536 33554432000 134512640 135114423 4293663744 18446744073709551615 134517031 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/25812/statm: 83999 1536 59 147 0 83851 0
[pid=25810/tid=25816] ppid=25692 vsize=593448 CPUtime=0 cores=0-7
/proc/25810/task/25816/stat : 25816 (python) S 25692 25692 25198 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 548580827 607690752 6682 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 514 18446604452420671424 0 0 -1 1 0 0 0
[pid=25817] ppid=25692 vsize=603348 CPUtime=0.8 cores=0-7
/proc/25817/stat : 25817 (python) S 25692 25692 25198 0 -1 4202560 1290 0 0 0 41 39 0 0 18 0 2 0 548580836 617828352 6705 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/25817/statm: 150837 6705 402 364 0 86680 0
[pid=25819] ppid=25817 vsize=268420 CPUtime=293.86 cores=0-7
/proc/25819/stat : 25819 (hybridGM3) T 25817 25819 25819 0 -1 4202496 5418 0 0 0 29386 0 0 0 25 0 1 0 548580837 274862080 5126 33554432000 134512640 135204385 4288975232 18446744073709551615 134538284 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/25819/statm: 67105 5126 63 169 0 66935 0
[pid=25817/tid=25842] ppid=25692 vsize=603348 CPUtime=0 cores=0-7
/proc/25817/task/25842/stat : 25842 (python) S 25692 25692 25198 0 -1 4202560 47 0 0 0 0 0 0 0 18 0 2 0 548605300 617828352 6705 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 514 18446604453931682752 0 0 -1 1 0 0 0
[pid=25821] ppid=25692 vsize=613848 CPUtime=1.1 cores=0-7
/proc/25821/stat : 25821 (python) S 25692 25692 25198 0 -1 4202560 1143 0 0 0 55 55 0 0 18 0 1 0 548586328 628580352 6708 33554432000 4194304 5683644 140733287656320 18446744073709551615 268272709190 0 0 16781312 514 7270709754668744961 0 0 17 4 0 0 0
/proc/25821/statm: 153462 6708 373 364 0 89302 0
[pid=25823] ppid=25821 vsize=458352 CPUtime=428.01 cores=0-7
/proc/25823/stat : 25823 (TNM) R 25821 25823 25823 0 -1 4202496 1525 0 0 0 42801 0 0 0 25 0 1 0 548586329 469352448 1239 33554432000 134512640 135008748 4294637568 18446744073709551615 134527729 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/25823/statm: 114588 1239 46 122 0 113488 0
[pid=25829] ppid=25692 vsize=624156 CPUtime=0.6 cores=0-7
/proc/25829/stat : 25829 (python) S 25692 25692 25198 0 -1 4202560 1303 0 0 0 31 29 0 0 18 0 2 0 548599807 639135744 6802 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/25829/statm: 156039 6802 401 364 0 91876 0
[pid=25831] ppid=25829 vsize=249876 CPUtime=197.56 cores=0-7
/proc/25831/stat : 25831 (FH) T 25829 25831 25831 0 -1 4202496 1554 0 0 0 19756 0 0 0 25 0 1 0 548599808 255873024 1264 33554432000 134512640 134936988 4293421088 18446744073709551615 134544807 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/25831/statm: 62469 1264 41 104 0 61973 0
[pid=25829/tid=25845] ppid=25692 vsize=624156 CPUtime=0 cores=0-7
/proc/25829/task/25845/stat : 25845 (python) S 25692 25692 25198 0 -1 4202560 47 0 0 0 0 0 0 0 18 0 2 0 548612836 639135744 6802 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 514 18446604451528394752 0 0 -1 1 0 0 0
[pid=25848] ppid=25692 vsize=634592 CPUtime=0.32 cores=0-7
/proc/25848/stat : 25848 (python) S 25692 25692 25198 0 -1 4202560 1344 0 0 0 16 16 0 0 19 0 2 0 548617245 649822208 6825 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/25848/statm: 158648 6825 407 364 0 94482 0
[pid=25850] ppid=25848 vsize=1019708 CPUtime=98.28 cores=0-7
/proc/25850/stat : 25850 (gnovelty+2) T 25848 25850 25850 0 -1 4202496 987 0 0 0 9828 0 0 0 25 0 1 0 548617245 1044180992 690 33554432000 134512640 135663704 4286591600 18446744073709551615 134525353 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/25850/statm: 254927 690 113 282 0 254643 0
[pid=25848/tid=25863] ppid=25692 vsize=634592 CPUtime=0 cores=0-7
/proc/25848/task/25863/stat : 25863 (python) S 25692 25692 25198 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 548630881 649822208 6825 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 514 18446604451528392960 0 0 -1 0 0 0 0
[pid=25856] ppid=25692 vsize=645544 CPUtime=0.26 cores=0-7
/proc/25856/stat : 25856 (python) S 25692 25692 25198 0 -1 4202560 1153 0 0 0 15 11 0 0 18 0 1 0 548625747 661037056 6925 33554432000 4194304 5683644 140733287656320 18446744073709551615 268272709190 0 0 16781312 514 7270709754668744962 0 0 17 7 0 0 0
/proc/25856/statm: 161386 6925 380 364 0 97217 0
[pid=25858] ppid=25856 vsize=268424 CPUtime=107.42 cores=0-7
/proc/25858/stat : 25858 (hybridGM3) R 25856 25858 25858 0 -1 4202496 5419 0 0 0 10742 0 0 0 25 0 1 0 548625748 274866176 5125 33554432000 134512640 135204385 4287450208 18446744073709551615 134538375 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/25858/statm: 67106 5125 63 169 0 66936 0
[pid=25859] ppid=25692 vsize=655472 CPUtime=0.24 cores=0-7
/proc/25859/stat : 25859 (python) S 25692 25692 25198 0 -1 4202560 1136 0 0 0 11 13 0 0 18 0 1 0 548625984 671203328 6926 33554432000 4194304 5683644 140733287656320 18446744073709551615 268272709190 0 0 16781312 514 7270709754668744962 0 0 17 1 0 0 0
/proc/25859/statm: 163868 6926 377 364 0 99696 0
[pid=25861] ppid=25859 vsize=335996 CPUtime=68.59 cores=0-7
/proc/25861/stat : 25861 (iPAWS) R 25859 25861 25861 0 -1 4202496 1826 0 0 0 6859 0 0 0 25 0 1 0 548625985 344059904 1535 33554432000 134512640 135114423 4289075808 18446744073709551615 134515597 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/25861/statm: 83999 1535 59 147 0 83851 0
[pid=25864] ppid=25692 vsize=665876 CPUtime=0.13 cores=0-7
/proc/25864/stat : 25864 (python) S 25692 25692 25198 0 -1 4202560 1133 0 0 0 6 7 0 0 18 0 1 0 548630892 681857024 6950 33554432000 4194304 5683644 140733287656320 18446744073709551615 268272709190 0 0 16781312 514 7270709754668744962 0 0 17 1 0 0 0
/proc/25864/statm: 166469 6950 379 364 0 102294 0
[pid=25866] ppid=25864 vsize=268420 CPUtime=47.55 cores=0-7
/proc/25866/stat : 25866 (hybridGM3) R 25864 25866 25866 0 -1 4202496 5419 0 0 0 4755 0 0 0 25 0 1 0 548630892 274862080 5126 33554432000 134512640 135204385 4291523216 18446744073709551615 134538041 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/25866/statm: 67105 5126 63 169 0 66935 0
[pid=25869] ppid=25692 vsize=676388 CPUtime=0 cores=0-7
/proc/25869/stat : 25869 (python) S 25692 25692 25198 0 -1 4202560 943 0 0 0 0 0 0 0 19 0 1 0 548636951 692621312 6938 33554432000 4194304 5683644 140733287656320 18446744073709551615 268272709190 0 0 16781312 514 6003902214341394431 0 0 17 1 0 0 0
/proc/25869/statm: 169097 6938 346 364 0 104919 0
[pid=25871] ppid=25869 vsize=415124 CPUtime=0.43 cores=0-7
/proc/25871/stat : 25871 (NCVWr) R 25869 25871 25871 0 -1 4202496 1299 0 0 0 43 0 0 0 21 0 1 0 548636951 425086976 1007 33554432000 134512640 135113558 4293828016 18446744073709551615 134526317 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/25871/statm: 103781 1007 55 147 0 102656 0
[pid=25692/tid=25699] ppid=25690 vsize=686368 CPUtime=0 cores=0-7
/proc/25692/task/25699/stat : 25699 (python) S 25690 25692 25198 0 -1 4202560 23 933 0 0 0 0 0 0 21 0 34 0 548500091 702840832 8537 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 6 0 0 0
[pid=25692/tid=25702] ppid=25690 vsize=686368 CPUtime=0 cores=0-7
/proc/25692/task/25702/stat : 25702 (python) S 25690 25692 25198 0 -1 4202560 21 933 0 0 0 0 0 0 20 0 34 0 548500095 702840832 8537 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 4 0 0 0
[pid=25692/tid=25705] ppid=25690 vsize=686368 CPUtime=0 cores=0-7
/proc/25692/task/25705/stat : 25705 (python) S 25690 25692 25198 0 -1 4202560 27 933 0 0 0 0 0 0 18 0 34 0 548500099 702840832 8537 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 0 0 0 0
[pid=25692/tid=25708] ppid=25690 vsize=686368 CPUtime=0 cores=0-7
/proc/25692/task/25708/stat : 25708 (python) S 25690 25692 25198 0 -1 4202560 30 933 0 0 0 0 0 0 18 0 34 0 548500103 702840832 8537 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 0 0 0 0
[pid=25692/tid=25711] ppid=25690 vsize=686368 CPUtime=0 cores=0-7
/proc/25692/task/25711/stat : 25711 (python) S 25690 25692 25198 0 -1 4202560 30 933 0 0 0 0 0 0 18 0 34 0 548500107 702840832 8537 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 2 0 0 0
[pid=25692/tid=25714] ppid=25690 vsize=686368 CPUtime=0 cores=0-7
/proc/25692/task/25714/stat : 25714 (python) S 25690 25692 25198 0 -1 4202560 23 933 0 0 0 0 0 0 25 0 34 0 548500111 702840832 8537 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 1 0 0 0
[pid=25692/tid=25717] ppid=25690 vsize=686368 CPUtime=0 cores=0-7
/proc/25692/task/25717/stat : 25717 (python) S 25690 25692 25198 0 -1 4202560 34 933 0 0 0 0 0 0 18 0 34 0 548500115 702840832 8537 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 1 0 0 0
[pid=25692/tid=25720] ppid=25690 vsize=686368 CPUtime=0 cores=0-7
/proc/25692/task/25720/stat : 25720 (python) S 25690 25692 25198 0 -1 4202560 104 933 0 0 0 0 0 0 18 0 34 0 548500120 702840832 8537 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 1 0 0 0
[pid=25692/tid=25728] ppid=25690 vsize=686368 CPUtime=0 cores=0-7
/proc/25692/task/25728/stat : 25728 (python) S 25690 25692 25198 0 -1 4202560 31 933 0 0 0 0 0 0 18 0 34 0 548510118 702840832 8537 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 0 0 0 0
[pid=25692/tid=25732] ppid=25690 vsize=686368 CPUtime=0 cores=0-7
/proc/25692/task/25732/stat : 25732 (python) S 25690 25692 25198 0 -1 4202560 108 933 0 0 0 0 0 0 18 0 34 0 548510126 702840832 8537 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 1 0 0 0
[pid=25692/tid=25735] ppid=25690 vsize=686368 CPUtime=0 cores=0-7
/proc/25692/task/25735/stat : 25735 (python) S 25690 25692 25198 0 -1 4202560 41 933 0 0 0 0 0 0 18 0 34 0 548510131 702840832 8537 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 1 0 0 0
[pid=25692/tid=25739] ppid=25690 vsize=686368 CPUtime=0 cores=0-7
/proc/25692/task/25739/stat : 25739 (python) S 25690 25692 25198 0 -1 4202560 33 933 0 0 0 0 0 0 22 0 34 0 548510143 702840832 8537 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 2 0 0 0
[pid=25692/tid=25743] ppid=25690 vsize=686368 CPUtime=0 cores=0-7
/proc/25692/task/25743/stat : 25743 (python) S 25690 25692 25198 0 -1 4202560 32 933 0 0 0 0 0 0 18 0 34 0 548510724 702840832 8537 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 2 0 0 0
[pid=25692/tid=25748] ppid=25690 vsize=686368 CPUtime=0 cores=0-7
/proc/25692/task/25748/stat : 25748 (python) S 25690 25692 25198 0 -1 4202560 43 933 0 0 0 0 0 0 18 0 34 0 548512535 702840832 8537 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 2 0 0 0
[pid=25692/tid=25752] ppid=25690 vsize=686368 CPUtime=0 cores=0-7
/proc/25692/task/25752/stat : 25752 (python) S 25690 25692 25198 0 -1 4202560 48 933 0 0 0 0 0 0 18 0 34 0 548513161 702840832 8537 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 2 0 0 0
[pid=25692/tid=25756] ppid=25690 vsize=686368 CPUtime=0 cores=0-7
/proc/25692/task/25756/stat : 25756 (python) S 25690 25692 25198 0 -1 4202560 30 933 0 0 0 0 0 0 18 0 34 0 548514354 702840832 8537 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 2 0 0 0
[pid=25692/tid=25762] ppid=25690 vsize=686368 CPUtime=0 cores=0-7
/proc/25692/task/25762/stat : 25762 (python) S 25690 25692 25198 0 -1 4202560 31 933 0 0 0 0 0 0 20 0 34 0 548520143 702840832 8537 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 1 0 0 0
[pid=25692/tid=25766] ppid=25690 vsize=686368 CPUtime=0 cores=0-7
/proc/25692/task/25766/stat : 25766 (python) S 25690 25692 25198 0 -1 4202560 125 933 0 0 0 0 0 0 20 0 34 0 548520158 702840832 8537 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 2 0 0 0
[pid=25692/tid=25780] ppid=25690 vsize=686368 CPUtime=0 cores=0-7
/proc/25692/task/25780/stat : 25780 (python) S 25690 25692 25198 0 -1 4202560 68 933 0 0 0 0 0 0 18 0 34 0 548537699 702840832 8537 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 2 0 0 0
[pid=25692/tid=25784] ppid=25690 vsize=686368 CPUtime=0 cores=0-7
/proc/25692/task/25784/stat : 25784 (python) S 25690 25692 25198 0 -1 4202560 40 933 0 0 0 0 0 0 18 0 34 0 548541021 702840832 8537 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 2 0 0 0
[pid=25692/tid=25788] ppid=25690 vsize=686368 CPUtime=0 cores=0-7
/proc/25692/task/25788/stat : 25788 (python) S 25690 25692 25198 0 -1 4202560 36 933 0 0 0 0 0 0 20 0 34 0 548542315 702840832 8537 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 2 0 0 0
[pid=25692/tid=25791] ppid=25690 vsize=686368 CPUtime=0 cores=0-7
/proc/25692/task/25791/stat : 25791 (python) S 25690 25692 25198 0 -1 4202560 38 933 0 0 0 0 0 0 20 0 34 0 548545314 702840832 8537 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 1 0 0 0
[pid=25692/tid=25797] ppid=25690 vsize=686368 CPUtime=0 cores=0-7
/proc/25692/task/25797/stat : 25797 (python) S 25690 25692 25198 0 -1 4202560 39 933 0 0 0 0 0 0 20 0 34 0 548554769 702840832 8537 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 2 0 0 0
[pid=25692/tid=25806] ppid=25690 vsize=686368 CPUtime=0 cores=0-7
/proc/25692/task/25806/stat : 25806 (python) S 25690 25692 25198 0 -1 4202560 45 933 0 0 0 0 0 0 18 0 34 0 548564803 702840832 8537 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 2 0 0 0
[pid=25692/tid=25811] ppid=25690 vsize=686368 CPUtime=0 cores=0-7
/proc/25692/task/25811/stat : 25811 (python) S 25690 25692 25198 0 -1 4202560 31 933 0 0 0 0 0 0 20 0 34 0 548570806 702840832 8537 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 2 0 0 0
[pid=25692/tid=25818] ppid=25690 vsize=686368 CPUtime=0 cores=0-7
/proc/25692/task/25818/stat : 25818 (python) S 25690 25692 25198 0 -1 4202560 36 933 0 0 0 0 0 0 18 0 34 0 548580836 702840832 8537 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 2 0 0 0
[pid=25692/tid=25822] ppid=25690 vsize=686368 CPUtime=0 cores=0-7
/proc/25692/task/25822/stat : 25822 (python) S 25690 25692 25198 0 -1 4202560 29 933 0 0 0 0 0 0 20 0 34 0 548586328 702840832 8537 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 2 0 0 0
[pid=25692/tid=25830] ppid=25690 vsize=686368 CPUtime=0 cores=0-7
/proc/25692/task/25830/stat : 25830 (python) S 25690 25692 25198 0 -1 4202560 28 933 0 0 0 0 0 0 18 0 34 0 548599808 702840832 8537 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 2 0 0 0
[pid=25692/tid=25849] ppid=25690 vsize=686368 CPUtime=0 cores=0-7
/proc/25692/task/25849/stat : 25849 (python) S 25690 25692 25198 0 -1 4202560 29 933 0 0 0 0 0 0 19 0 34 0 548617245 702840832 8537 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 2 0 0 0
[pid=25692/tid=25857] ppid=25690 vsize=686368 CPUtime=0 cores=0-7
/proc/25692/task/25857/stat : 25857 (python) S 25690 25692 25198 0 -1 4202560 29 933 0 0 0 0 0 0 20 0 34 0 548625747 702840832 8537 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 2 0 0 0
[pid=25692/tid=25860] ppid=25690 vsize=686368 CPUtime=0 cores=0-7
/proc/25692/task/25860/stat : 25860 (python) S 25690 25692 25198 0 -1 4202560 58 933 0 0 0 0 0 0 19 0 34 0 548625985 702840832 8537 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 3 0 0 0
[pid=25692/tid=25865] ppid=25690 vsize=686368 CPUtime=0 cores=0-7
/proc/25692/task/25865/stat : 25865 (python) S 25690 25692 25198 0 -1 4202560 28 933 0 0 0 0 0 0 19 0 34 0 548630892 702840832 8537 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 2 0 0 0
[pid=25692/tid=25870] ppid=25690 vsize=686368 CPUtime=0 cores=0-7
/proc/25692/task/25870/stat : 25870 (python) S 25690 25692 25198 0 -1 4202560 29 933 0 0 0 0 0 0 19 0 34 0 548636951 702840832 8537 33554432000 4194304 5683644 140733287656320 18446744073709551615 268284512689 0 0 16781312 2 18446604440557480768 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 9624.96
Current children cumulated vsize (KiB) 32284808

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 25692 and gives
#  childrusage.ru_utime.tv_sec=6
#  childrusage.ru_utime.tv_usec=253049
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=244962
# CPU time returned by wait4() is 6.49801
# while last known CPU time is 9624.96
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1371.81
CPU time (s): 9624.96
CPU user time (s): 9613.6
CPU system time (s): 11.36
CPU usage (%): 701.627
Max. virtual memory (cumulated for all children) (KiB): 32284808

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 6.25305
system time used= 0.244962
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 56374
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3304
involuntary context switches= 108

runsolver used 13.248 second user time and 31.0873 second system time

The end

Launcher Data 

Begin job on node122 at 2011-05-02 22:43:31
IDJOB=3321882
IDBENCH=82660
IDSOLVER=1712
FILE ID=node122/3321882-1304369011
RUNJOBID= node122-1304366467-25212
PBS_JOBID= 13181749
Free space on /tmp= 73504 MiB

SOLVER NAME= borg-sat 11.03.02-random
BENCH NAME= SAT11/random/large/unif-k7-r85-v300-c25500-S69399346-065-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3321882-1304369011/watcher-3321882-1304369011 -o /tmp/evaluation-result-3321882-1304369011/solver-3321882-1304369011 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3321882-1304369011.cnf -seed 353230720 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 98486271f2f72889c040eabfd36ebd13
RANDOM SEED=353230720

node122.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.64
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.97
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      23346624 kB
Buffers:        383724 kB
Cached:        3120780 kB
SwapCached:         64 kB
Active:        6308208 kB
Inactive:      2919820 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      23346624 kB
SwapTotal:    67111528 kB
SwapFree:     67111464 kB
Dirty:           32744 kB
Writeback:           0 kB
AnonPages:     5723648 kB
Mapped:          21408 kB
Slab:           287280 kB
PageTables:      22312 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 30396968 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73472 MiB
End job on node122 at 2011-05-02 23:06:25