Trace number 3385424

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)? (MO) 10487.3 1497.22

General information on the benchmark

Namerandom/large/
unif-k7-r85-v250-c21250-S58287803-001-UNKNOWN.cnf
MD5SUM0d5d355ade4a956f83626786ca6018b7
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables250
Number of clauses21250
Sum of the clauses size148750
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 521250

Solver Data 

0.37/0.67	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.37/0.67	c INFO: solving HOME/instance-3385424-1305566469.cnf
0.37/0.67	c INFO: computing task features
0.54/0.81	c INFO: computed variable-clause graph statistics
0.54/0.82	c INFO: computed clause balance statistics
1.24/1.55	c INFO: computed variable balance statistics
1.54/1.85	c INFO: computed small-clause counts
1.54/1.85	c INFO: computed Horn-clause counts
1.54/1.87	c INFO: computed variable graph statistics
1.54/1.87	c INFO: collected features for HOME/instance-3385424-1305566469.cnf in 1.20 s
1.64/1.90	c INFO: running gnovelty+2@0 for 97 with 4998 remaining (b = 0.36)
1.64/1.94	c INFO: running gnovelty+2@0 for 97 with 4998 remaining (b = 0.13)
1.64/1.97	c INFO: running adaptg2wsat2009++@0 for 97 with 4998 remaining (b = 0.08)
1.74/2.00	c INFO: running adaptg2wsat2009++@0 for 97 with 4998 remaining (b = 0.07)
1.74/2.03	c INFO: running adaptg2wsat2009++@0 for 97 with 4998 remaining (b = 0.05)
1.74/2.06	c INFO: running gnovelty+2@0 for 97 with 4998 remaining (b = 0.04)
1.74/2.10	c INFO: running FH@0 for 97 with 4998 remaining (b = 0.03)
2.50/2.13	c INFO: running adaptg2wsat2009++@0 for 97 with 4998 remaining (b = 0.03)
701.62/102.30	c INFO: running adaptg2wsat2009++@0 for 97 with 4898 remaining (b = 0.02)
702.32/102.33	c INFO: running FH@0 for 97 with 4898 remaining (b = 0.02)
702.32/102.37	c INFO: running hybridGM3@0 for 194 with 4898 remaining (b = 0.04)
703.64/102.68	c INFO: running gnovelty+2@0 for 97 with 4898 remaining (b = 0.02)
743.96/108.32	c INFO: running hybridGM3@0 for 194 with 4892 remaining (b = 0.03)
829.02/120.59	c INFO: running hybridGM3@0 for 194 with 4880 remaining (b = 0.03)
954.68/138.60	c INFO: running TNM@0 for 1552 with 4862 remaining (b = 0.20)
1037.61/150.42	c INFO: running adaptg2wsat2009++@0 for 97 with 4850 remaining (b = 0.01)
1401.70/202.69	c INFO: running FH@0 for 97 with 4798 remaining (b = 0.01)
1490.20/215.35	c INFO: running hybridGM3@0 for 194 with 4785 remaining (b = 0.02)
1527.82/220.79	c INFO: running adaptg2wsat2009++@0 for 97 with 4779 remaining (b = 0.01)
1953.84/280.87	c INFO: running adaptg2wsat2009++@0 for 97 with 4719 remaining (b = 0.01)
2164.24/311.10	c INFO: running hybridGM3@97 for 97 with 4689 remaining (b = 0.01)
2192.10/315.09	c INFO: running adaptg2wsat2009++@0 for 97 with 4685 remaining (b = 0.01)
2206.73/317.15	c INFO: running hybridGM3@99 for 97 with 4683 remaining (b = 0.01)
2334.30/335.46	c INFO: running hybridGM3@99 for 97 with 4665 remaining (b = 0.01)
2507.01/360.27	c INFO: running adaptg2wsat2009++@100 for 97 with 4640 remaining (b = 0.01)
2811.47/402.30	c INFO: running hybridGM3@0 for 194 with 4598 remaining (b = 0.02)
2923.60/418.46	c INFO: running hybridGM3@0 for 194 with 4582 remaining (b = 0.02)
2924.20/418.52	c INFO: running TNM@0 for 1552 with 4582 remaining (b = 0.15)
2924.20/418.58	c INFO: running hybridGM3@299 for 97 with 4582 remaining (b = 0.01)
3135.98/448.92	c INFO: running adaptg2wsat2009++@97 for 97 with 4551 remaining (b = 0.01)
3173.01/454.25	c INFO: running gnovelty+2@0 for 97 with 4546 remaining (b = 0.01)
3399.26/486.77	c INFO: running hybridGM3@97 for 97 with 4513 remaining (b = 0.01)
3672.31/525.97	c INFO: running hybridGM3@394 for 97 with 4474 remaining (b = 0.01)
3998.36/572.75	c INFO: running FH@0 for 291 with 4427 remaining (b = 0.03)
4149.49/594.43	c INFO: running iPAWS@0 for 97 with 4406 remaining (b = 0.01)
4203.82/602.21	c INFO: running adaptg2wsat2009++@294 for 291 with 4398 remaining (b = 0.02)
4217.05/604.16	c INFO: running hybridGM3@95 for 97 with 4396 remaining (b = 0.01)
4336.84/621.34	c INFO: running hybridGM3@0 for 194 with 4379 remaining (b = 0.02)
4512.26/645.32	c INFO: running hybridGM3@96 for 97 with 4355 remaining (b = 0.01)
4936.88/705.54	c INFO: running hybridGM3@187 for 97 with 4295 remaining (b = 0.01)
4946.63/706.86	c INFO: running FH@99 for 291 with 4293 remaining (b = 0.02)
5427.27/775.87	c INFO: running hybridGM3@193 for 97 with 4224 remaining (b = 0.01)
5805.83/830.17	c INFO: running hybridGM3@0 for 194 with 4170 remaining (b = 0.01)
5937.35/849.05	c INFO: running TNM@0 for 1552 with 4151 remaining (b = 0.12)
6159.86/880.79	c INFO: running iPAWS@0 for 97 with 4119 remaining (b = 0.01)
6386.73/913.36	c INFO: running hybridGM3@387 for 194 with 4087 remaining (b = 0.02)
6532.86/934.33	c INFO: running adaptg2wsat2009++@490 for 97 with 4066 remaining (b = 0.01)
7026.32/1005.26	c INFO: running adaptg2wsat2009++@99 for 97 with 3995 remaining (b = 0.01)
7275.46/1041.05	c INFO: running gnovelty+2@0 for 97 with 3959 remaining (b = 0.01)
7531.60/1077.88	c INFO: running hybridGM3@100 for 97 with 3922 remaining (b = 0.01)
7672.81/1098.17	c INFO: running FH@197 for 194 with 3902 remaining (b = 0.01)
7871.86/1126.71	c INFO: running adaptg2wsat2009++@299 for 291 with 3873 remaining (b = 0.02)
7930.45/1135.14	c INFO: running NCVWr@0 for 485 with 3865 remaining (b = 0.03)
8009.66/1146.53	c INFO: running FH@99 for 291 with 3854 remaining (b = 0.02)
8363.90/1197.42	c INFO: running hybridGM3@390 for 97 with 3803 remaining (b = 0.01)
9139.14/1308.87	c INFO: running iPAWS@0 for 97 with 3691 remaining (b = 0.01)
9299.89/1331.99	c INFO: running FH@296 for 97 with 3668 remaining (b = 0.01)
9879.55/1415.27	c INFO: running FH@0 for 97 with 3585 remaining (b = 0.01)
10194.20/1457.47	c INFO: running hybridGM3@0 for 194 with 3543 remaining (b = 0.01)
10232.40/1462.84	c INFO: running adaptg2wsat2009++@492 for 97 with 3537 remaining (b = 0.01)
10484.81/1496.73	c INFO: running NCVWr@0 for 485 with 3503 remaining (b = 0.03)
10487.31/1497.17	c INFO: running iPAWS@0 for 97 with 3503 remaining (b = 0.01)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3385424-1305566469/watcher-3385424-1305566469 -o /tmp/evaluation-result-3385424-1305566469/solver-3385424-1305566469 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385424-1305566469.cnf -seed 1823898591 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 9.24 8.71 6.82 2/403 31268
/proc/meminfo: memFree=23845536/32951124 swapFree=46575272/67111528
[pid=31268] ppid=31266 vsize=15772 CPUtime=0 cores=0-7
/proc/31268/stat : 31268 (python) S 31266 31268 30681 0 -1 4202496 926 0 0 0 0 0 0 0 22 0 1 0 400838245 16150528 806 33554432000 4194304 5683644 140737437777424 18446744073709551615 268660660224 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 0
/proc/31268/statm: 3943 806 356 364 0 430 0

[startup+0.049707 s]
/proc/loadavg: 9.24 8.71 6.82 2/403 31268
/proc/meminfo: memFree=23845536/32951124 swapFree=46575272/67111528
[pid=31268] ppid=31266 vsize=18880 CPUtime=0.02 cores=0-7
/proc/31268/stat : 31268 (python) R 31266 31268 30681 0 -1 4202496 1287 0 0 0 2 0 0 0 22 0 1 0 400838245 19333120 1140 33554432000 4194304 5683644 140737437777424 18446744073709551615 5282100 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/31268/statm: 4720 1140 421 364 0 689 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 18880

[startup+0.1007 s]
/proc/loadavg: 9.24 8.71 6.82 2/403 31268
/proc/meminfo: memFree=23845536/32951124 swapFree=46575272/67111528
[pid=31268] ppid=31266 vsize=87704 CPUtime=0.07 cores=0-7
/proc/31268/stat : 31268 (python) R 31266 31268 30681 0 -1 4202496 2637 0 0 0 5 2 0 0 22 0 1 0 400838245 89808896 2354 33554432000 4194304 5683644 140737437777424 18446744073709551615 4539840 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/31268/statm: 21925 2353 717 364 0 1621 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 87704

[startup+0.300667 s]
/proc/loadavg: 9.24 8.71 6.82 2/403 31268
/proc/meminfo: memFree=23845536/32951124 swapFree=46575272/67111528
[pid=31268] ppid=31266 vsize=135748 CPUtime=0.19 cores=0-7
/proc/31268/stat : 31268 (python) R 31266 31268 30681 0 -1 4202496 5961 935 1 0 13 6 0 0 17 0 1 0 400838245 139005952 4150 33554432000 4194304 5683644 140737437777424 18446744073709551615 268660658181 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/31268/statm: 33937 4150 1032 364 0 3123 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 135748

[startup+0.70062 s]
/proc/loadavg: 9.24 8.71 6.82 2/403 31268
/proc/meminfo: memFree=23845536/32951124 swapFree=46575272/67111528
[pid=31268] ppid=31266 vsize=279508 CPUtime=0.44 cores=0-7
/proc/31268/stat : 31268 (python) R 31266 31268 30681 0 -1 4202496 9532 935 1 0 31 13 0 0 16 0 1 0 400838245 286216192 7218 33554432000 4194304 5683644 140737437777424 18446744073709551615 4445115 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/31268/statm: 69877 7218 1761 364 0 5801 0
Current children cumulated CPU time (s) 0.44
Current children cumulated vsize (KiB) 279508

[startup+1.50047 s]
/proc/loadavg: 9.24 8.71 6.82 3/404 31273
/proc/meminfo: memFree=23817816/32951124 swapFree=46575272/67111528
[pid=31268] ppid=31266 vsize=283912 CPUtime=1.24 cores=0-7
/proc/31268/stat : 31268 (python) R 31266 31268 30681 0 -1 4202496 11838 935 1 0 111 13 0 0 17 0 1 0 400838245 290725888 8363 33554432000 4194304 5683644 140737437777424 18446744073709551615 268660310951 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/31268/statm: 70978 8363 1826 364 0 6902 0
Current children cumulated CPU time (s) 1.24
Current children cumulated vsize (KiB) 283912

[startup+3.10114 s]
/proc/loadavg: 8.66 8.60 6.79 9/422 31291
/proc/meminfo: memFree=23780960/32951124 swapFree=46575272/67111528
[pid=31268] ppid=31266 vsize=362116 CPUtime=1.87 cores=0-7
/proc/31268/stat : 31268 (python) S 31266 31268 30681 0 -1 4202496 18182 935 1 0 172 15 0 0 23 0 9 0 400838245 370806784 7496 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672488907 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/31268/statm: 90529 7496 1891 364 0 26426 0
[pid=31274] ppid=31268 vsize=279924 CPUtime=0 cores=0-7
/proc/31274/stat : 31274 (python) S 31268 31268 30681 0 -1 4202560 739 0 0 0 0 0 0 0 18 0 1 0 400838436 286642176 5769 33554432000 4194304 5683644 140737437777424 18446744073709551615 268660682255 0 0 16781312 514 13496785885588324611 0 0 17 6 0 0 0
/proc/31274/statm: 69981 5769 355 364 0 5899 0
[pid=31276] ppid=31274 vsize=1018820 CPUtime=1.18 cores=0-7
/proc/31276/stat : 31276 (gnovelty+2) R 31274 31276 31276 0 -1 4202496 858 0 0 0 118 0 0 0 22 0 1 0 400838437 1043271680 596 33554432000 134512640 135663704 4291676112 18446744073709551615 134524548 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/31276/statm: 254705 596 113 282 0 254421 0
[pid=31277] ppid=31268 vsize=290820 CPUtime=0 cores=0-7
/proc/31277/stat : 31277 (python) S 31268 31268 30681 0 -1 4202560 744 0 0 0 0 0 0 0 18 0 1 0 400838439 297799680 5792 33554432000 4194304 5683644 140737437777424 18446744073709551615 268660682310 0 0 16781312 514 4279479579506082050 0 0 17 0 0 0 0
/proc/31277/statm: 72705 5792 349 364 0 8620 0
[pid=31279] ppid=31277 vsize=1018820 CPUtime=1.15 cores=0-7
/proc/31279/stat : 31279 (gnovelty+2) R 31277 31279 31279 0 -1 4202496 858 0 0 0 115 0 0 0 25 0 1 0 400838440 1043271680 596 33554432000 134512640 135663704 4292375808 18446744073709551615 134525255 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/31279/statm: 254705 596 113 282 0 254421 0
[pid=31280] ppid=31268 vsize=300592 CPUtime=0 cores=0-7
/proc/31280/stat : 31280 (python) S 31268 31268 30681 0 -1 4202560 739 0 0 0 0 0 0 0 18 0 1 0 400838443 307806208 5929 33554432000 4194304 5683644 140737437777424 18446744073709551615 268660682310 0 0 16781312 514 2414095980577161215 0 0 17 2 0 0 0
/proc/31280/statm: 75148 5929 347 364 0 11060 0
[pid=31282] ppid=31280 vsize=419160 CPUtime=1.1 cores=0-7
/proc/31282/stat : 31282 (adaptg2wsat2009) R 31280 31282 31282 0 -1 4202496 1260 0 0 0 110 0 0 0 25 0 1 0 400838443 429219840 992 33554432000 134512640 135107411 4287004944 18446744073709551615 134523319 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/31282/statm: 104790 992 49 146 0 103666 0
[pid=31283] ppid=31268 vsize=310848 CPUtime=0 cores=0-7
/proc/31283/stat : 31283 (python) S 31268 31268 30681 0 -1 4202560 748 0 0 0 0 0 0 0 18 0 1 0 400838446 318308352 5932 33554432000 4194304 5683644 140737437777424 18446744073709551615 268660682310 0 0 16781312 514 6348276128108511231 0 0 17 6 0 0 0
/proc/31283/statm: 77712 5932 347 364 0 13621 0
[pid=31285] ppid=31283 vsize=419160 CPUtime=1.09 cores=0-7
/proc/31285/stat : 31285 (adaptg2wsat2009) R 31283 31285 31285 0 -1 4202496 1254 0 0 0 109 0 0 0 24 0 1 0 400838446 429219840 990 33554432000 134512640 135107411 4287351936 18446744073709551615 134523300 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/31285/statm: 104790 990 49 146 0 103666 0
[pid=31286] ppid=31268 vsize=321104 CPUtime=0 cores=0-7
/proc/31286/stat : 31286 (python) S 31268 31268 30681 0 -1 4202560 757 0 0 0 0 0 0 0 18 0 1 0 400838449 328810496 5937 33554432000 4194304 5683644 140737437777424 18446744073709551615 268660682310 0 0 16781312 514 4493123209998106623 0 0 17 6 0 0 0
/proc/31286/statm: 80276 5937 347 364 0 16182 0
[pid=31288] ppid=31286 vsize=419160 CPUtime=1.05 cores=0-7
/proc/31288/stat : 31288 (adaptg2wsat2009) R 31286 31288 31288 0 -1 4202496 1258 0 0 0 105 0 0 0 25 0 1 0 400838449 429219840 991 33554432000 134512640 135107411 4294508160 18446744073709551615 134523257 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/31288/statm: 104790 991 49 146 0 103666 0
[pid=31289] ppid=31268 vsize=331360 CPUtime=0 cores=0-7
/proc/31289/stat : 31289 (python) S 31268 31268 30681 0 -1 4202560 763 0 0 0 0 0 0 0 19 0 1 0 400838452 339312640 5941 33554432000 4194304 5683644 140737437777424 18446744073709551615 268660682310 0 0 16781312 514 17518070993539563519 0 0 17 1 0 0 0
/proc/31289/statm: 82840 5941 347 364 0 18743 0
[pid=31291] ppid=31289 vsize=1018820 CPUtime=1 cores=0-7
/proc/31291/stat : 31291 (gnovelty+2) R 31289 31291 31291 0 -1 4202496 860 0 0 0 100 0 0 0 23 0 1 0 400838453 1043271680 596 33554432000 134512640 135663704 4292175936 18446744073709551615 134525506 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/31291/statm: 254705 596 113 282 0 254421 0
[pid=31268/tid=31275] ppid=31266 vsize=362116 CPUtime=0 cores=0-7
/proc/31268/task/31275/stat : 31275 (python) S 31266 31268 30681 0 -1 4202560 26 935 0 0 0 0 0 0 19 0 9 0 400838436 370806784 7496 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 6 0 0 0
[pid=31268/tid=31278] ppid=31266 vsize=362116 CPUtime=0 cores=0-7
/proc/31268/task/31278/stat : 31278 (python) S 31266 31268 30681 0 -1 4202560 22 935 0 0 0 0 0 0 18 0 9 0 400838440 370806784 7496 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 6 0 0 0
[pid=31268/tid=31281] ppid=31266 vsize=362116 CPUtime=0 cores=0-7
/proc/31268/task/31281/stat : 31281 (python) S 31266 31268 30681 0 -1 4202560 24 935 0 0 0 0 0 0 23 0 9 0 400838443 370806784 7496 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 2 0 0 0
[pid=31268/tid=31284] ppid=31266 vsize=362116 CPUtime=0 cores=0-7
/proc/31268/task/31284/stat : 31284 (python) S 31266 31268 30681 0 -1 4202560 28 935 0 0 0 0 0 0 20 0 9 0 400838446 370806784 7496 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 2 0 0 0
[pid=31268/tid=31287] ppid=31266 vsize=362116 CPUtime=0 cores=0-7
/proc/31268/task/31287/stat : 31287 (python) S 31266 31268 30681 0 -1 4202560 23 935 0 0 0 0 0 0 22 0 9 0 400838449 370806784 7496 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 6 0 0 0
[pid=31268/tid=31290] ppid=31266 vsize=362116 CPUtime=0 cores=0-7
/proc/31268/task/31290/stat : 31290 (python) S 31266 31268 30681 0 -1 4202560 23 935 0 0 0 0 0 0 19 0 9 0 400838452 370806784 7496 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 1 0 0 0
[pid=31268/tid=31294] ppid=31266 vsize=362116 CPUtime=0 cores=0-7
/proc/31268/task/31294/stat : 31294 (python) S 31266 31268 30681 0 -1 4202560 25 935 0 0 0 0 0 0 21 0 9 0 400838456 370806784 7496 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 8.44
Current children cumulated vsize (KiB) 6510704

[startup+6.30157 s]
/proc/loadavg: 8.66 8.60 6.79 10/428 31300
/proc/meminfo: memFree=23762064/32951124 swapFree=46575272/67111528
[pid=31268] ppid=31266 vsize=362116 CPUtime=1.87 cores=0-7
/proc/31268/stat : 31268 (python) S 31266 31268 30681 0 -1 4202496 18182 935 1 0 172 15 0 0 23 0 9 0 400838245 370806784 7496 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672488907 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/31268/statm: 90529 7496 1891 364 0 26426 0
[pid=31274] ppid=31268 vsize=279924 CPUtime=0.01 cores=0-7
/proc/31274/stat : 31274 (python) S 31268 31268 30681 0 -1 4202560 902 0 0 0 0 1 0 0 18 0 1 0 400838436 286642176 5792 33554432000 4194304 5683644 140737437777424 18446744073709551615 268660682255 0 0 16781312 514 13496785885588324611 0 0 17 6 0 0 0
/proc/31274/statm: 69981 5792 378 364 0 5899 0
[pid=31276] ppid=31274 vsize=1018820 CPUtime=4.21 cores=0-7
/proc/31276/stat : 31276 (gnovelty+2) R 31274 31276 31276 0 -1 4202496 858 0 0 0 421 0 0 0 25 0 1 0 400838437 1043271680 596 33554432000 134512640 135663704 4291676112 18446744073709551615 134525684 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/31276/statm: 254705 596 113 282 0 254421 0
[pid=31277] ppid=31268 vsize=290820 CPUtime=0.02 cores=0-7
/proc/31277/stat : 31277 (python) S 31268 31268 30681 0 -1 4202560 906 0 0 0 1 1 0 0 18 0 1 0 400838439 297799680 5815 33554432000 4194304 5683644 140737437777424 18446744073709551615 268660682310 0 0 16781312 514 4279479579506082050 0 0 17 0 0 0 0
/proc/31277/statm: 72705 5815 372 364 0 8620 0
[pid=31279] ppid=31277 vsize=1018820 CPUtime=4.34 cores=0-7
/proc/31279/stat : 31279 (gnovelty+2) R 31277 31279 31279 0 -1 4202496 858 0 0 0 434 0 0 0 25 0 1 0 400838440 1043271680 596 33554432000 134512640 135663704 4292375808 18446744073709551615 134524918 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/31279/statm: 254705 596 113 282 0 254421 0
[pid=31280] ppid=31268 vsize=300592 CPUtime=0.01 cores=0-7
/proc/31280/stat : 31280 (python) S 31268 31268 30681 0 -1 4202560 908 0 0 0 0 1 0 0 18 0 1 0 400838443 307806208 5952 33554432000 4194304 5683644 140737437777424 18446744073709551615 268660682310 0 0 16781312 514 2414095980577128707 0 0 17 2 0 0 0
/proc/31280/statm: 75148 5952 370 364 0 11060 0
[pid=31282] ppid=31280 vsize=419428 CPUtime=4.26 cores=0-7
/proc/31282/stat : 31282 (adaptg2wsat2009) R 31280 31282 31282 0 -1 4202496 1445 0 0 0 426 0 0 0 25 0 1 0 400838443 429494272 1177 33554432000 134512640 135107411 4287004944 18446744073709551615 134532371 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/31282/statm: 104857 1177 59 146 0 103733 0
[pid=31283] ppid=31268 vsize=310848 CPUtime=0.01 cores=0-7
/proc/31283/stat : 31283 (python) S 31268 31268 30681 0 -1 4202560 916 0 0 0 1 0 0 0 18 0 1 0 400838446 318308352 5955 33554432000 4194304 5683644 140737437777424 18446744073709551615 268660682310 0 0 16781312 514 6348276128108478723 0 0 17 6 0 0 0
/proc/31283/statm: 77712 5955 370 364 0 13621 0
[pid=31285] ppid=31283 vsize=419428 CPUtime=4.27 cores=0-7
/proc/31285/stat : 31285 (adaptg2wsat2009) R 31283 31285 31285 0 -1 4202496 1439 0 0 0 427 0 0 0 25 0 1 0 400838446 429494272 1175 33554432000 134512640 135107411 4287351936 18446744073709551615 134531797 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/31285/statm: 104857 1175 59 146 0 103733 0
[pid=31286] ppid=31268 vsize=321104 CPUtime=0.01 cores=0-7
/proc/31286/stat : 31286 (python) S 31268 31268 30681 0 -1 4202560 937 0 0 0 1 0 0 0 18 0 1 0 400838449 328810496 5960 33554432000 4194304 5683644 140737437777424 18446744073709551615 268660682310 0 0 16781312 514 4493123209998074115 0 0 17 6 0 0 0
/proc/31286/statm: 80276 5960 370 364 0 16182 0
[pid=31288] ppid=31286 vsize=419428 CPUtime=4.19 cores=0-7
/proc/31288/stat : 31288 (adaptg2wsat2009) R 31286 31288 31288 0 -1 4202496 1443 0 0 0 419 0 0 0 25 0 1 0 400838449 429494272 1176 33554432000 134512640 135107411 4294508160 18446744073709551615 134531781 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/31288/statm: 104857 1176 59 146 0 103733 0
[pid=31289] ppid=31268 vsize=331360 CPUtime=0.01 cores=0-7
/proc/31289/stat : 31289 (python) S 31268 31268 30681 0 -1 4202560 943 0 0 0 1 0 0 0 18 0 1 0 400838452 339312640 5964 33554432000 4194304 5683644 140737437777424 18446744073709551615 268660682310 0 0 16781312 514 17518070993539531009 0 0 17 1 0 0 0
/proc/31289/statm: 82840 5964 370 364 0 18743 0
[pid=31291] ppid=31289 vsize=1018820 CPUtime=4.18 cores=0-7
/proc/31291/stat : 31291 (gnovelty+2) R 31289 31291 31291 0 -1 4202496 860 0 0 0 418 0 0 0 25 0 1 0 400838453 1043271680 596 33554432000 134512640 135663704 4292175936 18446744073709551615 134524594 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/31291/statm: 254705 596 113 282 0 254421 0
[pid=31292] ppid=31268 vsize=341616 CPUtime=0.01 cores=0-7
/proc/31292/stat : 31292 (python) S 31268 31268 30681 0 -1 4202560 945 0 0 0 0 1 0 0 18 0 1 0 400838455 349814784 5968 33554432000 4194304 5683644 140737437777424 18446744073709551615 268660682310 0 0 16781312 514 4501598520502878467 0 0 17 0 0 0 0
/proc/31292/statm: 85404 5968 370 364 0 21304 0
[pid=31295] ppid=31292 vsize=247428 CPUtime=2.18 cores=0-7
/proc/31295/stat : 31295 (FH) R 31292 31295 31295 0 -1 4202496 1096 0 0 0 218 0 0 0 25 0 1 0 400838458 253366272 833 33554432000 134512640 134936988 4288385968 18446744073709551615 134523337 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/31295/statm: 61857 833 37 104 0 61361 0
[pid=31297] ppid=31268 vsize=352052 CPUtime=0.01 cores=0-7
/proc/31297/stat : 31297 (python) S 31268 31268 30681 0 -1 4202560 992 0 0 0 1 0 0 0 18 0 1 0 400838459 360501248 5980 33554432000 4194304 5683644 140737437777424 18446744073709551615 268660682310 0 0 16781312 514 1351880611849011460 0 0 17 0 0 0 0
/proc/31297/statm: 88013 5980 378 364 0 23910 0
[pid=31300] ppid=31297 vsize=419160 CPUtime=2.08 cores=0-7
/proc/31300/stat : 31300 (adaptg2wsat2009) R 31297 31300 31300 0 -1 4202496 1265 0 0 0 208 0 0 0 25 0 1 0 400838459 429219840 991 33554432000 134512640 135107411 4290269776 18446744073709551615 134523331 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/31300/statm: 104790 991 49 146 0 103666 0
[pid=31268/tid=31275] ppid=31266 vsize=362116 CPUtime=0 cores=0-7
/proc/31268/task/31275/stat : 31275 (python) S 31266 31268 30681 0 -1 4202560 26 935 0 0 0 0 0 0 19 0 9 0 400838436 370806784 7496 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 6 0 0 0
[pid=31268/tid=31278] ppid=31266 vsize=362116 CPUtime=0 cores=0-7
/proc/31268/task/31278/stat : 31278 (python) S 31266 31268 30681 0 -1 4202560 22 935 0 0 0 0 0 0 18 0 9 0 400838440 370806784 7496 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 6 0 0 0
[pid=31268/tid=31281] ppid=31266 vsize=362116 CPUtime=0 cores=0-7
/proc/31268/task/31281/stat : 31281 (python) S 31266 31268 30681 0 -1 4202560 24 935 0 0 0 0 0 0 23 0 9 0 400838443 370806784 7496 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 2 0 0 0
[pid=31268/tid=31284] ppid=31266 vsize=362116 CPUtime=0 cores=0-7
/proc/31268/task/31284/stat : 31284 (python) S 31266 31268 30681 0 -1 4202560 28 935 0 0 0 0 0 0 20 0 9 0 400838446 370806784 7496 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 2 0 0 0
[pid=31268/tid=31287] ppid=31266 vsize=362116 CPUtime=0 cores=0-7
/proc/31268/task/31287/stat : 31287 (python) S 31266 31268 30681 0 -1 4202560 23 935 0 0 0 0 0 0 22 0 9 0 400838449 370806784 7496 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 6 0 0 0
[pid=31268/tid=31290] ppid=31266 vsize=362116 CPUtime=0 cores=0-7
/proc/31268/task/31290/stat : 31290 (python) S 31266 31268 30681 0 -1 4202560 23 935 0 0 0 0 0 0 19 0 9 0 400838452 370806784 7496 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 1 0 0 0
[pid=31268/tid=31294] ppid=31266 vsize=362116 CPUtime=0 cores=0-7
/proc/31268/task/31294/stat : 31294 (python) S 31266 31268 30681 0 -1 4202560 25 935 0 0 0 0 0 0 21 0 9 0 400838456 370806784 7496 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 0 0 0 0

################
# More data... #
################

[pid=31375] ppid=31373 vsize=1018820 CPUtime=101.91 cores=0-7
/proc/31375/stat : 31375 (gnovelty+2) T 31373 31375 31375 0 -1 4202496 882 0 0 0 10191 0 0 0 25 0 1 0 400883671 1043271680 596 33554432000 134512640 135663704 4290191104 18446744073709551615 134525255 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/31375/statm: 254705 596 113 282 0 254421 0
[pid=31373/tid=31381] ppid=31268 vsize=561992 CPUtime=0 cores=0-7
/proc/31373/task/31381/stat : 31381 (python) S 31268 31268 30681 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 400895515 575479808 6333 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 514 18446604460726387712 0 0 -1 1 0 0 0
[pid=31382] ppid=31268 vsize=572280 CPUtime=0.37 cores=0-7
/proc/31382/stat : 31382 (python) S 31268 31268 30681 0 -1 4202560 1259 0 0 0 21 16 0 0 19 0 2 0 400897689 586014720 6348 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672488907 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/31382/statm: 143070 6348 408 364 0 78922 0
[pid=31385] ppid=31382 vsize=335996 CPUtime=102.4 cores=0-7
/proc/31385/stat : 31385 (iPAWS) T 31382 31385 31385 0 -1 4202496 1717 0 0 0 10240 0 0 0 25 0 1 0 400897689 344059904 1429 33554432000 134512640 135114423 4287202368 18446744073709551615 134517052 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/31385/statm: 83999 1429 59 147 0 83851 0
[pid=31382/tid=31395] ppid=31268 vsize=572280 CPUtime=0 cores=0-7
/proc/31382/task/31395/stat : 31395 (python) S 31268 31268 30681 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 400908926 586014720 6348 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 514 18446604459699140416 0 0 -1 0 0 0 0
[pid=31388] ppid=31268 vsize=582956 CPUtime=0.7 cores=0-7
/proc/31388/stat : 31388 (python) S 31268 31268 30681 0 -1 4202560 1324 0 0 0 43 27 0 0 19 0 2 0 400900379 596946944 6403 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672488907 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/31388/statm: 145739 6403 408 364 0 81588 0
[pid=31390] ppid=31388 vsize=268428 CPUtime=194.14 cores=0-7
/proc/31390/stat : 31390 (hybridGM3) T 31388 31390 31390 0 -1 4202496 5301 0 0 0 19414 0 0 0 25 0 1 0 400900380 274870272 5010 33554432000 134512640 135204385 4290783120 18446744073709551615 134538375 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/31390/statm: 67107 5010 63 169 0 66937 0
[pid=31388/tid=31400] ppid=31268 vsize=582956 CPUtime=0 cores=0-7
/proc/31388/task/31400/stat : 31400 (python) S 31268 31268 30681 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 400921258 596946944 6403 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 514 18446604459699138624 0 0 -1 3 0 0 0
[pid=31401] ppid=31268 vsize=593004 CPUtime=1.54 cores=0-7
/proc/31401/stat : 31401 (python) S 31268 31268 30681 0 -1 4202560 1309 0 0 0 77 77 0 0 18 0 2 0 400921263 607236096 6412 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672488907 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/31401/statm: 148251 6412 402 364 0 84097 0
[pid=31404] ppid=31401 vsize=268424 CPUtime=393.17 cores=0-7
/proc/31404/stat : 31404 (hybridGM3) T 31401 31404 31404 0 -1 4202496 5304 0 0 0 39316 1 0 0 25 0 1 0 400921264 274866176 5010 33554432000 134512640 135204385 4287604064 18446744073709551615 134538041 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/31404/statm: 67106 5010 63 169 0 66936 0
[pid=31401/tid=31419] ppid=31268 vsize=593004 CPUtime=0 cores=0-7
/proc/31401/task/31419/stat : 31419 (python) S 31268 31268 30681 0 -1 4202560 48 0 0 0 0 0 0 0 18 0 2 0 400942345 607236096 6412 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 514 18446604459699139520 0 0 -1 7 0 0 0
[pid=31406] ppid=31268 vsize=603184 CPUtime=2.15 cores=0-7
/proc/31406/stat : 31406 (python) S 31268 31268 30681 0 -1 4202560 1114 0 0 0 119 96 0 0 18 0 1 0 400923151 617660416 6387 33554432000 4194304 5683644 140737437777424 18446744073709551615 268660682310 0 0 16781312 514 8003125236255916292 0 0 17 0 0 0 0
/proc/31406/statm: 150796 6387 374 364 0 86639 0
[pid=31408] ppid=31406 vsize=457664 CPUtime=638.37 cores=0-7
/proc/31408/stat : 31408 (TNM) R 31406 31408 31408 0 -1 4202496 1321 0 0 0 63837 0 0 0 25 0 1 0 400923152 468647936 1036 33554432000 134512640 135008748 4289928272 18446744073709551615 134527773 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/31408/statm: 114416 1036 46 122 0 113316 0
[pid=31411] ppid=31268 vsize=613648 CPUtime=0.41 cores=0-7
/proc/31411/stat : 31411 (python) S 31268 31268 30681 0 -1 4202560 1259 0 0 0 23 18 0 0 19 0 2 0 400926325 628375552 6420 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672488907 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/31411/statm: 153412 6420 402 364 0 89252 0
[pid=31413] ppid=31411 vsize=335996 CPUtime=97.34 cores=0-7
/proc/31413/stat : 31413 (iPAWS) T 31411 31413 31413 0 -1 4202496 1714 0 0 0 9734 0 0 0 25 0 1 0 400926325 344059904 1429 33554432000 134512640 135114423 4289902176 18446744073709551615 134517436 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/31413/statm: 83999 1429 59 147 0 83851 0
[pid=31411/tid=31417] ppid=31268 vsize=613648 CPUtime=0 cores=0-7
/proc/31411/task/31417/stat : 31417 (python) S 31268 31268 30681 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 400938766 628375552 6420 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 514 18446604455859205440 0 0 -1 1 0 0 0
[pid=31420] ppid=31268 vsize=623620 CPUtime=0.36 cores=0-7
/proc/31420/stat : 31420 (python) S 31268 31268 30681 0 -1 4202560 1227 0 0 0 18 18 0 0 19 0 2 0 400942351 638586880 6441 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672488907 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/31420/statm: 155905 6441 402 364 0 91742 0
[pid=31422] ppid=31420 vsize=1018820 CPUtime=98.28 cores=0-7
/proc/31422/stat : 31422 (gnovelty+2) T 31420 31422 31422 0 -1 4202496 881 0 0 0 9828 0 0 0 25 0 1 0 400942351 1043271680 596 33554432000 134512640 135663704 4287489264 18446744073709551615 134525722 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/31422/statm: 254705 596 113 282 0 254421 0
[pid=31420/tid=31430] ppid=31268 vsize=623620 CPUtime=0 cores=0-7
/proc/31420/task/31430/stat : 31430 (python) S 31268 31268 30681 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 400952893 638586880 6441 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 514 18446604455859206336 0 0 -1 0 0 0 0
[pid=31425] ppid=31268 vsize=633708 CPUtime=1.2 cores=0-7
/proc/31425/stat : 31425 (python) S 31268 31268 30681 0 -1 4202560 1093 0 0 0 66 54 0 0 18 0 1 0 400951760 648916992 6716 33554432000 4194304 5683644 140737437777424 18446744073709551615 268660682310 0 0 16781312 514 8003125236255916293 0 0 17 2 0 0 0
/proc/31425/statm: 158427 6716 379 364 0 94261 0
[pid=31427] ppid=31425 vsize=419068 CPUtime=323.41 cores=0-7
/proc/31427/stat : 31427 (NCVWr) R 31425 31427 31427 0 -1 4202496 1359 0 0 0 32341 0 0 0 25 0 1 0 400951760 429125632 1077 33554432000 134512640 135113558 4292947024 18446744073709551615 134531745 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/31427/statm: 104767 1077 58 147 0 103642 0
[pid=31435] ppid=31268 vsize=643964 CPUtime=0.36 cores=0-7
/proc/31435/stat : 31435 (python) S 31268 31268 30681 0 -1 4202560 1267 0 0 0 19 17 0 0 19 0 2 0 400969133 659419136 6677 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672488907 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/31435/statm: 160991 6677 407 364 0 96822 0
[pid=31437] ppid=31435 vsize=335996 CPUtime=100.23 cores=0-7
/proc/31437/stat : 31437 (iPAWS) T 31435 31437 31437 0 -1 4202496 1712 0 0 0 10022 1 0 0 25 0 1 0 400969133 344059904 1427 33554432000 134512640 135114423 4292761296 18446744073709551615 134514777 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/31437/statm: 83999 1427 59 147 0 83851 0
[pid=31435/tid=31443] ppid=31268 vsize=643964 CPUtime=0 cores=0-7
/proc/31435/task/31443/stat : 31443 (python) S 31268 31268 30681 0 -1 4202560 120 0 0 0 0 0 0 0 19 0 2 0 400979768 659419136 6677 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 514 18446604471162312000 0 0 -1 0 0 0 0
[pid=31444] ppid=31268 vsize=654220 CPUtime=0.26 cores=0-7
/proc/31444/stat : 31444 (python) S 31268 31268 30681 0 -1 4202560 1033 0 0 0 15 11 0 0 18 0 1 0 400979773 669921280 6665 33554432000 4194304 5683644 140737437777424 18446744073709551615 268660682310 0 0 16781312 514 8003125236255916292 0 0 17 0 0 0 0
/proc/31444/statm: 163555 6665 379 364 0 99383 0
[pid=31446] ppid=31444 vsize=249192 CPUtime=56.55 cores=0-7
/proc/31446/stat : 31446 (FH) R 31444 31446 31446 0 -1 4202496 1344 0 0 0 5655 0 0 0 25 0 1 0 400979778 255172608 1059 33554432000 134512640 134936988 4294275376 18446744073709551615 134553505 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/31446/statm: 62298 1059 41 104 0 61802 0
[pid=31448] ppid=31268 vsize=664476 CPUtime=0.12 cores=0-7
/proc/31448/stat : 31448 (python) S 31268 31268 30681 0 -1 4202560 1050 0 0 0 6 6 0 0 18 0 1 0 400983993 680423424 6731 33554432000 4194304 5683644 140737437777424 18446744073709551615 268660682310 0 0 16781312 514 8003125236255916289 0 0 17 1 0 0 0
/proc/31448/statm: 166119 6731 379 364 0 101944 0
[pid=31450] ppid=31448 vsize=268424 CPUtime=37.11 cores=0-7
/proc/31450/stat : 31450 (hybridGM3) R 31448 31450 31450 0 -1 4202496 5296 0 0 0 3711 0 0 0 25 0 1 0 400983993 274866176 5009 33554432000 134512640 135204385 4286631776 18446744073709551615 134538034 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/31450/statm: 67106 5009 63 169 0 66936 0
[pid=31452] ppid=31268 vsize=674732 CPUtime=0 cores=0-7
/proc/31452/stat : 31452 (python) S 31268 31268 30681 0 -1 4202560 949 0 0 0 0 0 0 0 19 0 1 0 400987919 690925568 6759 33554432000 4194304 5683644 140737437777424 18446744073709551615 268660682310 0 0 16781312 514 14256762275393175551 0 0 17 4 0 0 0
/proc/31452/statm: 168683 6759 352 364 0 104505 0
[pid=31454] ppid=31452 vsize=414596 CPUtime=0.3 cores=0-7
/proc/31454/stat : 31454 (NCVWr) R 31452 31454 31454 0 -1 4202496 1143 0 0 0 30 0 0 0 21 0 1 0 400987919 424546304 854 33554432000 134512640 135113558 4289610960 18446744073709551615 134526456 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/31454/statm: 103649 854 55 147 0 102524 0
[pid=31268/tid=31275] ppid=31266 vsize=684976 CPUtime=0 cores=0-7
/proc/31268/task/31275/stat : 31275 (python) S 31266 31268 30681 0 -1 4202560 26 935 0 0 0 0 0 0 19 0 34 0 400838436 701415424 8353 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 6 0 0 0
[pid=31268/tid=31278] ppid=31266 vsize=684976 CPUtime=0 cores=0-7
/proc/31268/task/31278/stat : 31278 (python) S 31266 31268 30681 0 -1 4202560 22 935 0 0 0 0 0 0 18 0 34 0 400838440 701415424 8353 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 6 0 0 0
[pid=31268/tid=31281] ppid=31266 vsize=684976 CPUtime=0 cores=0-7
/proc/31268/task/31281/stat : 31281 (python) S 31266 31268 30681 0 -1 4202560 28 935 0 0 0 0 0 0 18 0 34 0 400838443 701415424 8353 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 2 0 0 0
[pid=31268/tid=31284] ppid=31266 vsize=684976 CPUtime=0 cores=0-7
/proc/31268/task/31284/stat : 31284 (python) S 31266 31268 30681 0 -1 4202560 33 935 0 0 0 0 0 0 18 0 34 0 400838446 701415424 8353 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 2 0 0 0
[pid=31268/tid=31287] ppid=31266 vsize=684976 CPUtime=0 cores=0-7
/proc/31268/task/31287/stat : 31287 (python) S 31266 31268 30681 0 -1 4202560 28 935 0 0 0 0 0 0 18 0 34 0 400838449 701415424 8353 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 6 0 0 0
[pid=31268/tid=31290] ppid=31266 vsize=684976 CPUtime=0 cores=0-7
/proc/31268/task/31290/stat : 31290 (python) S 31266 31268 30681 0 -1 4202560 23 935 0 0 0 0 0 0 19 0 34 0 400838452 701415424 8353 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 1 0 0 0
[pid=31268/tid=31294] ppid=31266 vsize=684976 CPUtime=0 cores=0-7
/proc/31268/task/31294/stat : 31294 (python) S 31266 31268 30681 0 -1 4202560 28 935 0 0 0 0 0 0 18 0 34 0 400838456 701415424 8353 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 0 0 0 0
[pid=31268/tid=31298] ppid=31266 vsize=684976 CPUtime=0 cores=0-7
/proc/31268/task/31298/stat : 31298 (python) S 31266 31268 30681 0 -1 4202560 31 935 0 0 0 0 0 0 18 0 34 0 400838459 701415424 8353 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 0 0 0 0
[pid=31268/tid=31308] ppid=31266 vsize=684976 CPUtime=0 cores=0-7
/proc/31268/task/31308/stat : 31308 (python) S 31266 31268 30681 0 -1 4202560 48 935 0 0 0 0 0 0 18 0 34 0 400848476 701415424 8353 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 1 0 0 0
[pid=31268/tid=31313] ppid=31266 vsize=684976 CPUtime=0 cores=0-7
/proc/31268/task/31313/stat : 31313 (python) S 31266 31268 30681 0 -1 4202560 138 935 0 0 0 0 0 0 18 0 34 0 400848479 701415424 8353 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 0 0 0 0
[pid=31268/tid=31316] ppid=31266 vsize=684976 CPUtime=0 cores=0-7
/proc/31268/task/31316/stat : 31316 (python) S 31266 31268 30681 0 -1 4202560 27 935 0 0 0 0 0 0 18 0 34 0 400848483 701415424 8353 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 0 0 0 0
[pid=31268/tid=31320] ppid=31266 vsize=684976 CPUtime=0 cores=0-7
/proc/31268/task/31320/stat : 31320 (python) S 31266 31268 30681 0 -1 4202560 34 935 0 0 0 0 0 0 25 0 34 0 400848514 701415424 8353 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 0 0 0 0
[pid=31268/tid=31324] ppid=31266 vsize=684976 CPUtime=0 cores=0-7
/proc/31268/task/31324/stat : 31324 (python) S 31266 31268 30681 0 -1 4202560 33 935 0 0 0 0 0 0 18 0 34 0 400849078 701415424 8353 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 0 0 0 0
[pid=31268/tid=31328] ppid=31266 vsize=684976 CPUtime=0 cores=0-7
/proc/31268/task/31328/stat : 31328 (python) S 31266 31268 30681 0 -1 4202560 36 935 0 0 0 0 0 0 18 0 34 0 400850305 701415424 8353 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 0 0 0 0
[pid=31268/tid=31333] ppid=31266 vsize=684976 CPUtime=0 cores=0-7
/proc/31268/task/31333/stat : 31333 (python) S 31266 31268 30681 0 -1 4202560 22 935 0 0 0 0 0 0 20 0 34 0 400852106 701415424 8353 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 7 0 0 0
[pid=31268/tid=31337] ppid=31266 vsize=684976 CPUtime=0 cores=0-7
/proc/31268/task/31337/stat : 31337 (python) S 31266 31268 30681 0 -1 4202560 46 935 0 0 0 0 0 0 18 0 34 0 400853288 701415424 8353 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 7 0 0 0
[pid=31268/tid=31343] ppid=31266 vsize=684976 CPUtime=0 cores=0-7
/proc/31268/task/31343/stat : 31343 (python) S 31266 31268 30681 0 -1 4202560 32 935 0 0 0 0 0 0 18 0 34 0 400858515 701415424 8353 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 7 0 0 0
[pid=31268/tid=31347] ppid=31266 vsize=684976 CPUtime=0 cores=0-7
/proc/31268/task/31347/stat : 31347 (python) S 31266 31268 30681 0 -1 4202560 33 935 0 0 0 0 0 0 18 0 34 0 400859781 701415424 8353 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 7 0 0 0
[pid=31268/tid=31363] ppid=31266 vsize=684976 CPUtime=0 cores=0-7
/proc/31268/task/31363/stat : 31363 (python) S 31266 31268 30681 0 -1 4202560 38 935 0 0 0 0 0 0 18 0 34 0 400878476 701415424 8353 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 7 0 0 0
[pid=31268/tid=31366] ppid=31266 vsize=684976 CPUtime=0 cores=0-7
/proc/31268/task/31366/stat : 31366 (python) S 31266 31268 30681 0 -1 4202560 33 935 0 0 0 0 0 0 18 0 34 0 400880092 701415424 8353 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 4 0 0 0
[pid=31268/tid=31369] ppid=31266 vsize=684976 CPUtime=0 cores=0-7
/proc/31268/task/31369/stat : 31369 (python) S 31266 31268 30681 0 -1 4202560 25 935 0 0 0 0 0 0 20 0 34 0 400880098 701415424 8353 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 7 0 0 0
[pid=31268/tid=31374] ppid=31266 vsize=684976 CPUtime=0 cores=0-7
/proc/31268/task/31374/stat : 31374 (python) S 31266 31268 30681 0 -1 4202560 33 935 0 0 0 0 0 0 19 0 34 0 400883671 701415424 8353 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 7 0 0 0
[pid=31268/tid=31384] ppid=31266 vsize=684976 CPUtime=0 cores=0-7
/proc/31268/task/31384/stat : 31384 (python) S 31266 31268 30681 0 -1 4202560 32 935 0 0 0 0 0 0 19 0 34 0 400897689 701415424 8353 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 7 0 0 0
[pid=31268/tid=31389] ppid=31266 vsize=684976 CPUtime=0 cores=0-7
/proc/31268/task/31389/stat : 31389 (python) S 31266 31268 30681 0 -1 4202560 29 935 0 0 0 0 0 0 19 0 34 0 400900380 701415424 8353 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 7 0 0 0
[pid=31268/tid=31402] ppid=31266 vsize=684976 CPUtime=0 cores=0-7
/proc/31268/task/31402/stat : 31402 (python) S 31266 31268 30681 0 -1 4202560 38 935 0 0 0 0 0 0 18 0 34 0 400921264 701415424 8353 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 7 0 0 0
[pid=31268/tid=31407] ppid=31266 vsize=684976 CPUtime=0 cores=0-7
/proc/31268/task/31407/stat : 31407 (python) S 31266 31268 30681 0 -1 4202560 30 935 0 0 0 0 0 0 19 0 34 0 400923151 701415424 8353 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 7 0 0 0
[pid=31268/tid=31412] ppid=31266 vsize=684976 CPUtime=0 cores=0-7
/proc/31268/task/31412/stat : 31412 (python) S 31266 31268 30681 0 -1 4202560 26 935 0 0 0 0 0 0 19 0 34 0 400926325 701415424 8353 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 7 0 0 0
[pid=31268/tid=31421] ppid=31266 vsize=684976 CPUtime=0 cores=0-7
/proc/31268/task/31421/stat : 31421 (python) S 31266 31268 30681 0 -1 4202560 29 935 0 0 0 0 0 0 20 0 34 0 400942351 701415424 8353 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 0 0 0 0
[pid=31268/tid=31426] ppid=31266 vsize=684976 CPUtime=0 cores=0-7
/proc/31268/task/31426/stat : 31426 (python) S 31266 31268 30681 0 -1 4202560 39 935 0 0 0 0 0 0 20 0 34 0 400951760 701415424 8353 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 7 0 0 0
[pid=31268/tid=31436] ppid=31266 vsize=684976 CPUtime=0 cores=0-7
/proc/31268/task/31436/stat : 31436 (python) S 31266 31268 30681 0 -1 4202560 45 935 0 0 0 0 0 0 19 0 34 0 400969133 701415424 8353 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 7 0 0 0
[pid=31268/tid=31445] ppid=31266 vsize=684976 CPUtime=0 cores=0-7
/proc/31268/task/31445/stat : 31445 (python) S 31266 31268 30681 0 -1 4202560 35 935 0 0 0 0 0 0 18 0 34 0 400979773 701415424 8353 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 2 0 0 0
[pid=31268/tid=31449] ppid=31266 vsize=684976 CPUtime=0 cores=0-7
/proc/31268/task/31449/stat : 31449 (python) S 31266 31268 30681 0 -1 4202560 33 935 0 0 0 0 0 0 19 0 34 0 400983993 701415424 8353 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 6 0 0 0
[pid=31268/tid=31453] ppid=31266 vsize=684976 CPUtime=0 cores=0-7
/proc/31268/task/31453/stat : 31453 (python) S 31266 31268 30681 0 -1 4202560 36 935 0 0 0 0 0 0 20 0 34 0 400987919 701415424 8353 33554432000 4194304 5683644 140737437777424 18446744073709551615 268672485809 0 0 16781312 2 18446604447642171584 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 10487.3
Current children cumulated vsize (KiB) 32201756

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 31268 and gives
#  childrusage.ru_utime.tv_sec=2867
#  childrusage.ru_utime.tv_usec=843021
#  childrusage.ru_stime.tv_sec=17
#  childrusage.ru_stime.tv_usec=748301
# CPU time returned by wait4() is 2885.59
# while last known CPU time is 10487.3
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1497.22
CPU time (s): 10487.3
CPU user time (s): 10468.4
CPU system time (s): 18.89
CPU usage (%): 700.454
Max. virtual memory (cumulated for all children) (KiB): 32201756

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2867.84
system time used= 17.7483
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 93010
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 17189
involuntary context switches= 22566

runsolver used 17.3764 second user time and 39.438 second system time

The end

Launcher Data 

Begin job on node107 at 2011-05-16 19:21:10
IDJOB=3385424
IDBENCH=82689
IDSOLVER=1870
FILE ID=node107/3385424-1305566469
RUNJOBID= node107-1305564966-30695
PBS_JOBID= 13350143
Free space on /tmp= 73264 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/large/unif-k7-r85-v250-c21250-S58287803-001-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3385424-1305566469/watcher-3385424-1305566469 -o /tmp/evaluation-result-3385424-1305566469/solver-3385424-1305566469 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385424-1305566469.cnf -seed 1823898591 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 0d5d355ade4a956f83626786ca6018b7
RANDOM SEED=1823898591

node107.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5359.47
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.829
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      23845692 kB
Buffers:        229956 kB
Cached:        2414592 kB
SwapCached:       9812 kB
Active:        7378644 kB
Inactive:      1420408 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      23845692 kB
SwapTotal:    67111528 kB
SwapFree:     46575272 kB
Dirty:           32516 kB
Writeback:           0 kB
AnonPages:     6151860 kB
Mapped:          26824 kB
Slab:           153120 kB
PageTables:      81976 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 111504276 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73232 MiB
End job on node107 at 2011-05-16 19:46:09