Trace number 3385404

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)? (MO) 9381.69 1332.22

General information on the benchmark

Namerandom/large/
unif-k7-r85-v400-c34000-S586055573-067-UNKNOWN.cnf
MD5SUMd673a60901b4a946b283fdbb02f99fdc
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables400
Number of clauses34000
Sum of the clauses size238000
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 534000

Solver Data 

0.36/0.68	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.36/0.68	c INFO: solving HOME/instance-3385404-1305566268.cnf
0.36/0.68	c INFO: computing task features
0.64/0.91	c INFO: computed variable-clause graph statistics
0.64/0.92	c INFO: computed clause balance statistics
1.86/2.13	c INFO: computed variable balance statistics
2.35/2.62	c INFO: computed small-clause counts
2.35/2.62	c INFO: computed Horn-clause counts
2.35/2.66	c INFO: computed variable graph statistics
2.35/2.66	c INFO: collected features for HOME/instance-3385404-1305566268.cnf in 1.98 s
2.35/2.69	c INFO: running gnovelty+2@0 for 97 with 4998 remaining (b = 0.38)
2.44/2.72	c INFO: running gnovelty+2@0 for 97 with 4997 remaining (b = 0.13)
2.44/2.75	c INFO: running adaptg2wsat2009++@0 for 97 with 4997 remaining (b = 0.08)
2.44/2.79	c INFO: running adaptg2wsat2009++@0 for 97 with 4997 remaining (b = 0.07)
2.53/2.82	c INFO: running adaptg2wsat2009++@0 for 97 with 4997 remaining (b = 0.05)
2.53/2.85	c INFO: running gnovelty+2@0 for 97 with 4997 remaining (b = 0.04)
2.53/2.88	c INFO: running FH@0 for 97 with 4997 remaining (b = 0.03)
2.64/2.95	c INFO: running adaptg2wsat2009++@0 for 97 with 4997 remaining (b = 0.03)
712.33/103.11	c INFO: running adaptg2wsat2009++@0 for 97 with 4897 remaining (b = 0.02)
712.33/103.16	c INFO: running FH@0 for 97 with 4897 remaining (b = 0.02)
712.33/103.20	c INFO: running hybridGM3@0 for 194 with 4897 remaining (b = 0.04)
713.01/103.26	c INFO: running gnovelty+2@0 for 97 with 4897 remaining (b = 0.02)
838.99/121.20	c INFO: running hybridGM3@0 for 194 with 4879 remaining (b = 0.03)
839.64/121.35	c INFO: running adaptg2wsat2009++@0 for 97 with 4879 remaining (b = 0.01)
883.84/127.65	c INFO: running hybridGM3@0 for 194 with 4873 remaining (b = 0.03)
922.91/133.22	c INFO: running hybridGM3@0 for 194 with 4867 remaining (b = 0.02)
1454.54/208.53	c INFO: running TNM@0 for 1552 with 4792 remaining (b = 0.18)
1465.80/210.18	c INFO: running FH@0 for 97 with 4790 remaining (b = 0.01)
1546.55/221.76	c INFO: running adaptg2wsat2009++@0 for 97 with 4778 remaining (b = 0.01)
1629.92/233.70	c INFO: running adaptg2wsat2009++@0 for 97 with 4767 remaining (b = 0.01)
2149.10/306.09	c INFO: running hybridGM3@100 for 97 with 4694 remaining (b = 0.01)
2222.23/316.60	c INFO: running adaptg2wsat2009++@0 for 97 with 4684 remaining (b = 0.01)
2316.95/330.10	c INFO: running hybridGM3@98 for 97 with 4670 remaining (b = 0.01)
2442.34/348.14	c INFO: running hybridGM3@98 for 97 with 4652 remaining (b = 0.01)
2533.61/361.29	c INFO: running adaptg2wsat2009++@99 for 97 with 4639 remaining (b = 0.01)
2575.46/367.22	c INFO: running hybridGM3@0 for 194 with 4633 remaining (b = 0.02)
2654.93/378.64	c INFO: running hybridGM3@98 for 97 with 4622 remaining (b = 0.01)
2859.54/407.48	c INFO: running hybridGM3@0 for 194 with 4593 remaining (b = 0.02)
2989.72/426.03	c INFO: running TNM@0 for 1455 with 4574 remaining (b = 0.14)
3070.19/437.52	c INFO: running hybridGM3@295 for 97 with 4563 remaining (b = 0.01)
3154.16/449.54	c INFO: running adaptg2wsat2009++@99 for 97 with 4551 remaining (b = 0.01)
3325.77/474.02	c INFO: running hybridGM3@191 for 97 with 4526 remaining (b = 0.01)
3344.56/476.75	c INFO: running iPAWS@0 for 97 with 4523 remaining (b = 0.01)
3874.25/549.96	c INFO: running hybridGM3@390 for 97 with 4450 remaining (b = 0.01)
3937.03/558.97	c INFO: running FH@0 for 291 with 4441 remaining (b = 0.03)
4136.24/587.47	c INFO: running hybridGM3@292 for 97 with 4413 remaining (b = 0.01)
4274.69/607.29	c INFO: running hybridGM3@0 for 194 with 4393 remaining (b = 0.02)
4338.28/616.34	c INFO: running adaptg2wsat2009++@98 for 97 with 4384 remaining (b = 0.01)
4380.94/622.47	c INFO: running gnovelty+2@0 for 97 with 4378 remaining (b = 0.01)
4590.29/651.36	c INFO: running hybridGM3@97 for 97 with 4349 remaining (b = 0.01)
4893.73/694.90	c INFO: running hybridGM3@390 for 97 with 4305 remaining (b = 0.01)
5150.52/731.81	c INFO: running TNM@0 for 1552 with 4268 remaining (b = 0.11)
5256.23/747.10	c INFO: running adaptg2wsat2009++@297 for 291 with 4253 remaining (b = 0.02)
5268.11/748.77	c INFO: running FH@98 for 291 with 4251 remaining (b = 0.02)
5709.67/810.27	c INFO: running iPAWS@0 for 97 with 4190 remaining (b = 0.01)
5731.92/813.43	c INFO: running gnovelty+2@0 for 97 with 4187 remaining (b = 0.01)
6275.60/891.20	c INFO: running hybridGM3@0 for 194 with 4109 remaining (b = 0.01)
6654.15/944.04	c INFO: running FH@0 for 97 with 4056 remaining (b = 0.01)
6757.17/958.86	c INFO: running hybridGM3@0 for 194 with 4041 remaining (b = 0.01)
7403.98/1050.50	c INFO: running NCVWr@0 for 485 with 3950 remaining (b = 0.03)
7406.36/1050.90	c INFO: running FH@293 for 97 with 3949 remaining (b = 0.01)
7563.15/1073.31	c INFO: running adaptg2wsat2009++@393 for 194 with 3927 remaining (b = 0.01)
7960.51/1130.29	c INFO: running hybridGM3@103 for 97 with 3870 remaining (b = 0.01)
8197.67/1164.37	c INFO: running hybridGM3@0 for 194 with 3836 remaining (b = 0.01)
8347.77/1185.83	c INFO: running hybridGM3@98 for 97 with 3814 remaining (b = 0.01)
8847.63/1255.81	c INFO: running adaptg2wsat2009++@102 for 97 with 3744 remaining (b = 0.01)
9154.20/1299.31	c INFO: running FH@97 for 291 with 3701 remaining (b = 0.02)
9375.88/1331.27	c INFO: running iPAWS@0 for 97 with 3669 remaining (b = 0.01)
9381.69/1332.11	c INFO: running hybridGM3@197 for 291 with 3668 remaining (b = 0.02)
9381.69/1332.11	Traceback (most recent call last):
9381.69/1332.11	  File "HOME/solve", line 33, in <module>
9381.69/1332.11	    plac.call(main)
9381.69/1332.11	  File "/home/evaluation/evaluation/tools64/lib/python2.7/site-packages/plac-0.8.1-py2.7.egg/plac_core.py", line 311, in call
9381.69/1332.15	    cmd, result = parser_from(obj).consume(arglist)
9381.69/1332.15	  File "/home/evaluation/evaluation/tools64/lib/python2.7/site-packages/plac-0.8.1-py2.7.egg/plac_core.py", line 192, in consume
9381.69/1332.15	    return cmd, self.func(*(args + varargs + extraopts), **kwargs)
9381.69/1332.15	  File "HOME/borg/build/lib/borg/tools/solve.py", line 84, in main
9381.69/1332.15	    (_, answer, _) = solver(input_path, budget / cores, cores)
9381.69/1332.15	  File "HOME/borg/build/lib/borg/bilevel.py", line 202, in __call__
9381.69/1332.15	    solver.go(max_cost * borg.defaults.machine_speed)
9381.69/1332.15	  File "HOME/borg/build/lib/borg/solvers.py", line 179, in go
9381.69/1332.15	    self._process.start()
9381.69/1332.15	  File "/home/evaluation/evaluation/tools64/lib/python2.7/multiprocessing/process.py", line 94, in start
9381.69/1332.15	    assert self._popen is None, 'cannot start a process twice'
9381.69/1332.15	AssertionError: cannot start a process twice

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3385404-1305566268/watcher-3385404-1305566268 -o /tmp/evaluation-result-3385404-1305566268/solver-3385404-1305566268 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385404-1305566268.cnf -seed 1589053202 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.48 8.45 6.35 3/438 31406
/proc/meminfo: memFree=17707332/32951124 swapFree=57414336/67111528
[pid=31406] ppid=31404 vsize=16236 CPUtime=0 cores=0-7
/proc/31406/stat : 31406 (python) R 31404 31406 31171 0 -1 4202496 996 0 0 0 0 0 0 0 20 0 1 0 746857273 16625664 869 33554432000 4194304 5683644 140735023496272 18446744073709551615 252457726661 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/31406/statm: 4059 872 371 364 0 546 0

[startup+0.028543 s]
/proc/loadavg: 8.48 8.45 6.35 3/438 31406
/proc/meminfo: memFree=17707332/32951124 swapFree=57414336/67111528
[pid=31406] ppid=31404 vsize=16392 CPUtime=0.01 cores=0-7
/proc/31406/stat : 31406 (python) R 31404 31406 31171 0 -1 4202496 1116 0 0 0 1 0 0 0 20 0 1 0 746857273 16785408 984 33554432000 4194304 5683644 140735023496272 18446744073709551615 252458063872 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/31406/statm: 4098 984 409 364 0 585 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 16392

[startup+0.10051 s]
/proc/loadavg: 8.48 8.45 6.35 3/438 31406
/proc/meminfo: memFree=17707332/32951124 swapFree=57414336/67111528
[pid=31406] ppid=31404 vsize=87704 CPUtime=0.07 cores=0-7
/proc/31406/stat : 31406 (python) R 31404 31406 31171 0 -1 4202496 2637 0 0 0 5 2 0 0 18 0 1 0 746857273 89808896 2354 33554432000 4194304 5683644 140735023496272 18446744073709551615 252458063872 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/31406/statm: 21926 2354 717 364 0 1622 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 87704

[startup+0.300522 s]
/proc/loadavg: 8.48 8.45 6.35 3/438 31406
/proc/meminfo: memFree=17707332/32951124 swapFree=57414336/67111528
[pid=31406] ppid=31404 vsize=135748 CPUtime=0.2 cores=0-7
/proc/31406/stat : 31406 (python) R 31404 31406 31171 0 -1 4202496 5961 935 1 0 14 6 0 0 17 0 1 0 746857273 139005952 4151 33554432000 4194304 5683644 140735023496272 18446744073709551615 252458061829 0 2147483391 16781312 2 0 0 0 17 2 0 0 0
/proc/31406/statm: 33937 4151 1032 364 0 3123 0
Current children cumulated CPU time (s) 0.2
Current children cumulated vsize (KiB) 135748

[startup+0.700433 s]
/proc/loadavg: 8.48 8.45 6.35 3/438 31406
/proc/meminfo: memFree=17707332/32951124 swapFree=57414336/67111528
[pid=31406] ppid=31404 vsize=279896 CPUtime=0.44 cores=0-7
/proc/31406/stat : 31406 (python) R 31404 31406 31171 0 -1 4202496 9622 935 1 0 32 12 0 0 16 0 1 0 746857273 286613504 7307 33554432000 4194304 5683644 140735023496272 18446744073709551615 252457426692 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/31406/statm: 69974 7307 1761 364 0 5898 0
Current children cumulated CPU time (s) 0.44
Current children cumulated vsize (KiB) 279896

[startup+1.50025 s]
/proc/loadavg: 8.48 8.45 6.35 3/439 31411
/proc/meminfo: memFree=17673472/32951124 swapFree=57414336/67111528
[pid=31406] ppid=31404 vsize=289484 CPUtime=1.24 cores=0-7
/proc/31406/stat : 31406 (python) R 31404 31406 31171 0 -1 4202496 14016 935 1 0 111 13 0 0 17 0 1 0 746857273 296431616 9756 33554432000 4194304 5683644 140735023496272 18446744073709551615 47868498781518 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/31406/statm: 72371 9756 1826 364 0 8295 0
Current children cumulated CPU time (s) 1.24
Current children cumulated vsize (KiB) 289484

[startup+3.10092 s]
/proc/loadavg: 8.04 8.36 6.33 3/439 31411
/proc/meminfo: memFree=17672768/32951124 swapFree=57414336/67111528
[pid=31406] ppid=31404 vsize=429948 CPUtime=2.65 cores=0-7
/proc/31406/stat : 31406 (python) S 31404 31406 31171 0 -1 4202496 20119 935 1 0 251 14 0 0 20 0 9 0 746857273 440266752 8204 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469892555 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31406/statm: 107487 8204 1891 364 0 43384 0
Current children cumulated CPU time (s) 2.65
Current children cumulated vsize (KiB) 429948

[startup+6.30132 s]
/proc/loadavg: 8.04 8.36 6.33 10/463 31436
/proc/meminfo: memFree=17613416/32951124 swapFree=57414336/67111528
[pid=31406] ppid=31404 vsize=429948 CPUtime=2.65 cores=0-7
/proc/31406/stat : 31406 (python) S 31404 31406 31171 0 -1 4202496 20119 935 1 0 251 14 0 0 20 0 9 0 746857273 440266752 8204 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469892555 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31406/statm: 107487 8204 1891 364 0 43384 0
[pid=31412] ppid=31406 vsize=282228 CPUtime=0 cores=0-7
/proc/31412/stat : 31412 (python) S 31406 31406 31171 0 -1 4202560 738 0 0 0 0 0 0 0 18 0 1 0 746857543 289001472 6328 33554432000 4194304 5683644 140735023496272 18446744073709551615 252458085903 0 0 16781312 514 17449768506588365058 0 0 17 6 0 0 0
/proc/31412/statm: 70557 6328 355 364 0 6475 0
[pid=31414] ppid=31412 vsize=1020188 CPUtime=2.06 cores=0-7
/proc/31414/stat : 31414 (gnovelty+2) R 31412 31414 31414 0 -1 4202496 1143 0 0 0 206 0 0 0 25 0 1 0 746857543 1044672512 876 33554432000 134512640 135663704 4287348384 18446744073709551615 134525278 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/31414/statm: 255047 876 113 282 0 254763 0
[pid=31415] ppid=31406 vsize=293116 CPUtime=0 cores=0-7
/proc/31415/stat : 31415 (python) S 31406 31406 31171 0 -1 4202560 750 0 0 0 0 0 0 0 18 0 1 0 746857546 300150784 6366 33554432000 4194304 5683644 140735023496272 18446744073709551615 252458085958 0 0 16781312 514 9761367478667018499 0 0 17 2 0 0 0
/proc/31415/statm: 73279 6366 349 364 0 9194 0
[pid=31417] ppid=31415 vsize=1020188 CPUtime=3.56 cores=0-7
/proc/31417/stat : 31417 (gnovelty+2) R 31415 31417 31417 0 -1 4202496 1141 0 0 0 356 0 0 0 25 0 1 0 746857546 1044672512 877 33554432000 134512640 135663704 4290471312 18446744073709551615 134524553 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/31417/statm: 255047 877 113 282 0 254763 0
[pid=31418] ppid=31406 vsize=302888 CPUtime=0 cores=0-7
/proc/31418/stat : 31418 (python) S 31406 31406 31171 0 -1 4202560 745 0 0 0 0 0 0 0 18 0 1 0 746857549 310157312 6503 33554432000 4194304 5683644 140735023496272 18446744073709551615 252458085958 0 0 16781312 514 6224504104169734402 0 0 17 2 0 0 0
/proc/31418/statm: 75722 6503 347 364 0 11634 0
[pid=31420] ppid=31418 vsize=421140 CPUtime=3.53 cores=0-7
/proc/31420/stat : 31420 (adaptg2wsat2009) R 31418 31420 31420 0 -1 4202496 1804 0 0 0 353 0 0 0 25 0 1 0 746857549 431247360 1538 33554432000 134512640 135107411 4290590512 18446744073709551615 134523232 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/31420/statm: 105285 1538 49 146 0 104161 0
[pid=31421] ppid=31406 vsize=313144 CPUtime=0 cores=0-7
/proc/31421/stat : 31421 (python) S 31406 31406 31171 0 -1 4202560 754 0 0 0 0 0 0 0 18 0 1 0 746857552 320659456 6507 33554432000 4194304 5683644 140735023496272 18446744073709551615 252458085958 0 0 16781312 514 1077755595492131074 0 0 17 1 0 0 0
/proc/31421/statm: 78286 6507 347 364 0 14195 0
[pid=31423] ppid=31421 vsize=421140 CPUtime=3.46 cores=0-7
/proc/31423/stat : 31423 (adaptg2wsat2009) R 31421 31423 31423 0 -1 4202496 1803 0 0 0 346 0 0 0 25 0 1 0 746857552 431247360 1538 33554432000 134512640 135107411 4291234384 18446744073709551615 134523424 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/31423/statm: 105285 1538 49 146 0 104161 0
[pid=31424] ppid=31406 vsize=323588 CPUtime=0 cores=0-7
/proc/31424/stat : 31424 (python) S 31406 31406 31171 0 -1 4202560 763 0 0 0 0 0 0 0 18 0 1 0 746857555 331354112 6513 33554432000 4194304 5683644 140735023496272 18446744073709551615 252458085958 0 0 16781312 514 6094461290539876610 0 0 17 1 0 0 0
/proc/31424/statm: 80897 6513 349 364 0 16803 0
[pid=31426] ppid=31424 vsize=421140 CPUtime=3.42 cores=0-7
/proc/31426/stat : 31426 (adaptg2wsat2009) R 31424 31426 31426 0 -1 4202496 1804 0 0 0 342 0 0 0 25 0 1 0 746857555 431247360 1538 33554432000 134512640 135107411 4290145248 18446744073709551615 134523261 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/31426/statm: 105285 1538 49 146 0 104161 0
[pid=31427] ppid=31406 vsize=399192 CPUtime=0 cores=0-7
/proc/31427/stat : 31427 (python) S 31406 31406 31171 0 -1 4202560 736 0 0 0 0 0 0 0 18 0 1 0 746857558 408772608 6688 33554432000 4194304 5683644 140735023496272 18446744073709551615 252458085958 0 0 16781312 514 9761359782085624066 0 0 17 1 0 0 0
/proc/31427/statm: 99798 6688 348 364 0 35701 0
[pid=31429] ppid=31427 vsize=1020188 CPUtime=3.42 cores=0-7
/proc/31429/stat : 31429 (gnovelty+2) R 31427 31429 31429 0 -1 4202496 1133 0 0 0 342 0 0 0 25 0 1 0 746857561 1044672512 876 33554432000 134512640 135663704 4293803856 18446744073709551615 134524916 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/31429/statm: 255047 876 113 282 0 254763 0
[pid=31430] ppid=31406 vsize=409448 CPUtime=0 cores=0-7
/proc/31430/stat : 31430 (python) S 31406 31406 31171 0 -1 4202560 744 0 0 0 0 0 0 0 18 0 1 0 746857561 419274752 6652 33554432000 4194304 5683644 140735023496272 18446744073709551615 252458085958 0 0 16781312 514 10265639245292994819 0 0 17 1 0 0 0
/proc/31430/statm: 102362 6652 348 364 0 38262 0
[pid=31432] ppid=31430 vsize=249144 CPUtime=3.41 cores=0-7
/proc/31432/stat : 31432 (FH) R 31430 31432 31432 0 -1 4202496 1550 0 0 0 341 0 0 0 25 0 1 0 746857562 255123456 1295 33554432000 134512640 134936988 4287175840 18446744073709551615 134523505 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/31432/statm: 62286 1295 37 104 0 61790 0
[pid=31433] ppid=31406 vsize=419704 CPUtime=0 cores=0-7
/proc/31433/stat : 31433 (python) S 31406 31406 31171 0 -1 4202560 780 0 0 0 0 0 0 0 18 0 1 0 746857569 429776896 6663 33554432000 4194304 5683644 140735023496272 18446744073709551615 252458085958 0 0 16781312 514 11048632536635769091 0 0 17 1 0 0 0
/proc/31433/statm: 104926 6663 354 364 0 40823 0
[pid=31435] ppid=31433 vsize=421140 CPUtime=1.51 cores=0-7
/proc/31435/stat : 31435 (adaptg2wsat2009) R 31433 31435 31435 0 -1 4202496 1804 0 0 0 151 0 0 0 25 0 1 0 746857584 431247360 1539 33554432000 134512640 135107411 4291634224 18446744073709551615 134523229 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/31435/statm: 105285 1539 49 146 0 104161 0
[pid=31406/tid=31413] ppid=31404 vsize=429948 CPUtime=0 cores=0-7
/proc/31406/task/31413/stat : 31413 (python) S 31404 31406 31171 0 -1 4202560 25 935 0 0 0 0 0 0 20 0 9 0 746857543 440266752 8204 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 6 0 0 0
[pid=31406/tid=31416] ppid=31404 vsize=429948 CPUtime=0 cores=0-7
/proc/31406/task/31416/stat : 31416 (python) S 31404 31406 31171 0 -1 4202560 24 935 0 0 0 0 0 0 20 0 9 0 746857546 440266752 8204 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 6 0 0 0
[pid=31406/tid=31419] ppid=31404 vsize=429948 CPUtime=0 cores=0-7
/proc/31406/task/31419/stat : 31419 (python) S 31404 31406 31171 0 -1 4202560 22 935 0 0 0 0 0 0 22 0 9 0 746857549 440266752 8204 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 0 0 0 0
[pid=31406/tid=31422] ppid=31404 vsize=429948 CPUtime=0 cores=0-7
/proc/31406/task/31422/stat : 31422 (python) S 31404 31406 31171 0 -1 4202560 25 935 0 0 0 0 0 0 19 0 9 0 746857552 440266752 8204 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 0 0 0 0
[pid=31406/tid=31425] ppid=31404 vsize=429948 CPUtime=0 cores=0-7
/proc/31406/task/31425/stat : 31425 (python) S 31404 31406 31171 0 -1 4202560 23 935 0 0 0 0 0 0 20 0 9 0 746857555 440266752 8204 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 7 0 0 0
[pid=31406/tid=31428] ppid=31404 vsize=429948 CPUtime=0 cores=0-7
/proc/31406/task/31428/stat : 31428 (python) S 31404 31406 31171 0 -1 4202560 94 935 0 0 0 0 0 0 19 0 9 0 746857558 440266752 8204 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 6 0 0 0
[pid=31406/tid=31431] ppid=31404 vsize=429948 CPUtime=0 cores=0-7
/proc/31406/task/31431/stat : 31431 (python) S 31404 31406 31171 0 -1 4202560 31 935 0 0 0 0 0 0 20 0 9 0 746857561 440266752 8204 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 1 0 0 0
[pid=31406/tid=31434] ppid=31404 vsize=429948 CPUtime=0 cores=0-7
/proc/31406/task/31434/stat : 31434 (python) S 31404 31406 31171 0 -1 4202560 32 935 0 0 0 0 0 0 21 0 9 0 746857569 440266752 8204 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 27.02
Current children cumulated vsize (KiB) 8167524

[startup+12.7011 s]
/proc/loadavg: 8.19 8.38 6.36 10/463 31436
/proc/meminfo: memFree=17606496/32951124 swapFree=57414336/67111528
[pid=31406] ppid=31404 vsize=429948 CPUtime=2.65 cores=0-7
/proc/31406/stat : 31406 (python) S 31404 31406 31171 0 -1 4202496 20119 935 1 0 251 14 0 0 20 0 9 0 746857273 440266752 8204 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469892555 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31406/statm: 107487 8204 1891 364 0 43384 0
[pid=31412] ppid=31406 vsize=282228 CPUtime=0.01 cores=0-7
/proc/31412/stat : 31412 (python) S 31406 31406 31171 0 -1 4202560 900 0 0 0 1 0 0 0 18 0 1 0 746857543 289001472 6351 33554432000 4194304 5683644 140735023496272 18446744073709551615 252458085903 0 0 16781312 514 17449768506588365058 0 0 17 6 0 0 0
/proc/31412/statm: 70557 6351 378 364 0 6475 0
[pid=31414] ppid=31412 vsize=1020188 CPUtime=5.19 cores=0-7
/proc/31414/stat : 31414 (gnovelty+2) R 31412 31414 31414 0 -1 4202496 1143 0 0 0 519 0 0 0 25 0 1 0 746857543 1044672512 876 33554432000 134512640 135663704 4287348384 18446744073709551615 134525691 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/31414/statm: 255047 876 113 282 0 254763 0
[pid=31415] ppid=31406 vsize=293116 CPUtime=0.01 cores=0-7
/proc/31415/stat : 31415 (python) S 31406 31406 31171 0 -1 4202560 924 0 0 0 0 1 0 0 18 0 1 0 746857546 300150784 6389 33554432000 4194304 5683644 140735023496272 18446744073709551615 252458085958 0 0 16781312 514 9761367478667018499 0 0 17 2 0 0 0
/proc/31415/statm: 73279 6389 372 364 0 9194 0
[pid=31417] ppid=31415 vsize=1020188 CPUtime=9.96 cores=0-7
/proc/31417/stat : 31417 (gnovelty+2) R 31415 31417 31417 0 -1 4202496 1141 0 0 0 996 0 0 0 25 0 1 0 746857546 1044672512 877 33554432000 134512640 135663704 4290471312 18446744073709551615 134525479 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/31417/statm: 255047 877 113 282 0 254763 0
[pid=31418] ppid=31406 vsize=302888 CPUtime=0.01 cores=0-7
/proc/31418/stat : 31418 (python) S 31406 31406 31171 0 -1 4202560 921 0 0 0 1 0 0 0 18 0 1 0 746857549 310157312 6526 33554432000 4194304 5683644 140735023496272 18446744073709551615 252458085958 0 0 16781312 514 6224504104169734402 0 0 17 2 0 0 0
/proc/31418/statm: 75722 6526 370 364 0 11634 0
[pid=31420] ppid=31418 vsize=421804 CPUtime=9.88 cores=0-7
/proc/31420/stat : 31420 (adaptg2wsat2009) R 31418 31420 31420 0 -1 4202496 2138 0 0 0 988 0 0 0 25 0 1 0 746857549 431927296 1872 33554432000 134512640 135107411 4290590512 18446744073709551615 134532242 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/31420/statm: 105451 1872 59 146 0 104327 0
[pid=31421] ppid=31406 vsize=313144 CPUtime=0.01 cores=0-7
/proc/31421/stat : 31421 (python) S 31406 31406 31171 0 -1 4202560 937 0 0 0 0 1 0 0 18 0 1 0 746857552 320659456 6530 33554432000 4194304 5683644 140735023496272 18446744073709551615 252458085958 0 0 16781312 514 1077755595492131074 0 0 17 1 0 0 0
/proc/31421/statm: 78286 6530 370 364 0 14195 0
[pid=31423] ppid=31421 vsize=421804 CPUtime=9.84 cores=0-7
/proc/31423/stat : 31423 (adaptg2wsat2009) R 31421 31423 31423 0 -1 4202496 2135 0 0 0 984 0 0 0 25 0 1 0 746857552 431927296 1870 33554432000 134512640 135107411 4291234384 18446744073709551615 134532349 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/31423/statm: 105451 1870 59 146 0 104327 0
[pid=31424] ppid=31406 vsize=323588 CPUtime=0.01 cores=0-7
/proc/31424/stat : 31424 (python) S 31406 31406 31171 0 -1 4202560 946 0 0 0 0 1 0 0 18 0 1 0 746857555 331354112 6536 33554432000 4194304 5683644 140735023496272 18446744073709551615 252458085958 0 0 16781312 514 6094461290539876610 0 0 17 1 0 0 0
/proc/31424/statm: 80897 6536 372 364 0 16803 0
[pid=31426] ppid=31424 vsize=421804 CPUtime=9.71 cores=0-7
/proc/31426/stat : 31426 (adaptg2wsat2009) R 31424 31426 31426 0 -1 4202496 2137 0 0 0 971 0 0 0 25 0 1 0 746857555 431927296 1871 33554432000 134512640 135107411 4290145248 18446744073709551615 134532277 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/31426/statm: 105451 1871 59 146 0 104327 0
[pid=31427] ppid=31406 vsize=399192 CPUtime=0.01 cores=0-7
/proc/31427/stat : 31427 (python) S 31406 31406 31171 0 -1 4202560 816 0 0 0 0 1 0 0 18 0 1 0 746857558 408772608 6711 33554432000 4194304 5683644 140735023496272 18446744073709551615 252458085958 0 0 16781312 514 9761359782085624066 0 0 17 1 0 0 0
/proc/31427/statm: 99798 6711 371 364 0 35701 0
[pid=31429] ppid=31427 vsize=1020188 CPUtime=9.82 cores=0-7
/proc/31429/stat : 31429 (gnovelty+2) R 31427 31429 31429 0 -1 4202496 1133 0 0 0 982 0 0 0 25 0 1 0 746857561 1044672512 876 33554432000 134512640 135663704 4293803856 18446744073709551615 134525255 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/31429/statm: 255047 876 113 282 0 254763 0
[pid=31430] ppid=31406 vsize=409448 CPUtime=0.02 cores=0-7
/proc/31430/stat : 31430 (python) S 31406 31406 31171 0 -1 4202560 827 0 0 0 1 1 0 0 18 0 1 0 746857561 419274752 6675 33554432000 4194304 5683644 140735023496272 18446744073709551615 252458085958 0 0 16781312 514 10265639245292994819 0 0 17 1 0 0 0
/proc/31430/statm: 102362 6675 371 364 0 38262 0

################
# More data... #
################

/proc/31507/statm: 83999 1750 59 147 0 83851 0
[pid=31505/tid=31512] ppid=31406 vsize=563288 CPUtime=0 cores=0-7
/proc/31505/task/31512/stat : 31512 (python) S 31406 31406 31171 0 -1 4202560 85 0 0 0 0 0 0 0 19 0 2 0 746917997 576806912 7032 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 514 18446604445248958528 0 0 -1 0 0 0 0
[pid=31513] ppid=31406 vsize=573544 CPUtime=0.74 cores=0-7
/proc/31513/stat : 31513 (python) S 31406 31406 31171 0 -1 4202560 1199 0 0 0 34 40 0 0 19 0 2 0 746918003 587309056 7051 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469892555 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/31513/statm: 143386 7051 407 364 0 79238 0
[pid=31515] ppid=31513 vsize=268424 CPUtime=194.79 cores=0-7
/proc/31515/stat : 31515 (hybridGM3) T 31513 31515 31515 0 -1 4202496 5659 0 0 0 19479 0 0 0 25 0 1 0 746918005 274866176 5378 33554432000 134512640 135204385 4287248416 18446744073709551615 134538041 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/31515/statm: 67106 5378 63 169 0 66936 0
[pid=31513/tid=31530] ppid=31406 vsize=573544 CPUtime=0 cores=0-7
/proc/31513/task/31530/stat : 31530 (python) S 31406 31406 31171 0 -1 4202560 85 0 0 0 0 0 0 0 19 0 2 0 746938295 587309056 7051 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 514 18446604445248959424 0 0 -1 1 0 0 0
[pid=31518] ppid=31406 vsize=583800 CPUtime=0.46 cores=0-7
/proc/31518/stat : 31518 (python) S 31406 31406 31171 0 -1 4202560 1198 0 0 0 23 23 0 0 19 0 2 0 746919521 597811200 7043 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469892555 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/31518/statm: 145950 7043 406 364 0 81799 0
[pid=31520] ppid=31518 vsize=1020188 CPUtime=101.78 cores=0-7
/proc/31520/stat : 31520 (gnovelty+2) T 31518 31520 31520 0 -1 4202496 1162 0 0 0 10178 0 0 0 25 0 1 0 746919521 1044672512 876 33554432000 134512640 135663704 4288726288 18446744073709551615 134524553 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/31520/statm: 255047 876 113 282 0 254763 0
[pid=31518/tid=31527] ppid=31406 vsize=583800 CPUtime=0 cores=0-7
/proc/31518/task/31527/stat : 31527 (python) S 31406 31406 31171 0 -1 4202560 82 0 0 0 0 0 0 0 19 0 2 0 746931977 597811200 7043 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 514 18446604453859183552 0 0 -1 3 0 0 0
[pid=31524] ppid=31406 vsize=594056 CPUtime=2.22 cores=0-7
/proc/31524/stat : 31524 (python) S 31406 31406 31171 0 -1 4202560 999 0 0 0 106 116 0 0 18 0 1 0 746930454 608313344 7063 33554432000 4194304 5683644 140735023496272 18446744073709551615 252458085958 0 0 16781312 514 4442888993331183873 0 0 17 1 0 0 0
/proc/31524/statm: 148514 7063 372 364 0 84360 0
[pid=31526] ppid=31524 vsize=459764 CPUtime=579.01 cores=0-7
/proc/31526/stat : 31526 (TNM) R 31524 31526 31526 0 -1 4202496 1942 0 0 0 57901 0 0 0 25 0 1 0 746930455 470798336 1657 33554432000 134512640 135008748 4293013008 18446744073709551615 134527773 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/31526/statm: 114941 1657 46 122 0 113841 0
[pid=31531] ppid=31406 vsize=604312 CPUtime=0.54 cores=0-7
/proc/31531/stat : 31531 (python) S 31406 31406 31171 0 -1 4202560 1195 0 0 0 28 26 0 0 19 0 2 0 746938300 618815488 7049 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469892555 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/31531/statm: 151078 7049 400 364 0 86921 0
[pid=31533] ppid=31531 vsize=335996 CPUtime=103 cores=0-7
/proc/31533/stat : 31533 (iPAWS) T 31531 31533 31533 0 -1 4202496 2031 0 0 0 10299 1 0 0 25 0 1 0 746938301 344059904 1749 33554432000 134512640 135114423 4293909536 18446744073709551615 134517052 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/31533/statm: 83999 1749 59 147 0 83851 0
[pid=31531/tid=31548] ppid=31406 vsize=604312 CPUtime=0 cores=0-7
/proc/31531/task/31548/stat : 31548 (python) S 31406 31406 31171 0 -1 4202560 86 0 0 0 0 0 0 0 19 0 2 0 746953153 618815488 7049 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 514 18446604444477681792 0 0 -1 0 0 0 0
[pid=31534] ppid=31406 vsize=614568 CPUtime=0.48 cores=0-7
/proc/31534/stat : 31534 (python) S 31406 31406 31171 0 -1 4202560 1195 0 0 0 24 24 0 0 19 0 2 0 746938616 629317632 7126 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469892555 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/31534/statm: 153642 7126 400 364 0 89482 0
[pid=31536] ppid=31534 vsize=1020188 CPUtime=103.02 cores=0-7
/proc/31536/stat : 31536 (gnovelty+2) T 31534 31536 31536 0 -1 4202496 1159 0 0 0 10302 0 0 0 25 0 1 0 746938617 1044672512 876 33554432000 134512640 135663704 4288815136 18446744073709551615 134524882 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/31536/statm: 255047 876 113 282 0 254763 0
[pid=31534/tid=31544] ppid=31406 vsize=614568 CPUtime=0 cores=0-7
/proc/31534/task/31544/stat : 31544 (python) S 31406 31406 31171 0 -1 4202560 82 0 0 0 0 0 0 0 19 0 2 0 746951671 629317632 7126 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 514 18446604453861255296 0 0 -1 1 0 0 0
[pid=31539] ppid=31406 vsize=624824 CPUtime=1.37 cores=0-7
/proc/31539/stat : 31539 (python) S 31406 31406 31171 0 -1 4202560 1213 0 0 0 68 69 0 0 18 0 2 0 746946394 639819776 7062 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469892555 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/31539/statm: 156206 7062 401 364 0 92043 0
[pid=31541] ppid=31539 vsize=268424 CPUtime=294.89 cores=0-7
/proc/31541/stat : 31541 (hybridGM3) T 31539 31541 31541 0 -1 4202496 5661 0 0 0 29487 2 0 0 25 0 1 0 746946394 274866176 5377 33554432000 134512640 135204385 4290420048 18446744073709551615 134538068 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/31541/statm: 67106 5377 63 169 0 66936 0
[pid=31539/tid=31561] ppid=31406 vsize=624824 CPUtime=0 cores=0-7
/proc/31539/task/31561/stat : 31561 (python) S 31406 31406 31171 0 -1 4202560 85 0 0 0 0 0 0 0 18 0 2 0 746970297 639819776 7062 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 514 18446604453881467008 0 0 -1 1 0 0 0
[pid=31545] ppid=31406 vsize=635080 CPUtime=0.41 cores=0-7
/proc/31545/stat : 31545 (python) S 31406 31406 31171 0 -1 4202560 1252 0 0 0 24 17 0 0 19 0 2 0 746951677 650321920 7127 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469892555 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/31545/statm: 158770 7127 406 364 0 94604 0
[pid=31547] ppid=31545 vsize=251288 CPUtime=98.9 cores=0-7
/proc/31547/stat : 31547 (FH) T 31545 31547 31547 0 -1 4202496 1992 0 0 0 9890 0 0 0 25 0 1 0 746951678 257318912 1693 33554432000 134512640 134936988 4288365824 18446744073709551615 134552486 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/31547/statm: 62822 1693 41 104 0 62326 0
[pid=31545/tid=31554] ppid=31406 vsize=635080 CPUtime=0 cores=0-7
/proc/31545/task/31554/stat : 31554 (python) S 31406 31406 31171 0 -1 4202560 84 0 0 0 0 0 0 0 19 0 2 0 746962318 650321920 7127 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 514 18446604453985334208 0 0 -1 4 0 0 0
[pid=31549] ppid=31406 vsize=645336 CPUtime=1.27 cores=0-7
/proc/31549/stat : 31549 (python) S 31406 31406 31171 0 -1 4202560 1265 0 0 0 63 64 0 0 18 0 2 0 746953160 660824064 7190 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469892555 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/31549/statm: 161334 7190 407 364 0 97165 0
[pid=31551] ppid=31549 vsize=268424 CPUtime=295.26 cores=0-7
/proc/31551/stat : 31551 (hybridGM3) T 31549 31551 31551 0 -1 4202496 5666 0 0 0 29525 1 0 0 25 0 1 0 746953160 274866176 5377 33554432000 134512640 135204385 4289318672 18446744073709551615 134537711 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/31551/statm: 67106 5377 63 169 0 66936 0
[pid=31549/tid=31566] ppid=31406 vsize=645336 CPUtime=0 cores=0-7
/proc/31549/task/31566/stat : 31566 (python) S 31406 31406 31171 0 -1 4202560 87 0 0 0 0 0 0 0 18 0 2 0 746975850 660824064 7190 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 514 18446604440953960448 0 0 -1 1 0 0 0
[pid=31555] ppid=31406 vsize=655592 CPUtime=1.04 cores=0-7
/proc/31555/stat : 31555 (python) S 31406 31406 31171 0 -1 4202560 1069 0 0 0 50 54 0 0 18 0 1 0 746962324 671326208 7113 33554432000 4194304 5683644 140735023496272 18446744073709551615 252458085958 0 0 16781312 514 4442888993331183873 0 0 17 6 0 0 0
/proc/31555/statm: 163898 7113 378 364 0 99726 0
[pid=31557] ppid=31555 vsize=421344 CPUtime=224.69 cores=0-7
/proc/31557/stat : 31557 (NCVWr) R 31555 31557 31557 0 -1 4202496 1999 0 0 0 22469 0 0 0 25 0 1 0 746962324 431456256 1712 33554432000 134512640 135113558 4289165008 18446744073709551615 134533479 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/31557/statm: 105336 1712 58 147 0 104211 0
[pid=31562] ppid=31406 vsize=665848 CPUtime=0.63 cores=0-7
/proc/31562/stat : 31562 (python) S 31406 31406 31171 0 -1 4202560 1087 0 0 0 33 30 0 0 18 0 1 0 746973710 681828352 7118 33554432000 4194304 5683644 140735023496272 18446744073709551615 252458085958 0 0 16781312 514 4442888993331183873 0 0 17 1 0 0 0
/proc/31562/statm: 166462 7118 378 364 0 102287 0
[pid=31564] ppid=31562 vsize=268424 CPUtime=157.48 cores=0-7
/proc/31564/stat : 31564 (hybridGM3) R 31562 31564 31564 0 -1 4202496 5669 0 0 0 15747 1 0 0 25 0 1 0 746973711 274866176 5377 33554432000 134512640 135204385 4292283488 18446744073709551615 134538435 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/31564/statm: 67106 5377 63 169 0 66936 0
[pid=31574] ppid=31406 vsize=676104 CPUtime=0 cores=0-7
/proc/31574/stat : 31574 (python) S 31406 31406 31171 0 -1 4202560 952 0 0 0 0 0 0 0 19 0 1 0 746990401 692330496 7112 33554432000 4194304 5683644 140735023496272 18446744073709551615 252458085958 0 0 16781312 514 4522556036762042367 0 0 17 1 0 0 0
/proc/31574/statm: 169026 7112 352 364 0 104848 0
[pid=31576] ppid=31574 vsize=335996 CPUtime=0.34 cores=0-7
/proc/31576/stat : 31576 (iPAWS) R 31574 31576 31576 0 -1 4202496 2036 0 0 0 34 0 0 0 21 0 1 0 746990401 344059904 1747 33554432000 134512640 135114423 4294896048 18446744073709551615 134547183 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/31576/statm: 83999 1747 59 147 0 83851 0
[pid=31406/tid=31413] ppid=31404 vsize=686348 CPUtime=0 cores=0-7
/proc/31406/task/31413/stat : 31413 (python) S 31404 31406 31171 0 -1 4202560 25 935 0 0 0 0 0 0 20 0 34 0 746857543 702820352 8706 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 6 0 0 0
[pid=31406/tid=31416] ppid=31404 vsize=686348 CPUtime=0 cores=0-7
/proc/31406/task/31416/stat : 31416 (python) S 31404 31406 31171 0 -1 4202560 24 935 0 0 0 0 0 0 20 0 34 0 746857546 702820352 8706 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 6 0 0 0
[pid=31406/tid=31419] ppid=31404 vsize=686348 CPUtime=0 cores=0-7
/proc/31406/task/31419/stat : 31419 (python) S 31404 31406 31171 0 -1 4202560 35 935 0 0 0 0 0 0 18 0 34 0 746857549 702820352 8706 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 0 0 0 0
[pid=31406/tid=31422] ppid=31404 vsize=686348 CPUtime=0 cores=0-7
/proc/31406/task/31422/stat : 31422 (python) S 31404 31406 31171 0 -1 4202560 32 935 0 0 0 0 0 0 18 0 34 0 746857552 702820352 8706 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 0 0 0 0
[pid=31406/tid=31425] ppid=31404 vsize=686348 CPUtime=0 cores=0-7
/proc/31406/task/31425/stat : 31425 (python) S 31404 31406 31171 0 -1 4202560 28 935 0 0 0 0 0 0 18 0 34 0 746857555 702820352 8706 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 7 0 0 0
[pid=31406/tid=31428] ppid=31404 vsize=686348 CPUtime=0 cores=0-7
/proc/31406/task/31428/stat : 31428 (python) S 31404 31406 31171 0 -1 4202560 94 935 0 0 0 0 0 0 19 0 34 0 746857558 702820352 8706 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 6 0 0 0
[pid=31406/tid=31431] ppid=31404 vsize=686348 CPUtime=0 cores=0-7
/proc/31406/task/31431/stat : 31431 (python) S 31404 31406 31171 0 -1 4202560 36 935 0 0 0 0 0 0 18 0 34 0 746857561 702820352 8706 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 1 0 0 0
[pid=31406/tid=31434] ppid=31404 vsize=686348 CPUtime=0 cores=0-7
/proc/31406/task/31434/stat : 31434 (python) S 31404 31406 31171 0 -1 4202560 36 935 0 0 0 0 0 0 18 0 34 0 746857569 702820352 8706 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 6 0 0 0
[pid=31406/tid=31441] ppid=31404 vsize=686348 CPUtime=0 cores=0-7
/proc/31406/task/31441/stat : 31441 (python) S 31404 31406 31171 0 -1 4202560 48 935 0 0 0 0 0 0 18 0 34 0 746867584 702820352 8706 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 6 0 0 0
[pid=31406/tid=31446] ppid=31404 vsize=686348 CPUtime=0 cores=0-7
/proc/31406/task/31446/stat : 31446 (python) S 31404 31406 31171 0 -1 4202560 47 935 0 0 0 0 0 0 18 0 34 0 746867589 702820352 8706 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 6 0 0 0
[pid=31406/tid=31449] ppid=31404 vsize=686348 CPUtime=0 cores=0-7
/proc/31406/task/31449/stat : 31449 (python) S 31404 31406 31171 0 -1 4202560 39 935 0 0 0 0 0 0 18 0 34 0 746867593 702820352 8706 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 6 0 0 0
[pid=31406/tid=31453] ppid=31404 vsize=686348 CPUtime=0 cores=0-7
/proc/31406/task/31453/stat : 31453 (python) S 31404 31406 31171 0 -1 4202560 34 935 0 0 0 0 0 0 24 0 34 0 746867599 702820352 8706 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 6 0 0 0
[pid=31406/tid=31458] ppid=31404 vsize=686348 CPUtime=0 cores=0-7
/proc/31406/task/31458/stat : 31458 (python) S 31404 31406 31171 0 -1 4202560 54 935 0 0 0 0 0 0 18 0 34 0 746869394 702820352 8706 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 6 0 0 0
[pid=31406/tid=31462] ppid=31404 vsize=686348 CPUtime=0 cores=0-7
/proc/31406/task/31462/stat : 31462 (python) S 31404 31406 31171 0 -1 4202560 55 935 0 0 0 0 0 0 18 0 34 0 746869409 702820352 8706 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 6 0 0 0
[pid=31406/tid=31466] ppid=31404 vsize=686348 CPUtime=0 cores=0-7
/proc/31406/task/31466/stat : 31466 (python) S 31404 31406 31171 0 -1 4202560 57 935 0 0 0 0 0 0 18 0 34 0 746870039 702820352 8706 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 6 0 0 0
[pid=31406/tid=31470] ppid=31404 vsize=686348 CPUtime=0 cores=0-7
/proc/31406/task/31470/stat : 31470 (python) S 31404 31406 31171 0 -1 4202560 42 935 0 0 0 0 0 0 18 0 34 0 746870595 702820352 8706 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 3 0 0 0
[pid=31406/tid=31476] ppid=31404 vsize=686348 CPUtime=0 cores=0-7
/proc/31406/task/31476/stat : 31476 (python) S 31404 31406 31171 0 -1 4202560 34 935 0 0 0 0 0 0 19 0 34 0 746878127 702820352 8706 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 6 0 0 0
[pid=31406/tid=31480] ppid=31404 vsize=686348 CPUtime=0 cores=0-7
/proc/31406/task/31480/stat : 31480 (python) S 31404 31406 31171 0 -1 4202560 43 935 0 0 0 0 0 0 18 0 34 0 746878292 702820352 8706 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 6 0 0 0
[pid=31406/tid=31493] ppid=31404 vsize=686348 CPUtime=0 cores=0-7
/proc/31406/task/31493/stat : 31493 (python) S 31404 31406 31171 0 -1 4202560 44 935 0 0 0 0 0 0 19 0 34 0 746893995 702820352 8706 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 6 0 0 0
[pid=31406/tid=31498] ppid=31404 vsize=686348 CPUtime=0 cores=0-7
/proc/31406/task/31498/stat : 31498 (python) S 31404 31406 31171 0 -1 4202560 45 935 0 0 0 0 0 0 18 0 34 0 746898022 702820352 8706 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 6 0 0 0
[pid=31406/tid=31502] ppid=31404 vsize=686348 CPUtime=0 cores=0-7
/proc/31406/task/31502/stat : 31502 (python) S 31404 31406 31171 0 -1 4202560 35 935 0 0 0 0 0 0 19 0 34 0 746899877 702820352 8706 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 6 0 0 0
[pid=31406/tid=31506] ppid=31404 vsize=686348 CPUtime=0 cores=0-7
/proc/31406/task/31506/stat : 31506 (python) S 31404 31406 31171 0 -1 4202560 41 935 0 0 0 0 0 0 19 0 34 0 746904949 702820352 8706 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 6 0 0 0
[pid=31406/tid=31514] ppid=31404 vsize=686348 CPUtime=0 cores=0-7
/proc/31406/task/31514/stat : 31514 (python) S 31404 31406 31171 0 -1 4202560 39 935 0 0 0 0 0 0 19 0 34 0 746918003 702820352 8706 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 6 0 0 0
[pid=31406/tid=31519] ppid=31404 vsize=686348 CPUtime=0 cores=0-7
/proc/31406/task/31519/stat : 31519 (python) S 31404 31406 31171 0 -1 4202560 33 935 0 0 0 0 0 0 19 0 34 0 746919521 702820352 8706 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 6 0 0 0
[pid=31406/tid=31525] ppid=31404 vsize=686348 CPUtime=0 cores=0-7
/proc/31406/task/31525/stat : 31525 (python) S 31404 31406 31171 0 -1 4202560 36 935 0 0 0 0 0 0 19 0 34 0 746930454 702820352 8706 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 6 0 0 0
[pid=31406/tid=31532] ppid=31404 vsize=686348 CPUtime=0 cores=0-7
/proc/31406/task/31532/stat : 31532 (python) S 31404 31406 31171 0 -1 4202560 35 935 0 0 0 0 0 0 19 0 34 0 746938300 702820352 8706 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 6 0 0 0
[pid=31406/tid=31535] ppid=31404 vsize=686348 CPUtime=0 cores=0-7
/proc/31406/task/31535/stat : 31535 (python) S 31404 31406 31171 0 -1 4202560 33 935 0 0 0 0 0 0 19 0 34 0 746938617 702820352 8706 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 6 0 0 0
[pid=31406/tid=31540] ppid=31404 vsize=686348 CPUtime=0 cores=0-7
/proc/31406/task/31540/stat : 31540 (python) S 31404 31406 31171 0 -1 4202560 37 935 0 0 0 0 0 0 18 0 34 0 746946394 702820352 8706 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 6 0 0 0
[pid=31406/tid=31546] ppid=31404 vsize=686348 CPUtime=0 cores=0-7
/proc/31406/task/31546/stat : 31546 (python) S 31404 31406 31171 0 -1 4202560 35 935 0 0 0 0 0 0 19 0 34 0 746951678 702820352 8706 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 6 0 0 0
[pid=31406/tid=31550] ppid=31404 vsize=686348 CPUtime=0 cores=0-7
/proc/31406/task/31550/stat : 31550 (python) S 31404 31406 31171 0 -1 4202560 39 935 0 0 0 0 0 0 18 0 34 0 746953160 702820352 8706 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 6 0 0 0
[pid=31406/tid=31556] ppid=31404 vsize=686348 CPUtime=0 cores=0-7
/proc/31406/task/31556/stat : 31556 (python) S 31404 31406 31171 0 -1 4202560 33 935 0 0 0 0 0 0 20 0 34 0 746962324 702820352 8706 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 6 0 0 0
[pid=31406/tid=31563] ppid=31404 vsize=686348 CPUtime=0 cores=0-7
/proc/31406/task/31563/stat : 31563 (python) S 31404 31406 31171 0 -1 4202560 36 935 0 0 0 0 0 0 19 0 34 0 746973710 702820352 8706 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 6 0 0 0
[pid=31406/tid=31575] ppid=31404 vsize=686348 CPUtime=0 cores=0-7
/proc/31406/task/31575/stat : 31575 (python) S 31404 31406 31171 0 -1 4202560 34 935 0 0 0 0 0 0 19 0 34 0 746990401 702820352 8706 33554432000 4194304 5683644 140735023496272 18446744073709551615 252469889457 0 0 16781312 2 18446604446895220672 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 9381.69
Current children cumulated vsize (KiB) 32415684

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 31406 and gives
#  childrusage.ru_utime.tv_sec=2230
#  childrusage.ru_utime.tv_usec=362933
#  childrusage.ru_stime.tv_sec=19
#  childrusage.ru_stime.tv_usec=619017
# CPU time returned by wait4() is 2249.98
# while last known CPU time is 9381.69
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1332.22
CPU time (s): 9381.69
CPU user time (s): 9362.29
CPU system time (s): 19.4
CPU usage (%): 704.215
Max. virtual memory (cumulated for all children) (KiB): 32415684

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2230.36
system time used= 19.619
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 113070
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 17513
involuntary context switches= 19049

runsolver used 16.0166 second user time and 36.9624 second system time

The end

Launcher Data 

Begin job on node111 at 2011-05-16 19:17:48
IDJOB=3385404
IDBENCH=82669
IDSOLVER=1870
FILE ID=node111/3385404-1305566268
RUNJOBID= node111-1305564966-31185
PBS_JOBID= 13350139
Free space on /tmp= 73564 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/large/unif-k7-r85-v400-c34000-S586055573-067-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3385404-1305566268/watcher-3385404-1305566268 -o /tmp/evaluation-result-3385404-1305566268/solver-3385404-1305566268 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385404-1305566268.cnf -seed 1589053202 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= d673a60901b4a946b283fdbb02f99fdc
RANDOM SEED=1589053202

node111.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5428.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      17707496 kB
Buffers:        309964 kB
Cached:        6416700 kB
SwapCached:      19200 kB
Active:        7613960 kB
Inactive:      7359388 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      17707496 kB
SwapTotal:    67111528 kB
SwapFree:     57414336 kB
Dirty:           32648 kB
Writeback:           0 kB
AnonPages:     8237024 kB
Mapped:          26308 kB
Slab:           134796 kB
PageTables:      62716 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 137470560 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73532 MiB
End job on node111 at 2011-05-16 19:40:02