Trace number 3385313

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)? (MO) 9517.73 1203.53

General information on the benchmark

Namerandom/large/
unif-k3-r4.2-v40000-c168000-S471691933-060-UNKNOWN.cnf
MD5SUM91da78a39fd68b6c63602d278464956f
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark593.197
Satisfiable
(Un)Satisfiability was proved
Number of variables40000
Number of clauses168000
Sum of the clauses size504000
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3168000
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.36/0.66	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.36/0.66	c INFO: solving HOME/instance-3385313-1305565062.cnf
0.36/0.66	c INFO: computing task features
0.94/1.28	c INFO: computed variable-clause graph statistics
1.03/1.30	c INFO: computed clause balance statistics
3.93/4.26	c INFO: computed variable balance statistics
6.24/6.52	c INFO: computed small-clause counts
6.24/6.54	c INFO: computed Horn-clause counts
8.23/8.55	c INFO: computed variable graph statistics
8.23/8.55	c INFO: collected features for HOME/instance-3385313-1305565062.cnf in 7.87 s
8.23/8.58	c INFO: running hybridGM3@0 for 97 with 4992 remaining (b = 0.35)
8.34/8.61	c INFO: running hybridGM3@0 for 97 with 4992 remaining (b = 0.21)
8.34/8.64	c INFO: running hybridGM3@0 for 97 with 4992 remaining (b = 0.14)
8.34/8.67	c INFO: running hybridGM3@0 for 97 with 4992 remaining (b = 0.10)
8.34/8.70	c INFO: running hybridGM3@0 for 97 with 4991 remaining (b = 0.07)
8.70/8.74	c INFO: running hybridGM3@0 for 194 with 4991 remaining (b = 0.10)
8.70/8.77	c INFO: running hybridGM3@0 for 194 with 4991 remaining (b = 0.08)
8.70/8.81	c INFO: running hybridGM3@0 for 194 with 4991 remaining (b = 0.06)
811.42/109.35	c INFO: running hybridGM3@0 for 97 with 4891 remaining (b = 0.03)
811.42/109.38	c INFO: running hybridGM3@0 for 97 with 4891 remaining (b = 0.03)
812.20/109.42	c INFO: running hybridGM3@0 for 97 with 4891 remaining (b = 0.03)
812.20/109.47	c INFO: running hybridGM3@0 for 97 with 4891 remaining (b = 0.03)
812.95/109.50	c INFO: running hybridGM3@0 for 97 with 4891 remaining (b = 0.03)
1572.17/204.70	c INFO: running hybridGM3@100 for 97 with 4795 remaining (b = 0.03)
1572.17/204.73	c INFO: running hybridGM3@100 for 97 with 4795 remaining (b = 0.03)
1572.17/204.77	c INFO: running TNM@0 for 97 with 4795 remaining (b = 0.03)
1572.95/204.82	c INFO: running hybridGM3@100 for 97 with 4795 remaining (b = 0.02)
1580.93/205.88	c INFO: running hybridGM3@99 for 97 with 4794 remaining (b = 0.02)
1581.74/205.92	c INFO: running hybridGM3@0 for 97 with 4794 remaining (b = 0.02)
1620.82/210.86	c INFO: running hybridGM3@100 for 97 with 4789 remaining (b = 0.02)
1628.78/211.88	c INFO: running hybridGM3@95 for 97 with 4788 remaining (b = 0.02)
2372.81/305.15	c INFO: running hybridGM3@0 for 97 with 4695 remaining (b = 0.02)
2379.99/306.07	c INFO: running hybridGM3@0 for 194 with 4694 remaining (b = 0.04)
2380.75/306.11	c INFO: running TNM@0 for 97 with 4694 remaining (b = 0.02)
2381.41/306.22	c INFO: running hybridGM3@296 for 97 with 4694 remaining (b = 0.02)
2381.41/306.26	c INFO: running hybridGM3@0 for 194 with 4694 remaining (b = 0.03)
2381.91/306.30	c INFO: running hybridGM3@295 for 97 with 4694 remaining (b = 0.02)
2389.41/307.30	c INFO: running hybridGM3@100 for 97 with 4693 remaining (b = 0.02)
2445.18/314.26	c INFO: running hybridGM3@299 for 97 with 4686 remaining (b = 0.02)
3141.13/401.59	c INFO: running TNM@0 for 97 with 4599 remaining (b = 0.02)
3141.79/401.67	c INFO: running hybridGM3@390 for 97 with 4599 remaining (b = 0.01)
3148.66/402.53	c INFO: running TNM@0 for 97 with 4598 remaining (b = 0.01)
3149.36/402.66	c INFO: running hybridGM3@396 for 97 with 4598 remaining (b = 0.01)
3180.34/406.51	c INFO: running TNM@0 for 97 with 4594 remaining (b = 0.01)
3205.04/409.62	c INFO: running hybridGM3@394 for 97 with 4591 remaining (b = 0.01)
3901.74/497.05	c INFO: running TNM@0 for 97 with 4503 remaining (b = 0.01)
3901.74/497.10	c INFO: running TNM@0 for 97 with 4503 remaining (b = 0.01)
3933.49/501.00	c INFO: running TNM@0 for 97 with 4499 remaining (b = 0.01)
3948.59/503.00	c INFO: running hybridGM3@100 for 97 with 4497 remaining (b = 0.01)
3951.75/503.32	c INFO: running TNM@95 for 1940 with 4497 remaining (b = 0.20)
3952.55/503.46	c INFO: running hybridGM3@100 for 97 with 4497 remaining (b = 0.01)
3980.44/506.92	c INFO: running hybridGM3@100 for 97 with 4493 remaining (b = 0.01)
4012.99/511.01	c INFO: running hybridGM3@0 for 194 with 4489 remaining (b = 0.02)
4701.84/597.51	c INFO: running hybridGM3@0 for 194 with 4403 remaining (b = 0.02)
4701.84/597.57	c INFO: running adaptg2wsat2009++@0 for 194 with 4403 remaining (b = 0.02)
4712.12/598.85	c INFO: running TNM@0 for 97 with 4401 remaining (b = 0.01)
4732.82/601.43	c INFO: running TNM@0 for 97 with 4399 remaining (b = 0.01)
4763.82/605.34	c INFO: running TNM@0 for 97 with 4395 remaining (b = 0.01)
4772.53/606.44	c INFO: running hybridGM3@297 for 97 with 4394 remaining (b = 0.01)
5510.71/699.29	c INFO: running TNM@101 for 1940 with 4301 remaining (b = 0.13)
5531.49/701.83	c INFO: running hybridGM3@397 for 97 with 4298 remaining (b = 0.01)
5531.49/701.89	c INFO: running hybridGM3@0 for 194 with 4298 remaining (b = 0.01)
5582.25/708.26	c INFO: running hybridGM3@100 for 97 with 4292 remaining (b = 0.01)
5610.19/711.80	c INFO: running adaptg2wsat2009++@0 for 194 with 4288 remaining (b = 0.01)
6267.07/794.36	c INFO: running hybridGM3@0 for 194 with 4206 remaining (b = 0.01)
6269.92/794.73	c INFO: running hybridGM3@99 for 97 with 4205 remaining (b = 0.01)
6337.86/803.25	c INFO: running adaptg2wsat2009++@94 for 97 with 4197 remaining (b = 0.01)
6388.86/809.68	c INFO: running adaptg2wsat2009++@0 for 194 with 4191 remaining (b = 0.01)
7028.99/890.14	c INFO: running hybridGM3@196 for 291 with 4110 remaining (b = 0.02)
7100.55/899.12	c INFO: running hybridGM3@0 for 194 with 4101 remaining (b = 0.01)
7112.48/900.69	c INFO: running adaptg2wsat2009++@191 for 97 with 4100 remaining (b = 0.01)
7223.83/914.69	c INFO: running FH@0 for 97 with 4086 remaining (b = 0.00)
7835.75/991.61	c INFO: running hybridGM3@195 for 291 with 4009 remaining (b = 0.01)
7911.22/1001.17	c INFO: running adaptg2wsat2009++@389 for 194 with 3999 remaining (b = 0.01)
7993.88/1011.52	c INFO: running hybridGM3@194 for 291 with 3989 remaining (b = 0.01)
8014.62/1014.16	c INFO: running TNM@0 for 97 with 3986 remaining (b = 0.00)
8667.56/1096.39	c INFO: running TNM@0 for 97 with 3904 remaining (b = 0.00)
8812.88/1114.61	c INFO: running TNM@0 for 97 with 3886 remaining (b = 0.00)
9334.07/1180.28	c INFO: running hybridGM3@393 for 97 with 3820 remaining (b = 0.01)
9482.64/1199.01	c INFO: running hybridGM3@97 for 97 with 3801 remaining (b = 0.00)
9513.56/1202.87	c INFO: running NCVWr@0 for 485 with 3797 remaining (b = 0.02)
9517.73/1203.43	c INFO: running hybridGM3@201 for 291 with 3797 remaining (b = 0.01)
9517.73/1203.43	Traceback (most recent call last):
9517.73/1203.43	  File "HOME/solve", line 33, in <module>
9517.73/1203.43	    plac.call(main)
9517.73/1203.43	  File "/home/evaluation/evaluation/tools64/lib/python2.7/site-packages/plac-0.8.1-py2.7.egg/plac_core.py", line 311, in call
9517.73/1203.43	    cmd, result = parser_from(obj).consume(arglist)
9517.73/1203.43	  File "/home/evaluation/evaluation/tools64/lib/python2.7/site-packages/plac-0.8.1-py2.7.egg/plac_core.py", line 192, in consume
9517.73/1203.43	    return cmd, self.func(*(args + varargs + extraopts), **kwargs)
9517.73/1203.43	  File "HOME/borg/build/lib/borg/tools/solve.py", line 84, in main
9517.73/1203.43	    (_, answer, _) = solver(input_path, budget / cores, cores)
9517.73/1203.43	  File "HOME/borg/build/lib/borg/bilevel.py", line 202, in __call__
9517.73/1203.43	    solver.go(max_cost * borg.defaults.machine_speed)
9517.73/1203.43	  File "HOME/borg/build/lib/borg/solvers.py", line 179, in go
9517.73/1203.43	    self._process.start()
9517.73/1203.43	  File "/home/evaluation/evaluation/tools64/lib/python2.7/multiprocessing/process.py", line 94, in start
9517.73/1203.43	    assert self._popen is None, 'cannot start a process twice'
9517.73/1203.43	AssertionError: cannot start a process twice

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3385313-1305565062/watcher-3385313-1305565062 -o /tmp/evaluation-result-3385313-1305565062/solver-3385313-1305565062 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385313-1305565062.cnf -seed 1978747136 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.57 2.27 0.81 1/441 11086
/proc/meminfo: memFree=14901528/32950928 swapFree=60199076/67111528
[pid=11086] ppid=11084 vsize=15760 CPUtime=0 cores=0-7
/proc/11086/stat : 11086 (python) S 11084 11086 11001 0 -1 4202496 844 0 0 0 0 0 0 0 17 0 1 0 668105013 16138240 727 33554432000 4194304 5683644 140734270847040 18446744073709551615 268272687104 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 0
/proc/11086/statm: 3940 727 334 364 0 427 0

[startup+0.037628 s]
/proc/loadavg: 6.57 2.27 0.81 1/441 11086
/proc/meminfo: memFree=14901528/32950928 swapFree=60199076/67111528
[pid=11086] ppid=11084 vsize=18884 CPUtime=0.01 cores=0-7
/proc/11086/stat : 11086 (python) S 11084 11086 11001 0 -1 4202496 1249 0 0 0 1 0 0 0 17 0 1 0 668105013 19337216 1104 33554432000 4194304 5683644 140734270847040 18446744073709551615 268272687104 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 0
/proc/11086/statm: 4721 1104 421 364 0 690 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 18884

[startup+0.100655 s]
/proc/loadavg: 6.57 2.27 0.81 1/441 11086
/proc/meminfo: memFree=14901528/32950928 swapFree=60199076/67111528
[pid=11086] ppid=11084 vsize=89768 CPUtime=0.05 cores=0-7
/proc/11086/stat : 11086 (python) R 11084 11086 11001 0 -1 4202496 2641 0 0 0 4 1 0 0 17 0 1 0 668105013 91922432 2357 33554432000 4194304 5683644 140734270847040 18446744073709551615 268267726737 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/11086/statm: 22442 2359 721 364 0 1622 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 89768

[startup+0.300637 s]
/proc/loadavg: 6.57 2.27 0.81 1/441 11086
/proc/meminfo: memFree=14901528/32950928 swapFree=60199076/67111528
[pid=11086] ppid=11084 vsize=135748 CPUtime=0.18 cores=0-7
/proc/11086/stat : 11086 (python) S 11084 11086 11001 0 -1 4202496 5964 937 1 0 14 4 0 0 18 0 1 0 668105013 139005952 4151 33554432000 4194304 5683644 140734270847040 18446744073709551615 268272685061 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 0
/proc/11086/statm: 33937 4151 1032 364 0 3123 0
Current children cumulated CPU time (s) 0.18
Current children cumulated vsize (KiB) 135748

[startup+0.700547 s]
/proc/loadavg: 6.57 2.27 0.81 1/441 11086
/proc/meminfo: memFree=14901528/32950928 swapFree=60199076/67111528
[pid=11086] ppid=11084 vsize=283524 CPUtime=0.44 cores=0-7
/proc/11086/stat : 11086 (python) R 11084 11086 11001 0 -1 4202496 10531 937 1 0 34 10 0 0 17 0 1 0 668105013 290328576 8212 33554432000 4194304 5683644 140734270847040 18446744073709551615 268272375507 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/11086/statm: 70881 8213 1761 364 0 6805 0
Current children cumulated CPU time (s) 0.44
Current children cumulated vsize (KiB) 283524

[startup+1.50033 s]
/proc/loadavg: 6.57 2.27 0.81 2/442 11091
/proc/meminfo: memFree=14856880/32950928 swapFree=60199076/67111528
[pid=11086] ppid=11084 vsize=317984 CPUtime=1.23 cores=0-7
/proc/11086/stat : 11086 (python) R 11084 11086 11001 0 -1 4202496 23799 937 1 0 112 11 0 0 18 0 1 0 668105013 325615616 16881 33554432000 4194304 5683644 140734270847040 18446744073709551615 48008164322151 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/11086/statm: 79496 16881 1826 364 0 15420 0
Current children cumulated CPU time (s) 1.23
Current children cumulated vsize (KiB) 317984

[startup+3.10104 s]
/proc/loadavg: 6.57 2.27 0.81 2/442 11091
/proc/meminfo: memFree=14839024/32950928 swapFree=60199076/67111528
[pid=11086] ppid=11084 vsize=317984 CPUtime=2.83 cores=0-7
/proc/11086/stat : 11086 (python) R 11084 11086 11001 0 -1 4202496 23799 937 1 0 272 11 0 0 21 0 1 0 668105013 325615616 16881 33554432000 4194304 5683644 140734270847040 18446744073709551615 4603233 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/11086/statm: 79496 16881 1826 364 0 15420 0
Current children cumulated CPU time (s) 2.83
Current children cumulated vsize (KiB) 317984

[startup+6.30051 s]
/proc/loadavg: 6.12 2.25 0.81 2/442 11091
/proc/meminfo: memFree=14839732/32950928 swapFree=60199076/67111528
[pid=11086] ppid=11084 vsize=317984 CPUtime=6.03 cores=0-7
/proc/11086/stat : 11086 (python) R 11084 11086 11001 0 -1 4202496 23800 937 1 0 592 11 0 0 25 0 1 0 668105013 325615616 16882 33554432000 4194304 5683644 140734270847040 18446744073709551615 48008164411841 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/11086/statm: 79496 16882 1827 364 0 15420 0
Current children cumulated CPU time (s) 6.03
Current children cumulated vsize (KiB) 317984

[startup+12.7013 s]
/proc/loadavg: 5.71 2.23 0.81 9/466 11115
/proc/meminfo: memFree=14545808/32950928 swapFree=60199076/67111528
[pid=11086] ppid=11084 vsize=462452 CPUtime=8.54 cores=0-7
/proc/11086/stat : 11086 (python) S 11084 11086 11001 0 -1 4202496 29864 937 1 0 840 14 0 0 25 0 9 0 668105013 473550848 16348 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284515787 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/11086/statm: 115613 16348 1889 364 0 51510 0
[pid=11092] ppid=11086 vsize=314880 CPUtime=0 cores=0-7
/proc/11092/stat : 11092 (python) S 11086 11086 11001 0 -1 4202560 740 0 0 0 0 0 0 0 15 0 1 0 668105871 322437120 14626 33554432000 4194304 5683644 140734270847040 18446744073709551615 268272709135 0 0 16781312 514 16926805592046010627 0 0 17 1 0 0 0
/proc/11092/statm: 78720 14626 353 364 0 14638 0
[pid=11094] ppid=11092 vsize=276236 CPUtime=4.1 cores=0-7
/proc/11094/stat : 11094 (hybridGM3) R 11092 11094 11094 0 -1 4202496 8561 0 0 0 410 0 0 0 25 0 1 0 668105872 282865664 8297 33554432000 134512640 135204385 4286671824 18446744073709551615 134537363 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/11094/statm: 69059 8297 64 169 0 68889 0
[pid=11095] ppid=11086 vsize=325136 CPUtime=0 cores=0-7
/proc/11095/stat : 11095 (python) S 11086 11086 11001 0 -1 4202560 744 0 0 0 0 0 0 0 15 0 1 0 668105874 332939264 14624 33554432000 4194304 5683644 140734270847040 18446744073709551615 268272709190 0 0 16781312 514 5353759614447878401 0 0 17 0 0 0 0
/proc/11095/statm: 81284 14624 347 364 0 17199 0
[pid=11097] ppid=11095 vsize=276244 CPUtime=4.07 cores=0-7
/proc/11097/stat : 11097 (hybridGM3) R 11095 11097 11097 0 -1 4202496 8565 0 0 0 407 0 0 0 25 0 1 0 668105875 282873856 8300 33554432000 134512640 135204385 4291455312 18446744073709551615 134537401 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/11097/statm: 69061 8300 64 169 0 68891 0
[pid=11098] ppid=11086 vsize=335392 CPUtime=0 cores=0-7
/proc/11098/stat : 11098 (python) S 11086 11086 11001 0 -1 4202560 748 0 0 0 0 0 0 0 15 0 1 0 668105878 343441408 14628 33554432000 4194304 5683644 140734270847040 18446744073709551615 268272709190 0 0 16781312 514 17712446232510562562 0 0 17 1 0 0 0
/proc/11098/statm: 83848 14628 347 364 0 19760 0
[pid=11100] ppid=11098 vsize=276240 CPUtime=4.04 cores=0-7
/proc/11100/stat : 11100 (hybridGM3) R 11098 11100 11100 0 -1 4202496 8561 0 0 0 403 1 0 0 25 0 1 0 668105878 282869760 8299 33554432000 134512640 135204385 4293776912 18446744073709551615 134536929 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/11100/statm: 69060 8299 64 169 0 68890 0
[pid=11101] ppid=11086 vsize=345648 CPUtime=0 cores=0-7
/proc/11101/stat : 11101 (python) S 11086 11086 11001 0 -1 4202560 762 0 0 0 0 0 0 0 15 0 1 0 668105881 353943552 14632 33554432000 4194304 5683644 140734270847040 18446744073709551615 268272709190 0 0 16781312 514 17712459426650095876 0 0 17 1 0 0 0
/proc/11101/statm: 86412 14632 347 364 0 22321 0
[pid=11103] ppid=11101 vsize=276236 CPUtime=4.01 cores=0-7
/proc/11103/stat : 11103 (hybridGM3) R 11101 11103 11103 0 -1 4202496 8562 0 0 0 400 1 0 0 25 0 1 0 668105881 282865664 8301 33554432000 134512640 135204385 4289628960 18446744073709551615 134545851 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/11103/statm: 69059 8301 64 169 0 68889 0
[pid=11104] ppid=11086 vsize=421440 CPUtime=0 cores=0-7
/proc/11104/stat : 11104 (python) S 11086 11086 11001 0 -1 4202560 728 0 0 0 0 0 0 0 15 0 1 0 668105884 431554560 14810 33554432000 4194304 5683644 140734270847040 18446744073709551615 268272709190 0 0 16781312 514 9121447313254875396 0 0 17 6 0 0 0
/proc/11104/statm: 105360 14810 348 364 0 41266 0
[pid=11106] ppid=11104 vsize=276240 CPUtime=3.96 cores=0-7
/proc/11106/stat : 11106 (hybridGM3) R 11104 11106 11106 0 -1 4202496 8552 0 0 0 396 0 0 0 25 0 1 0 668105884 282869760 8296 33554432000 134512640 135204385 4292620368 18446744073709551615 134536993 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/11106/statm: 69060 8296 64 169 0 68890 0
[pid=11107] ppid=11086 vsize=431696 CPUtime=0 cores=0-7
/proc/11107/stat : 11107 (python) S 11086 11086 11001 0 -1 4202560 733 0 0 0 0 0 0 0 15 0 1 0 668105887 442056704 14778 33554432000 4194304 5683644 140734270847040 18446744073709551615 268272709190 0 0 16781312 514 4438556921812713732 0 0 17 0 0 0 0
/proc/11107/statm: 107924 14778 348 364 0 43827 0
[pid=11109] ppid=11107 vsize=276244 CPUtime=3.89 cores=0-7
/proc/11109/stat : 11109 (hybridGM3) R 11107 11109 11109 0 -1 4202496 8555 0 0 0 388 1 0 0 25 0 1 0 668105888 282873856 8298 33554432000 134512640 135204385 4286708256 18446744073709551615 134537294 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/11109/statm: 69061 8298 64 169 0 68891 0
[pid=11110] ppid=11086 vsize=441952 CPUtime=0 cores=0-7
/proc/11110/stat : 11110 (python) S 11086 11086 11001 0 -1 4202560 747 0 0 0 0 0 0 0 15 0 1 0 668105891 452558848 14783 33554432000 4194304 5683644 140734270847040 18446744073709551615 268272709190 0 0 16781312 514 4438552523766202626 0 0 17 6 0 0 0
/proc/11110/statm: 110488 14783 348 364 0 46388 0
[pid=11112] ppid=11110 vsize=276240 CPUtime=3.9 cores=0-7
/proc/11112/stat : 11112 (hybridGM3) R 11110 11112 11112 0 -1 4202496 8558 0 0 0 389 1 0 0 25 0 1 0 668105891 282869760 8298 33554432000 134512640 135204385 4292956640 18446744073709551615 134536952 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/11112/statm: 69060 8298 64 169 0 68890 0
[pid=11113] ppid=11086 vsize=452208 CPUtime=0 cores=0-7
/proc/11113/stat : 11113 (python) S 11086 11086 11001 0 -1 4202560 780 0 0 0 0 0 0 0 15 0 1 0 668105894 463060992 14809 33554432000 4194304 5683644 140734270847040 18446744073709551615 268272709190 0 0 16781312 514 10080925140117324034 0 0 17 6 0 0 0
/proc/11113/statm: 113052 14809 354 364 0 48949 0
[pid=11115] ppid=11113 vsize=276240 CPUtime=3.85 cores=0-7
/proc/11115/stat : 11115 (hybridGM3) R 11113 11115 11115 0 -1 4202496 8568 0 0 0 385 0 0 0 25 0 1 0 668105895 282869760 8303 33554432000 134512640 135204385 4291483104 18446744073709551615 134536993 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/11115/statm: 69060 8303 64 169 0 68890 0
[pid=11086/tid=11093] ppid=11084 vsize=462452 CPUtime=0 cores=0-7
/proc/11086/task/11093/stat : 11093 (python) S 11084 11086 11001 0 -1 4202560 26 937 0 0 0 0 0 0 25 0 9 0 668105872 473550848 16348 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 2 0 0 0
[pid=11086/tid=11096] ppid=11084 vsize=462452 CPUtime=0 cores=0-7
/proc/11086/task/11096/stat : 11096 (python) S 11084 11086 11001 0 -1 4202560 23 937 0 0 0 0 0 0 25 0 9 0 668105875 473550848 16348 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 2 0 0 0
[pid=11086/tid=11099] ppid=11084 vsize=462452 CPUtime=0 cores=0-7
/proc/11086/task/11099/stat : 11099 (python) S 11084 11086 11001 0 -1 4202560 24 937 0 0 0 0 0 0 19 0 9 0 668105878 473550848 16348 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 4 0 0 0
[pid=11086/tid=11102] ppid=11084 vsize=462452 CPUtime=0 cores=0-7
/proc/11086/task/11102/stat : 11102 (python) S 11084 11086 11001 0 -1 4202560 21 937 0 0 0 0 0 0 18 0 9 0 668105881 473550848 16348 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 5 0 0 0
[pid=11086/tid=11105] ppid=11084 vsize=462452 CPUtime=0 cores=0-7
/proc/11086/task/11105/stat : 11105 (python) S 11084 11086 11001 0 -1 4202560 91 937 0 0 0 0 0 0 22 0 9 0 668105884 473550848 16348 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 6 0 0 0
[pid=11086/tid=11108] ppid=11084 vsize=462452 CPUtime=0 cores=0-7
/proc/11086/task/11108/stat : 11108 (python) S 11084 11086 11001 0 -1 4202560 33 937 0 0 0 0 0 0 23 0 9 0 668105888 473550848 16348 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 0 0 0 0
[pid=11086/tid=11111] ppid=11084 vsize=462452 CPUtime=0 cores=0-7
/proc/11086/task/11111/stat : 11111 (python) S 11084 11086 11001 0 -1 4202560 34 937 0 0 0 0 0 0 25 0 9 0 668105891 473550848 16348 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 0 0 0 0
[pid=11086/tid=11114] ppid=11084 vsize=462452 CPUtime=0 cores=0-7
/proc/11086/task/11114/stat : 11114 (python) S 11084 11086 11001 0 -1 4202560 35 937 0 0 0 0 0 0 25 0 9 0 668105894 473550848 16348 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 40.36
Current children cumulated vsize (KiB) 5740724

[startup+25.5011 s]
/proc/loadavg: 6.22 2.51 0.93 9/466 11116
/proc/meminfo: memFree=14546240/32950928 swapFree=60199076/67111528
[pid=11086] ppid=11084 vsize=462452 CPUtime=8.54 cores=0-7
/proc/11086/stat : 11086 (python) S 11084 11086 11001 0 -1 4202496 29864 937 1 0 840 14 0 0 25 0 9 0 668105013 473550848 16348 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284515787 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/11086/statm: 115613 16348 1889 364 0 51510 0
[pid=11092] ppid=11086 vsize=314880 CPUtime=0.05 cores=0-7
/proc/11092/stat : 11092 (python) S 11086 11086 11001 0 -1 4202560 908 0 0 0 3 2 0 0 15 0 1 0 668105871 322437120 14649 33554432000 4194304 5683644 140734270847040 18446744073709551615 268272709135 0 0 16781312 514 14017735315167346947 0 0 17 1 0 0 0
/proc/11092/statm: 78720 14649 376 364 0 14638 0
[pid=11094] ppid=11092 vsize=276236 CPUtime=16.9 cores=0-7
/proc/11094/stat : 11094 (hybridGM3) R 11092 11094 11094 0 -1 4202496 8561 0 0 0 1690 0 0 0 25 0 1 0 668105872 282865664 8297 33554432000 134512640 135204385 4286671824 18446744073709551615 134545851 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/11094/statm: 69059 8297 64 169 0 68889 0
[pid=11095] ppid=11086 vsize=325136 CPUtime=0.05 cores=0-7
/proc/11095/stat : 11095 (python) S 11086 11086 11001 0 -1 4202560 911 0 0 0 3 2 0 0 15 0 1 0 668105874 332939264 14647 33554432000 4194304 5683644 140734270847040 18446744073709551615 268272709190 0 0 16781312 514 14017735315167346945 0 0 17 0 0 0 0
/proc/11095/statm: 81284 14647 370 364 0 17199 0
[pid=11097] ppid=11095 vsize=276244 CPUtime=16.87 cores=0-7
/proc/11097/stat : 11097 (hybridGM3) R 11095 11097 11097 0 -1 4202496 8565 0 0 0 1687 0 0 0 25 0 1 0 668105875 282873856 8300 33554432000 134512640 135204385 4291455312 18446744073709551615 134537613 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/11097/statm: 69061 8300 64 169 0 68891 0
[pid=11098] ppid=11086 vsize=335392 CPUtime=0.06 cores=0-7
/proc/11098/stat : 11098 (python) S 11086 11086 11001 0 -1 4202560 932 0 0 0 3 3 0 0 15 0 1 0 668105878 343441408 14651 33554432000 4194304 5683644 140734270847040 18446744073709551615 268272709190 0 0 16781312 514 14017735315167346946 0 0 17 1 0 0 0
/proc/11098/statm: 83848 14651 370 364 0 19760 0
[pid=11100] ppid=11098 vsize=276240 CPUtime=16.84 cores=0-7
/proc/11100/stat : 11100 (hybridGM3) R 11098 11100 11100 0 -1 4202496 8561 0 0 0 1683 1 0 0 25 0 1 0 668105878 282869760 8299 33554432000 134512640 135204385 4293776912 18446744073709551615 134536993 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/11100/statm: 69060 8299 64 169 0 68890 0
[pid=11101] ppid=11086 vsize=345648 CPUtime=0.05 cores=0-7
/proc/11101/stat : 11101 (python) S 11086 11086 11001 0 -1 4202560 960 0 0 0 2 3 0 0 15 0 1 0 668105881 353943552 14655 33554432000 4194304 5683644 140734270847040 18446744073709551615 268272709190 0 0 16781312 514 14017735315167346948 0 0 17 1 0 0 0
/proc/11101/statm: 86412 14655 370 364 0 22321 0
[pid=11103] ppid=11101 vsize=276236 CPUtime=16.81 cores=0-7
/proc/11103/stat : 11103 (hybridGM3) R 11101 11103 11103 0 -1 4202496 8562 0 0 0 1680 1 0 0 25 0 1 0 668105881 282865664 8301 33554432000 134512640 135204385 4289628960 18446744073709551615 134545851 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/11103/statm: 69059 8301 64 169 0 68889 0
[pid=11104] ppid=11086 vsize=421440 CPUtime=0.05 cores=0-7
/proc/11104/stat : 11104 (python) S 11086 11086 11001 0 -1 4202560 806 0 0 0 3 2 0 0 15 0 1 0 668105884 431554560 14833 33554432000 4194304 5683644 140734270847040 18446744073709551615 268272709190 0 0 16781312 514 14017735315167346948 0 0 17 6 0 0 0
/proc/11104/statm: 105360 14833 371 364 0 41266 0
[pid=11106] ppid=11104 vsize=276240 CPUtime=16.57 cores=0-7
/proc/11106/stat : 11106 (hybridGM3) R 11104 11106 11106 0 -1 4202496 8552 0 0 0 1657 0 0 0 25 0 1 0 668105884 282869760 8296 33554432000 134512640 135204385 4292620368 18446744073709551615 134537272 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/11106/statm: 69060 8296 64 169 0 68890 0

################
# More data... #
################

[pid=11204/tid=11214] ppid=11086 vsize=606048 CPUtime=0 cores=0-7
/proc/11204/task/11214/stat : 11214 (python) S 11086 11086 11001 0 -1 4202560 80 0 0 0 0 0 0 0 19 0 2 0 668175193 620593152 15181 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 514 18446604465147352320 0 0 -1 4 0 0 0
[pid=11207] ppid=11086 vsize=616304 CPUtime=0.4 cores=0-7
/proc/11207/stat : 11207 (python) S 11086 11086 11001 0 -1 4202560 1195 0 0 0 21 19 0 0 19 0 2 0 668165547 631095296 15231 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/11207/statm: 154076 15231 406 364 0 89925 0
[pid=11209] ppid=11207 vsize=467024 CPUtime=101.77 cores=0-7
/proc/11209/stat : 11209 (TNM) T 11207 11209 11209 0 -1 4202496 6840 0 0 0 10176 1 0 0 25 0 1 0 668165548 478232576 6557 33554432000 134512640 135008748 4292705888 18446744073709551615 134529123 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/11209/statm: 116756 6557 46 122 0 115656 0
[pid=11207/tid=11219] ppid=11086 vsize=616304 CPUtime=0 cores=0-7
/proc/11207/task/11219/stat : 11219 (python) S 11086 11086 11001 0 -1 4202560 82 0 0 0 0 0 0 0 19 0 2 0 668176187 631095296 15231 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 514 18446604470214800640 0 0 -1 7 0 0 0
[pid=11215] ppid=11086 vsize=626560 CPUtime=0.74 cores=0-7
/proc/11215/stat : 11215 (python) S 11086 11086 11001 0 -1 4202560 1194 0 0 0 38 36 0 0 16 0 2 0 668175202 641597440 15143 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/11215/statm: 156640 15143 400 364 0 92486 0
[pid=11217] ppid=11215 vsize=276240 CPUtime=197.09 cores=0-7
/proc/11217/stat : 11217 (hybridGM3) T 11215 11217 11217 0 -1 4202496 8598 0 0 0 19708 1 0 0 25 0 1 0 668175203 282869760 8308 33554432000 134512640 135204385 4290708512 18446744073709551615 134547435 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/11217/statm: 69060 8308 64 169 0 68890 0
[pid=11215/tid=11236] ppid=11086 vsize=626560 CPUtime=0 cores=0-7
/proc/11215/task/11236/stat : 11236 (python) S 11086 11086 11001 0 -1 4202560 88 0 0 0 0 0 0 0 16 0 2 0 668194919 641597440 15143 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 514 18446604467168481728 0 0 -1 2 0 0 0
[pid=11220] ppid=11086 vsize=636816 CPUtime=0.76 cores=0-7
/proc/11220/stat : 11220 (python) S 11086 11086 11001 0 -1 4202560 1188 0 0 0 39 37 0 0 19 0 2 0 668176193 652099584 15249 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/11220/statm: 159204 15249 401 364 0 95047 0
[pid=11222] ppid=11220 vsize=434080 CPUtime=197.69 cores=0-7
/proc/11222/stat : 11222 (adaptg2wsat2009) T 11220 11222 11222 0 -1 4202496 7105 0 0 0 19768 1 0 0 25 0 1 0 668176194 444497920 6819 33554432000 134512640 135107411 4293369184 18446744073709551615 134535111 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/11222/statm: 108520 6819 59 146 0 107396 0
[pid=11220/tid=11240] ppid=11086 vsize=636816 CPUtime=0 cores=0-7
/proc/11220/task/11240/stat : 11240 (python) S 11086 11086 11001 0 -1 4202560 81 0 0 0 0 0 0 0 19 0 2 0 668196476 652099584 15249 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 514 18446604469405461376 0 0 -1 1 0 0 0
[pid=11226] ppid=11086 vsize=647072 CPUtime=0.75 cores=0-7
/proc/11226/stat : 11226 (python) S 11086 11086 11001 0 -1 4202560 1193 0 0 0 38 37 0 0 16 0 2 0 668184449 662601728 15205 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/11226/statm: 161768 15205 400 364 0 97608 0
[pid=11228] ppid=11226 vsize=276240 CPUtime=197.18 cores=0-7
/proc/11228/stat : 11228 (hybridGM3) T 11226 11228 11228 0 -1 4202496 8585 0 0 0 19716 2 0 0 25 0 1 0 668184450 282869760 8303 33554432000 134512640 135204385 4290874192 18446744073709551615 134536993 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/11228/statm: 69060 8303 64 169 0 68890 0
[pid=11226/tid=11246] ppid=11086 vsize=647072 CPUtime=0 cores=0-7
/proc/11226/task/11246/stat : 11246 (python) S 11086 11086 11001 0 -1 4202560 82 0 0 0 0 0 0 0 16 0 2 0 668204168 662601728 15205 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 514 18446604471101842752 0 0 -1 1 0 0 0
[pid=11231] ppid=11086 vsize=657328 CPUtime=0.78 cores=0-7
/proc/11231/stat : 11231 (python) S 11086 11086 11001 0 -1 4202560 1212 0 0 0 39 39 0 0 19 0 2 0 668185982 673103872 15162 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/11231/statm: 164332 15162 401 364 0 100169 0
[pid=11233] ppid=11231 vsize=434080 CPUtime=195.21 cores=0-7
/proc/11233/stat : 11233 (adaptg2wsat2009) T 11231 11233 11233 0 -1 4202496 7105 0 0 0 19517 4 0 0 25 0 1 0 668185982 444497920 6819 33554432000 134512640 135107411 4288507584 18446744073709551615 134531600 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/11233/statm: 108520 6819 59 146 0 107396 0
[pid=11231/tid=11248] ppid=11086 vsize=657328 CPUtime=0 cores=0-7
/proc/11231/task/11248/stat : 11248 (python) S 11086 11086 11001 0 -1 4202560 87 0 0 0 0 0 0 0 19 0 2 0 668206159 673103872 15162 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 514 18446604459609777216 0 0 -1 1 0 0 0
[pid=11237] ppid=11086 vsize=667584 CPUtime=0.78 cores=0-7
/proc/11237/stat : 11237 (python) S 11086 11086 11001 0 -1 4202560 1255 0 0 0 34 44 0 0 15 0 2 0 668194926 683606016 15285 33554432000 4194304 5683644 140734270847040 18446744073709551615 268272709190 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/11237/statm: 166896 15285 406 364 0 102730 0
[pid=11239] ppid=11237 vsize=276240 CPUtime=201.48 cores=0-7
/proc/11239/stat : 11239 (hybridGM3) R 11237 11239 11239 0 -1 4202496 8600 0 0 0 20147 1 0 0 25 0 1 0 668194926 282869760 8303 33554432000 134512640 135204385 4293900736 18446744073709551615 134537401 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/11239/statm: 69060 8303 64 169 0 68890 0
[pid=11237/tid=11255] ppid=11086 vsize=667584 CPUtime=0 cores=0-7
/proc/11237/task/11255/stat : 11255 (python) S 11086 11086 11001 0 -1 4202560 89 0 0 0 0 0 0 0 17 0 2 0 668214646 683606016 15285 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 514 18446604464025368640 0 0 -1 4 0 0 0
[pid=11241] ppid=11086 vsize=677840 CPUtime=0.38 cores=0-7
/proc/11241/stat : 11241 (python) S 11086 11086 11001 0 -1 4202560 1257 0 0 0 21 17 0 0 19 0 2 0 668196483 694108160 15294 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/11241/statm: 169460 15294 407 364 0 105291 0
[pid=11243] ppid=11241 vsize=262380 CPUtime=98.92 cores=0-7
/proc/11243/stat : 11243 (FH) T 11241 11243 11243 0 -1 4202496 7016 0 0 0 9891 1 0 0 25 0 1 0 668196483 268677120 6725 33554432000 134512640 134936988 4289819488 18446744073709551615 134546861 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/11243/statm: 65595 6725 41 104 0 65099 0
[pid=11241/tid=11249] ppid=11086 vsize=677840 CPUtime=0 cores=0-7
/proc/11241/task/11249/stat : 11249 (python) S 11086 11086 11001 0 -1 4202560 90 0 0 0 0 0 0 0 19 0 2 0 668206423 694108160 15294 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 514 18446604460832049344 0 0 -1 1 0 0 0
[pid=11250] ppid=11086 vsize=688096 CPUtime=0.39 cores=0-7
/proc/11250/stat : 11250 (python) S 11086 11086 11001 0 -1 4202560 1268 0 0 0 19 20 0 0 19 0 2 0 668206429 704610304 15185 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/11250/statm: 172024 15185 407 364 0 107852 0
[pid=11252] ppid=11250 vsize=467024 CPUtime=99.58 cores=0-7
/proc/11252/stat : 11252 (TNM) T 11250 11252 11252 0 -1 4202496 6849 0 0 0 9955 3 0 0 25 0 1 0 668206430 478232576 6559 33554432000 134512640 135008748 4287510112 18446744073709551615 134527693 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/11252/statm: 116756 6559 46 122 0 115656 0
[pid=11250/tid=11260] ppid=11086 vsize=688096 CPUtime=0 cores=0-7
/proc/11250/task/11260/stat : 11260 (python) S 11086 11086 11001 0 -1 4202560 92 0 0 0 0 0 0 0 19 0 2 0 668216468 704610304 15185 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 514 18446604457439761472 0 0 -1 0 0 0 0
[pid=11256] ppid=11086 vsize=698352 CPUtime=0.42 cores=0-7
/proc/11256/stat : 11256 (python) S 11086 11086 11001 0 -1 4202560 1266 0 0 0 19 23 0 0 19 0 2 0 668214652 715112448 15307 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/11256/statm: 174588 15307 407 364 0 110413 0
[pid=11258] ppid=11256 vsize=467024 CPUtime=101.73 cores=0-7
/proc/11258/stat : 11258 (TNM) T 11256 11258 11258 0 -1 4202496 6849 0 0 0 10172 1 0 0 25 0 1 0 668214653 478232576 6559 33554432000 134512640 135008748 4287275360 18446744073709551615 134527729 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/11258/statm: 116756 6559 46 122 0 115656 0
[pid=11256/tid=11266] ppid=11086 vsize=698352 CPUtime=0 cores=0-7
/proc/11256/task/11266/stat : 11266 (python) S 11086 11086 11001 0 -1 4202560 87 0 0 0 0 0 0 0 19 0 2 0 668225293 715112448 15307 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 514 18446604457396920576 0 0 -1 0 0 0 0
[pid=11261] ppid=11086 vsize=708608 CPUtime=0.34 cores=0-7
/proc/11261/stat : 11261 (python) S 11086 11086 11001 0 -1 4202560 1075 0 0 0 19 15 0 0 18 0 1 0 668216475 725614592 15231 33554432000 4194304 5683644 140734270847040 18446744073709551615 268272709190 0 0 16781312 514 14017735315167346950 0 0 17 0 0 0 0
/proc/11261/statm: 177152 15231 378 364 0 112974 0
[pid=11263] ppid=11261 vsize=467024 CPUtime=86.64 cores=0-7
/proc/11263/stat : 11263 (TNM) R 11261 11263 11263 0 -1 4202496 6849 0 0 0 8663 1 0 0 25 0 1 0 668216478 478232576 6558 33554432000 134512640 135008748 4294846384 18446744073709551615 134529176 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/11263/statm: 116756 6558 46 122 0 115656 0
[pid=11267] ppid=11086 vsize=718864 CPUtime=0 cores=0-7
/proc/11267/stat : 11267 (python) S 11086 11086 11001 0 -1 4202560 929 0 0 0 0 0 0 0 19 0 1 0 668225300 736116736 15185 33554432000 4194304 5683644 140734270847040 18446744073709551615 268272709190 0 0 16781312 514 5330803735805296639 0 0 17 1 0 0 0
/proc/11267/statm: 179716 15185 346 364 0 115535 0
[pid=11269] ppid=11267 vsize=423836 CPUtime=0.45 cores=0-7
/proc/11269/stat : 11269 (NCVWr) R 11267 11269 11269 0 -1 4202496 4207 0 0 0 45 0 0 0 21 0 1 0 668225301 434008064 3919 33554432000 134512640 135113558 4293254416 18446744073709551615 134526264 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/11269/statm: 105959 3919 55 147 0 104834 0
[pid=11086/tid=11093] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11093/stat : 11093 (python) S 11084 11086 11001 0 -1 4202560 45 937 0 0 0 0 0 0 18 0 35 0 668105872 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 2 0 0 0
[pid=11086/tid=11096] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11096/stat : 11096 (python) S 11084 11086 11001 0 -1 4202560 34 937 0 0 0 0 0 0 18 0 35 0 668105875 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 2 0 0 0
[pid=11086/tid=11099] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11099/stat : 11099 (python) S 11084 11086 11001 0 -1 4202560 47 937 0 0 0 0 0 0 15 0 35 0 668105878 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 4 0 0 0
[pid=11086/tid=11102] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11102/stat : 11102 (python) S 11084 11086 11001 0 -1 4202560 44 937 0 0 0 0 0 0 15 0 35 0 668105881 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 5 0 0 0
[pid=11086/tid=11105] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11105/stat : 11105 (python) S 11084 11086 11001 0 -1 4202560 108 937 0 0 0 0 0 0 18 0 35 0 668105884 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 3 0 0 0
[pid=11086/tid=11108] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11108/stat : 11108 (python) S 11084 11086 11001 0 -1 4202560 52 937 0 0 0 0 0 0 18 0 35 0 668105888 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 0 0 0 0
[pid=11086/tid=11111] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11111/stat : 11111 (python) S 11084 11086 11001 0 -1 4202560 47 937 0 0 0 0 0 0 18 0 35 0 668105891 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 0 0 0 0
[pid=11086/tid=11114] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11114/stat : 11114 (python) S 11084 11086 11001 0 -1 4202560 52 937 0 0 0 0 0 0 18 0 35 0 668105894 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 6 0 0 0
[pid=11086/tid=11136] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11136/stat : 11136 (python) S 11084 11086 11001 0 -1 4202560 48 937 0 0 0 0 0 0 18 0 35 0 668125490 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 6 0 0 0
[pid=11086/tid=11146] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11146/stat : 11146 (python) S 11084 11086 11001 0 -1 4202560 44 937 0 0 0 0 0 0 18 0 35 0 668135621 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 1 0 0 0
[pid=11086/tid=11149] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11149/stat : 11149 (python) S 11084 11086 11001 0 -1 4202560 43 937 0 0 0 0 0 0 18 0 35 0 668135625 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 3 0 0 0
[pid=11086/tid=11152] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11152/stat : 11152 (python) S 11084 11086 11001 0 -1 4202560 42 937 0 0 0 0 0 0 18 0 35 0 668135640 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 3 0 0 0
[pid=11086/tid=11161] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11161/stat : 11161 (python) S 11084 11086 11001 0 -1 4202560 42 937 0 0 0 0 0 0 20 0 35 0 668145172 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 0 0 0 0
[pid=11086/tid=11164] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11164/stat : 11164 (python) S 11084 11086 11001 0 -1 4202560 38 937 0 0 0 0 0 0 20 0 35 0 668145267 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 7 0 0 0
[pid=11086/tid=11168] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11168/stat : 11168 (python) S 11084 11086 11001 0 -1 4202560 36 937 0 0 0 0 0 0 20 0 35 0 668145665 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 1 0 0 0
[pid=11086/tid=11173] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11173/stat : 11173 (python) S 11084 11086 11001 0 -1 4202560 36 937 0 0 0 0 0 0 19 0 35 0 668154719 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 1 0 0 0
[pid=11086/tid=11176] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11176/stat : 11176 (python) S 11084 11086 11001 0 -1 4202560 36 937 0 0 0 0 0 0 21 0 35 0 668154724 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 4 0 0 0
[pid=11086/tid=11180] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11180/stat : 11180 (python) S 11084 11086 11001 0 -1 4202560 35 937 0 0 0 0 0 0 20 0 35 0 668155114 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 4 0 0 0
[pid=11086/tid=11188] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11188/stat : 11188 (python) S 11084 11086 11001 0 -1 4202560 49 937 0 0 0 0 0 0 18 0 35 0 668156115 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 4 0 0 0
[pid=11086/tid=11195] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11195/stat : 11195 (python) S 11084 11086 11001 0 -1 4202560 36 937 0 0 0 0 0 0 18 0 35 0 668164765 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 5 0 0 0
[pid=11086/tid=11198] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11198/stat : 11198 (python) S 11084 11086 11001 0 -1 4202560 53 937 0 0 0 0 0 0 18 0 35 0 668164771 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 1 0 0 0
[pid=11086/tid=11201] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11201/stat : 11201 (python) S 11084 11086 11001 0 -1 4202560 33 937 0 0 0 0 0 0 20 0 35 0 668164899 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 1 0 0 0
[pid=11086/tid=11205] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11205/stat : 11205 (python) S 11084 11086 11001 0 -1 4202560 33 937 0 0 0 0 0 0 20 0 35 0 668165157 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 1 0 0 0
[pid=11086/tid=11208] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11208/stat : 11208 (python) S 11084 11086 11001 0 -1 4202560 35 937 0 0 0 0 0 0 20 0 35 0 668165548 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 1 0 0 0
[pid=11086/tid=11216] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11216/stat : 11216 (python) S 11084 11086 11001 0 -1 4202560 34 937 0 0 0 0 0 0 20 0 35 0 668175203 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 0 0 0 0
[pid=11086/tid=11221] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11221/stat : 11221 (python) S 11084 11086 11001 0 -1 4202560 35 937 0 0 0 0 0 0 19 0 35 0 668176194 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 1 0 0 0
[pid=11086/tid=11227] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11227/stat : 11227 (python) S 11084 11086 11001 0 -1 4202560 33 937 0 0 0 0 0 0 20 0 35 0 668184449 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 0 0 0 0
[pid=11086/tid=11232] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11232/stat : 11232 (python) S 11084 11086 11001 0 -1 4202560 37 937 0 0 0 0 0 0 20 0 35 0 668185982 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 1 0 0 0
[pid=11086/tid=11238] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11238/stat : 11238 (python) S 11084 11086 11001 0 -1 4202560 40 937 0 0 0 0 0 0 18 0 35 0 668194926 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 1 0 0 0
[pid=11086/tid=11242] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11242/stat : 11242 (python) S 11084 11086 11001 0 -1 4202560 36 937 0 0 0 0 0 0 19 0 35 0 668196483 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 1 0 0 0
[pid=11086/tid=11251] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11251/stat : 11251 (python) S 11084 11086 11001 0 -1 4202560 33 937 0 0 0 0 0 0 19 0 35 0 668206430 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 1 0 0 0
[pid=11086/tid=11257] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11257/stat : 11257 (python) S 11084 11086 11001 0 -1 4202560 36 937 0 0 0 0 0 0 20 0 35 0 668214653 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 0 0 0 0
[pid=11086/tid=11262] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11262/stat : 11262 (python) S 11084 11086 11001 0 -1 4202560 31 937 0 0 0 0 0 0 19 0 35 0 668216475 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 1 0 0 0
[pid=11086/tid=11268] ppid=11084 vsize=729108 CPUtime=0 cores=0-7
/proc/11086/task/11268/stat : 11268 (python) S 11084 11086 11001 0 -1 4202560 32 937 0 0 0 0 0 0 20 0 35 0 668225300 746606592 16785 33554432000 4194304 5683644 140734270847040 18446744073709551615 268284512689 0 0 16781312 2 18446604440845820352 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 9517.73
Current children cumulated vsize (KiB) 31900884

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 11086 and gives
#  childrusage.ru_utime.tv_sec=2201
#  childrusage.ru_utime.tv_usec=440330
#  childrusage.ru_stime.tv_sec=18
#  childrusage.ru_stime.tv_usec=451194
# CPU time returned by wait4() is 2219.89
# while last known CPU time is 9517.73
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1203.53
CPU time (s): 9517.73
CPU user time (s): 9499.13
CPU system time (s): 18.6
CPU usage (%): 790.819
Max. virtual memory (cumulated for all children) (KiB): 31900884

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2201.44
system time used= 18.4512
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 161864
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 16356
involuntary context switches= 12249

runsolver used 12.5091 second user time and 33.027 second system time

The end

Launcher Data 

Begin job on node122 at 2011-05-16 18:57:42
IDJOB=3385313
IDBENCH=82578
IDSOLVER=1870
FILE ID=node122/3385313-1305565062
RUNJOBID= node122-1305564964-11015
PBS_JOBID= 13350128
Free space on /tmp= 73408 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/large/unif-k3-r4.2-v40000-c168000-S471691933-060-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3385313-1305565062/watcher-3385313-1305565062 -o /tmp/evaluation-result-3385313-1305565062/solver-3385313-1305565062 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385313-1305565062.cnf -seed 1978747136 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 91da78a39fd68b6c63602d278464956f
RANDOM SEED=1978747136

node122.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.64
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.97
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      14901932 kB
Buffers:        330172 kB
Cached:        4381420 kB
SwapCached:      50104 kB
Active:       13301836 kB
Inactive:      4469804 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      14901932 kB
SwapTotal:    67111528 kB
SwapFree:     60199076 kB
Dirty:           34928 kB
Writeback:           0 kB
AnonPages:    13021160 kB
Mapped:          28156 kB
Slab:           135816 kB
PageTables:      71772 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 129863556 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73376 MiB
End job on node122 at 2011-05-16 19:17:48