Trace number 3385405

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)? (MO) 9423.43 1192.53

General information on the benchmark

Namerandom/large/
unif-k3-r4.2-v40000-c168000-S386300535-065-UNKNOWN.cnf
MD5SUM2ca6073d25d8d5f2648d590c0edc75c0
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark1217.09
Satisfiable
(Un)Satisfiability was proved
Number of variables40000
Number of clauses168000
Sum of the clauses size504000
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3168000
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.35/0.67	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.35/0.67	c INFO: solving HOME/instance-3385405-1305566268.cnf
0.35/0.67	c INFO: computing task features
0.93/1.25	c INFO: computed variable-clause graph statistics
0.93/1.27	c INFO: computed clause balance statistics
4.04/4.36	c INFO: computed variable balance statistics
6.63/7.00	c INFO: computed small-clause counts
6.73/7.01	c INFO: computed Horn-clause counts
8.93/9.27	c INFO: computed variable graph statistics
8.93/9.27	c INFO: collected features for HOME/instance-3385405-1305566268.cnf in 8.58 s
9.03/9.30	c INFO: running hybridGM3@0 for 97 with 4991 remaining (b = 0.35)
9.03/9.33	c INFO: running hybridGM3@0 for 97 with 4991 remaining (b = 0.21)
9.03/9.36	c INFO: running hybridGM3@0 for 97 with 4991 remaining (b = 0.14)
9.03/9.39	c INFO: running hybridGM3@0 for 97 with 4991 remaining (b = 0.10)
9.13/9.43	c INFO: running hybridGM3@0 for 97 with 4991 remaining (b = 0.07)
9.13/9.46	c INFO: running hybridGM3@0 for 194 with 4991 remaining (b = 0.10)
9.13/9.49	c INFO: running hybridGM3@0 for 194 with 4991 remaining (b = 0.08)
9.22/9.53	c INFO: running hybridGM3@0 for 194 with 4991 remaining (b = 0.06)
811.98/110.09	c INFO: running hybridGM3@0 for 97 with 4890 remaining (b = 0.03)
812.62/110.13	c INFO: running hybridGM3@0 for 97 with 4890 remaining (b = 0.03)
812.62/110.16	c INFO: running hybridGM3@0 for 97 with 4890 remaining (b = 0.03)
812.62/110.19	c INFO: running hybridGM3@0 for 97 with 4890 remaining (b = 0.03)
813.39/110.24	c INFO: running hybridGM3@0 for 97 with 4890 remaining (b = 0.03)
1572.60/205.46	c INFO: running hybridGM3@100 for 97 with 4795 remaining (b = 0.03)
1572.60/205.49	c INFO: running hybridGM3@100 for 97 with 4795 remaining (b = 0.03)
1573.37/205.52	c INFO: running TNM@0 for 97 with 4795 remaining (b = 0.03)
1581.32/206.59	c INFO: running hybridGM3@100 for 97 with 4794 remaining (b = 0.02)
1582.13/206.65	c INFO: running hybridGM3@0 for 97 with 4794 remaining (b = 0.02)
1582.13/206.69	c INFO: running hybridGM3@0 for 97 with 4794 remaining (b = 0.02)
1621.16/211.58	c INFO: running hybridGM3@100 for 97 with 4789 remaining (b = 0.02)
1621.97/211.63	c INFO: running hybridGM3@98 for 97 with 4789 remaining (b = 0.02)
2333.24/300.83	c INFO: running hybridGM3@95 for 97 with 4699 remaining (b = 0.02)
2342.80/302.06	c INFO: running hybridGM3@98 for 97 with 4698 remaining (b = 0.02)
2373.89/305.93	c INFO: running hybridGM3@0 for 194 with 4694 remaining (b = 0.04)
2381.13/306.89	c INFO: running hybridGM3@0 for 194 with 4693 remaining (b = 0.03)
2381.94/306.99	c INFO: running TNM@0 for 97 with 4693 remaining (b = 0.02)
2382.48/307.03	c INFO: running hybridGM3@297 for 97 with 4693 remaining (b = 0.02)
2390.62/308.04	c INFO: running TNM@0 for 97 with 4692 remaining (b = 0.01)
2396.80/308.98	c INFO: running TNM@0 for 97 with 4691 remaining (b = 0.01)
3109.71/398.24	c INFO: running hybridGM3@189 for 97 with 4602 remaining (b = 0.01)
3143.06/402.42	c INFO: running hybridGM3@399 for 97 with 4598 remaining (b = 0.01)
3151.03/403.46	c INFO: running TNM@0 for 97 with 4597 remaining (b = 0.01)
3182.89/407.42	c INFO: running TNM@0 for 97 with 4593 remaining (b = 0.01)
3183.59/407.51	c INFO: running TNM@0 for 97 with 4593 remaining (b = 0.01)
3198.75/409.41	c INFO: running TNM@194 for 1843 with 4591 remaining (b = 0.19)
3869.33/493.64	c INFO: running hybridGM3@0 for 194 with 4507 remaining (b = 0.02)
3944.95/503.19	c INFO: running hybridGM3@100 for 97 with 4497 remaining (b = 0.01)
3950.67/503.85	c INFO: running hybridGM3@0 for 194 with 4496 remaining (b = 0.02)
3951.30/503.93	c INFO: running hybridGM3@0 for 194 with 4496 remaining (b = 0.02)
3952.52/504.13	c INFO: running hybridGM3@99 for 97 with 4496 remaining (b = 0.01)
3975.18/506.97	c INFO: running adaptg2wsat2009++@0 for 194 with 4493 remaining (b = 0.02)
3982.32/507.86	c INFO: running TNM@0 for 97 with 4492 remaining (b = 0.01)
4704.46/598.60	c INFO: running hybridGM3@0 for 194 with 4402 remaining (b = 0.01)
4760.13/605.56	c INFO: running TNM@0 for 97 with 4395 remaining (b = 0.01)
4781.70/608.31	c INFO: running TNM@0 for 97 with 4392 remaining (b = 0.01)
5439.28/690.92	c INFO: running TNM@0 for 97 with 4309 remaining (b = 0.01)
5520.40/701.17	c INFO: running hybridGM3@100 for 97 with 4299 remaining (b = 0.01)
5521.15/701.23	c INFO: running adaptg2wsat2009++@0 for 194 with 4299 remaining (b = 0.01)
5541.83/703.84	c INFO: running hybridGM3@196 for 291 with 4296 remaining (b = 0.02)
5551.45/705.06	c INFO: running TNM@0 for 97 with 4295 remaining (b = 0.01)
5629.27/714.81	c INFO: running TNM@0 for 97 with 4285 remaining (b = 0.01)
6237.63/791.39	c INFO: running TNM@0 for 97 with 4209 remaining (b = 0.01)
6274.20/795.93	c INFO: running hybridGM3@100 for 97 with 4204 remaining (b = 0.01)
6327.45/802.63	c INFO: running hybridGM3@296 for 194 with 4198 remaining (b = 0.01)
6342.56/804.53	c INFO: running TNM@197 for 1843 with 4196 remaining (b = 0.10)
6427.58/815.28	c INFO: running adaptg2wsat2009++@99 for 97 with 4185 remaining (b = 0.01)
7028.38/890.91	c INFO: running hybridGM3@194 for 291 with 4109 remaining (b = 0.02)
7080.02/897.38	c INFO: running adaptg2wsat2009++@0 for 194 with 4103 remaining (b = 0.01)
7085.56/898.07	c INFO: running hybridGM3@194 for 291 with 4102 remaining (b = 0.02)
7202.30/912.73	c INFO: running hybridGM3@195 for 291 with 4087 remaining (b = 0.01)
7895.76/1000.05	c INFO: running hybridGM3@295 for 194 with 4000 remaining (b = 0.01)
7898.92/1000.45	c INFO: running hybridGM3@395 for 97 with 4000 remaining (b = 0.01)
8656.74/1095.88	c INFO: running hybridGM3@0 for 194 with 3904 remaining (b = 0.01)
8739.33/1106.27	c INFO: running FH@0 for 97 with 3894 remaining (b = 0.00)
9334.25/1181.13	c INFO: running hybridGM3@291 for 194 with 3819 remaining (b = 0.01)
9390.63/1188.26	c INFO: running hybridGM3@201 for 291 with 3812 remaining (b = 0.01)
9419.25/1191.85	c INFO: running adaptg2wsat2009++@0 for 194 with 3808 remaining (b = 0.01)
9423.43/1192.42	c INFO: running hybridGM3@196 for 291 with 3808 remaining (b = 0.01)
9423.43/1192.42	Traceback (most recent call last):
9423.43/1192.42	  File "HOME/solve", line 33, in <module>
9423.43/1192.42	    plac.call(main)
9423.43/1192.42	  File "/home/evaluation/evaluation/tools64/lib/python2.7/site-packages/plac-0.8.1-py2.7.egg/plac_core.py", line 311, in call
9423.43/1192.42	    cmd, result = parser_from(obj).consume(arglist)
9423.43/1192.42	  File "/home/evaluation/evaluation/tools64/lib/python2.7/site-packages/plac-0.8.1-py2.7.egg/plac_core.py", line 192, in consume
9423.43/1192.42	    return cmd, self.func(*(args + varargs + extraopts), **kwargs)
9423.43/1192.42	  File "HOME/borg/build/lib/borg/tools/solve.py", line 84, in main
9423.43/1192.42	    (_, answer, _) = solver(input_path, budget / cores, cores)
9423.43/1192.42	  File "HOME/borg/build/lib/borg/bilevel.py", line 202, in __call__
9423.43/1192.42	    solver.go(max_cost * borg.defaults.machine_speed)
9423.43/1192.42	  File "HOME/borg/build/lib/borg/solvers.py", line 179, in go
9423.43/1192.42	    self._process.start()
9423.43/1192.42	  File "/home/evaluation/evaluation/tools64/lib/python2.7/multiprocessing/process.py", line 94, in start
9423.43/1192.42	    assert self._popen is None, 'cannot start a process twice'
9423.43/1192.42	AssertionError: cannot start a process twice

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3385405-1305566268/watcher-3385405-1305566268 -o /tmp/evaluation-result-3385405-1305566268/solver-3385405-1305566268 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385405-1305566268.cnf -seed 616990486 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.55 8.51 6.35 1/467 11298
/proc/meminfo: memFree=14121436/32950928 swapFree=60199076/67111528
[pid=11298] ppid=11296 vsize=15780 CPUtime=0 cores=0-7
/proc/11298/stat : 11298 (python) R 11296 11298 11001 0 -1 4202496 897 0 0 0 0 0 0 0 20 0 1 0 668225654 16158720 779 33554432000 4194304 5683644 140736708680848 18446744073709551615 4540219 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/11298/statm: 3944 778 343 364 0 431 0

[startup+0.0464019 s]
/proc/loadavg: 8.55 8.51 6.35 1/467 11298
/proc/meminfo: memFree=14121436/32950928 swapFree=60199076/67111528
[pid=11298] ppid=11296 vsize=18884 CPUtime=0.02 cores=0-7
/proc/11298/stat : 11298 (python) R 11296 11298 11001 0 -1 4202496 1286 0 0 0 2 0 0 0 20 0 1 0 668225654 19337216 1139 33554432000 4194304 5683644 140736708680848 18446744073709551615 4870108 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/11298/statm: 4721 1139 421 364 0 690 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 18884

[startup+0.100383 s]
/proc/loadavg: 8.55 8.51 6.35 1/467 11298
/proc/meminfo: memFree=14121436/32950928 swapFree=60199076/67111528
[pid=11298] ppid=11296 vsize=89772 CPUtime=0.06 cores=0-7
/proc/11298/stat : 11298 (python) R 11296 11298 11001 0 -1 4202496 2643 0 0 0 5 1 0 0 20 0 1 0 668225654 91926528 2359 33554432000 4194304 5683644 140736708680848 18446744073709551615 47633461825712 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/11298/statm: 22443 2362 724 364 0 1623 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 89772

[startup+0.300377 s]
/proc/loadavg: 8.55 8.51 6.35 1/467 11298
/proc/meminfo: memFree=14121436/32950928 swapFree=60199076/67111528
[pid=11298] ppid=11296 vsize=135752 CPUtime=0.19 cores=0-7
/proc/11298/stat : 11298 (python) S 11296 11298 11001 0 -1 4202496 5966 937 1 0 15 4 0 0 21 0 1 0 668225654 139010048 4151 33554432000 4194304 5683644 140736708680848 18446744073709551615 268272685061 0 2147483391 16781312 2 18446744071699199166 0 0 17 2 0 0 0
/proc/11298/statm: 33938 4151 1032 364 0 3124 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 135752

[startup+0.700204 s]
/proc/loadavg: 8.55 8.51 6.35 1/467 11298
/proc/meminfo: memFree=14121436/32950928 swapFree=60199076/67111528
[pid=11298] ppid=11296 vsize=283276 CPUtime=0.43 cores=0-7
/proc/11298/stat : 11298 (python) R 11296 11298 11001 0 -1 4202496 10460 937 1 0 33 10 0 0 20 0 1 0 668225654 290074624 8140 33554432000 4194304 5683644 140736708680848 18446744073709551615 4445115 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/11298/statm: 70819 8140 1761 364 0 6743 0
Current children cumulated CPU time (s) 0.43
Current children cumulated vsize (KiB) 283276

[startup+1.50106 s]
/proc/loadavg: 8.55 8.51 6.35 2/468 11303
/proc/meminfo: memFree=14076044/32950928 swapFree=60199076/67111528
[pid=11298] ppid=11296 vsize=317988 CPUtime=1.23 cores=0-7
/proc/11298/stat : 11298 (python) R 11296 11298 11001 0 -1 4202496 23799 937 1 0 111 12 0 0 22 0 1 0 668225654 325619712 16881 33554432000 4194304 5683644 140736708680848 18446744073709551615 268276097717 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/11298/statm: 79497 16881 1826 364 0 15421 0
Current children cumulated CPU time (s) 1.23
Current children cumulated vsize (KiB) 317988

[startup+3.10077 s]
/proc/loadavg: 7.95 8.38 6.32 2/468 11303
/proc/meminfo: memFree=14059684/32950928 swapFree=60199076/67111528
[pid=11298] ppid=11296 vsize=317988 CPUtime=2.83 cores=0-7
/proc/11298/stat : 11298 (python) R 11296 11298 11001 0 -1 4202496 23799 937 1 0 271 12 0 0 25 0 1 0 668225654 325619712 16881 33554432000 4194304 5683644 140736708680848 18446744073709551615 47633468605488 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/11298/statm: 79497 16881 1826 364 0 15421 0
Current children cumulated CPU time (s) 2.83
Current children cumulated vsize (KiB) 317988

[startup+6.3002 s]
/proc/loadavg: 7.95 8.38 6.32 2/468 11303
/proc/meminfo: memFree=14060056/32950928 swapFree=60199076/67111528
[pid=11298] ppid=11296 vsize=317988 CPUtime=6.03 cores=0-7
/proc/11298/stat : 11298 (python) R 11296 11298 11001 0 -1 4202496 23800 937 1 0 591 12 0 0 25 0 1 0 668225654 325619712 16882 33554432000 4194304 5683644 140736708680848 18446744073709551615 47633468440917 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/11298/statm: 79497 16882 1827 364 0 15421 0
Current children cumulated CPU time (s) 6.03
Current children cumulated vsize (KiB) 317988

[startup+12.7011 s]
/proc/loadavg: 7.44 8.26 6.30 9/492 11327
/proc/meminfo: memFree=13771316/32950928 swapFree=60199076/67111528
[pid=11298] ppid=11296 vsize=462456 CPUtime=9.25 cores=0-7
/proc/11298/stat : 11298 (python) S 11296 11298 11001 0 -1 4202496 29614 937 1 0 911 14 0 0 24 0 9 0 668225654 473554944 16359 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284515787 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/11298/statm: 115614 16359 1889 364 0 51511 0
[pid=11304] ppid=11298 vsize=314884 CPUtime=0 cores=0-7
/proc/11304/stat : 11304 (python) S 11298 11298 11001 0 -1 4202560 739 0 0 0 0 0 0 0 15 0 1 0 668226584 322441216 14626 33554432000 4194304 5683644 140736708680848 18446744073709551615 268272709135 0 0 16781312 514 7899023479633051907 0 0 17 1 0 0 0
/proc/11304/statm: 78721 14626 353 364 0 14639 0
[pid=11306] ppid=11304 vsize=276236 CPUtime=3.38 cores=0-7
/proc/11306/stat : 11306 (hybridGM3) R 11304 11306 11306 0 -1 4202496 8558 0 0 0 337 1 0 0 25 0 1 0 668226584 282865664 8296 33554432000 134512640 135204385 4292920048 18446744073709551615 134536744 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/11306/statm: 69059 8296 64 169 0 68889 0
[pid=11307] ppid=11298 vsize=325140 CPUtime=0 cores=0-7
/proc/11307/stat : 11307 (python) S 11298 11298 11001 0 -1 4202560 742 0 0 0 0 0 0 0 15 0 1 0 668226587 332943360 14624 33554432000 4194304 5683644 140736708680848 18446744073709551615 268272709190 0 0 16781312 514 10216173866426794243 0 0 17 1 0 0 0
/proc/11307/statm: 81285 14624 347 364 0 17200 0
[pid=11309] ppid=11307 vsize=276240 CPUtime=3.35 cores=0-7
/proc/11309/stat : 11309 (hybridGM3) R 11307 11309 11309 0 -1 4202496 8562 0 0 0 334 1 0 0 25 0 1 0 668226588 282869760 8298 33554432000 134512640 135204385 4287878032 18446744073709551615 134536641 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/11309/statm: 69060 8298 64 169 0 68890 0
[pid=11310] ppid=11298 vsize=400932 CPUtime=0 cores=0-7
/proc/11310/stat : 11310 (python) S 11298 11298 11001 0 -1 4202560 706 0 0 0 0 0 0 0 15 0 1 0 668226590 410554368 14802 33554432000 4194304 5683644 140736708680848 18446744073709551615 268272709190 0 0 16781312 514 15479755930916061443 0 0 17 1 0 0 0
/proc/11310/statm: 100233 14802 348 364 0 36145 0
[pid=11312] ppid=11310 vsize=276244 CPUtime=3.32 cores=0-7
/proc/11312/stat : 11312 (hybridGM3) R 11310 11312 11312 0 -1 4202496 8549 0 0 0 331 1 0 0 25 0 1 0 668226591 282873856 8297 33554432000 134512640 135204385 4290050432 18446744073709551615 134545851 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/11312/statm: 69061 8297 64 169 0 68891 0
[pid=11313] ppid=11298 vsize=411188 CPUtime=0 cores=0-7
/proc/11313/stat : 11313 (python) S 11298 11298 11001 0 -1 4202560 718 0 0 0 0 0 0 0 15 0 1 0 668226593 421056512 14763 33554432000 4194304 5683644 140736708680848 18446744073709551615 268272709190 0 0 16781312 514 3737745802454335745 0 0 17 1 0 0 0
/proc/11313/statm: 102797 14763 348 364 0 38706 0
[pid=11315] ppid=11313 vsize=276236 CPUtime=3.28 cores=0-7
/proc/11315/stat : 11315 (hybridGM3) R 11313 11315 11315 0 -1 4202496 8551 0 0 0 328 0 0 0 25 0 1 0 668226594 282865664 8298 33554432000 134512640 135204385 4292871216 18446744073709551615 134536663 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/11315/statm: 69059 8298 64 169 0 68889 0
[pid=11316] ppid=11298 vsize=421444 CPUtime=0 cores=0-7
/proc/11316/stat : 11316 (python) S 11298 11298 11001 0 -1 4202560 727 0 0 0 0 0 0 0 15 0 1 0 668226597 431558656 14785 33554432000 4194304 5683644 140736708680848 18446744073709551615 268272709190 0 0 16781312 514 10216169468380283139 0 0 17 1 0 0 0
/proc/11316/statm: 105361 14785 348 364 0 41267 0
[pid=11318] ppid=11316 vsize=276240 CPUtime=3.2 cores=0-7
/proc/11318/stat : 11318 (hybridGM3) R 11316 11318 11318 0 -1 4202496 8553 0 0 0 319 1 0 0 25 0 1 0 668226597 282869760 8297 33554432000 134512640 135204385 4289662688 18446744073709551615 134545975 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/11318/statm: 69060 8297 64 169 0 68890 0
[pid=11319] ppid=11298 vsize=431700 CPUtime=0 cores=0-7
/proc/11319/stat : 11319 (python) S 11298 11298 11001 0 -1 4202560 733 0 0 0 0 0 0 0 15 0 1 0 668226600 442060800 14806 33554432000 4194304 5683644 140736708680848 18446744073709551615 268272709190 0 0 16781312 514 10062273024863731972 0 0 17 2 0 0 0
/proc/11319/statm: 107925 14806 348 364 0 43828 0
[pid=11321] ppid=11319 vsize=276236 CPUtime=3.22 cores=0-7
/proc/11321/stat : 11321 (hybridGM3) R 11319 11321 11321 0 -1 4202496 8554 0 0 0 321 1 0 0 25 0 1 0 668226601 282865664 8295 33554432000 134512640 135204385 4289987008 18446744073709551615 134546082 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/11321/statm: 69059 8295 64 169 0 68889 0
[pid=11322] ppid=11298 vsize=441956 CPUtime=0 cores=0-7
/proc/11322/stat : 11322 (python) S 11298 11298 11001 0 -1 4202560 742 0 0 0 0 0 0 0 15 0 1 0 668226603 452562944 14827 33554432000 4194304 5683644 140736708680848 18446744073709551615 268272709190 0 0 16781312 514 2359719083269652737 0 0 17 2 0 0 0
/proc/11322/statm: 110489 14827 348 364 0 46389 0
[pid=11324] ppid=11322 vsize=276240 CPUtime=3.16 cores=0-7
/proc/11324/stat : 11324 (hybridGM3) R 11322 11324 11324 0 -1 4202496 8560 0 0 0 315 1 0 0 25 0 1 0 668226604 282869760 8300 33554432000 134512640 135204385 4287388736 18446744073709551615 134537272 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/11324/statm: 69060 8300 64 169 0 68890 0
[pid=11325] ppid=11298 vsize=452212 CPUtime=0 cores=0-7
/proc/11325/stat : 11325 (python) S 11298 11298 11001 0 -1 4202560 776 0 0 0 0 0 0 0 15 0 1 0 668226607 463065088 14820 33554432000 4194304 5683644 140736708680848 18446744073709551615 268272709190 0 0 16781312 514 1862528721282367747 0 0 17 2 0 0 0
/proc/11325/statm: 113053 14820 354 364 0 48950 0
[pid=11327] ppid=11325 vsize=276236 CPUtime=3.12 cores=0-7
/proc/11327/stat : 11327 (hybridGM3) R 11325 11327 11327 0 -1 4202496 8563 0 0 0 311 1 0 0 25 0 1 0 668226609 282865664 8298 33554432000 134512640 135204385 4289035136 18446744073709551615 134536986 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/11327/statm: 69059 8298 64 169 0 68889 0
[pid=11298/tid=11305] ppid=11296 vsize=462456 CPUtime=0 cores=0-7
/proc/11298/task/11305/stat : 11305 (python) S 11296 11298 11001 0 -1 4202560 26 937 0 0 0 0 0 0 25 0 9 0 668226584 473554944 16359 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 0 0 0 0
[pid=11298/tid=11308] ppid=11296 vsize=462456 CPUtime=0 cores=0-7
/proc/11298/task/11308/stat : 11308 (python) S 11296 11298 11001 0 -1 4202560 25 937 0 0 0 0 0 0 18 0 9 0 668226587 473554944 16359 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 3 0 0 0
[pid=11298/tid=11311] ppid=11296 vsize=462456 CPUtime=0 cores=0-7
/proc/11298/task/11311/stat : 11311 (python) S 11296 11298 11001 0 -1 4202560 81 937 0 0 0 0 0 0 21 0 9 0 668226591 473554944 16359 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 0 0 0 0
[pid=11298/tid=11314] ppid=11296 vsize=462456 CPUtime=0 cores=0-7
/proc/11298/task/11314/stat : 11314 (python) S 11296 11298 11001 0 -1 4202560 33 937 0 0 0 0 0 0 22 0 9 0 668226594 473554944 16359 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 5 0 0 0
[pid=11298/tid=11317] ppid=11296 vsize=462456 CPUtime=0 cores=0-7
/proc/11298/task/11317/stat : 11317 (python) S 11296 11298 11001 0 -1 4202560 34 937 0 0 0 0 0 0 19 0 9 0 668226597 473554944 16359 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 0 0 0 0
[pid=11298/tid=11320] ppid=11296 vsize=462456 CPUtime=0 cores=0-7
/proc/11298/task/11320/stat : 11320 (python) S 11296 11298 11001 0 -1 4202560 32 937 0 0 0 0 0 0 21 0 9 0 668226600 473554944 16359 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 2 0 0 0
[pid=11298/tid=11323] ppid=11296 vsize=462456 CPUtime=0 cores=0-7
/proc/11298/task/11323/stat : 11323 (python) S 11296 11298 11001 0 -1 4202560 36 937 0 0 0 0 0 0 23 0 9 0 668226604 473554944 16359 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 0 0 0 0
[pid=11298/tid=11326] ppid=11296 vsize=462456 CPUtime=0 cores=0-7
/proc/11298/task/11326/stat : 11326 (python) S 11296 11298 11001 0 -1 4202560 35 937 0 0 0 0 0 0 25 0 9 0 668226607 473554944 16359 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 35.28
Current children cumulated vsize (KiB) 5871820

[startup+25.5018 s]
/proc/loadavg: 7.53 8.25 6.31 9/492 11328
/proc/meminfo: memFree=13772432/32950928 swapFree=60199076/67111528
[pid=11298] ppid=11296 vsize=462456 CPUtime=9.25 cores=0-7
/proc/11298/stat : 11298 (python) S 11296 11298 11001 0 -1 4202496 29614 937 1 0 911 14 0 0 24 0 9 0 668225654 473554944 16359 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284515787 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/11298/statm: 115614 16359 1889 364 0 51511 0
[pid=11304] ppid=11298 vsize=314884 CPUtime=0.04 cores=0-7
/proc/11304/stat : 11304 (python) S 11298 11298 11001 0 -1 4202560 916 0 0 0 2 2 0 0 15 0 1 0 668226584 322441216 14649 33554432000 4194304 5683644 140736708680848 18446744073709551615 268272709135 0 0 16781312 514 9578188837184569603 0 0 17 1 0 0 0
/proc/11304/statm: 78721 14649 376 364 0 14639 0
[pid=11306] ppid=11304 vsize=276236 CPUtime=16.18 cores=0-7
/proc/11306/stat : 11306 (hybridGM3) R 11304 11306 11306 0 -1 4202496 8558 0 0 0 1617 1 0 0 25 0 1 0 668226584 282865664 8296 33554432000 134512640 135204385 4292920048 18446744073709551615 134547446 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/11306/statm: 69059 8296 64 169 0 68889 0
[pid=11307] ppid=11298 vsize=325140 CPUtime=0.03 cores=0-7
/proc/11307/stat : 11307 (python) S 11298 11298 11001 0 -1 4202560 914 0 0 0 1 2 0 0 15 0 1 0 668226587 332943360 14647 33554432000 4194304 5683644 140736708680848 18446744073709551615 268272709190 0 0 16781312 514 9578188837184569603 0 0 17 1 0 0 0
/proc/11307/statm: 81285 14647 370 364 0 17200 0
[pid=11309] ppid=11307 vsize=276240 CPUtime=16.15 cores=0-7
/proc/11309/stat : 11309 (hybridGM3) R 11307 11309 11309 0 -1 4202496 8568 0 0 0 1614 1 0 0 25 0 1 0 668226588 282869760 8304 33554432000 134512640 135204385 4287878032 18446744073709551615 134546144 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/11309/statm: 69060 8304 64 169 0 68890 0
[pid=11310] ppid=11298 vsize=400932 CPUtime=0.04 cores=0-7
/proc/11310/stat : 11310 (python) S 11298 11298 11001 0 -1 4202560 786 0 0 0 2 2 0 0 15 0 1 0 668226590 410554368 14825 33554432000 4194304 5683644 140736708680848 18446744073709551615 268272709190 0 0 16781312 514 9578188837184569603 0 0 17 1 0 0 0
/proc/11310/statm: 100233 14825 371 364 0 36145 0
[pid=11312] ppid=11310 vsize=276244 CPUtime=16.12 cores=0-7
/proc/11312/stat : 11312 (hybridGM3) R 11310 11312 11312 0 -1 4202496 8549 0 0 0 1611 1 0 0 25 0 1 0 668226591 282873856 8297 33554432000 134512640 135204385 4290050432 18446744073709551615 134537100 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/11312/statm: 69061 8297 64 169 0 68891 0
[pid=11313] ppid=11298 vsize=411188 CPUtime=0.04 cores=0-7
/proc/11313/stat : 11313 (python) S 11298 11298 11001 0 -1 4202560 799 0 0 0 2 2 0 0 15 0 1 0 668226593 421056512 14786 33554432000 4194304 5683644 140736708680848 18446744073709551615 268272709190 0 0 16781312 514 9578188837184569601 0 0 17 1 0 0 0
/proc/11313/statm: 102797 14786 371 364 0 38706 0
[pid=11315] ppid=11313 vsize=276236 CPUtime=16.08 cores=0-7
/proc/11315/stat : 11315 (hybridGM3) R 11313 11315 11315 0 -1 4202496 8556 0 0 0 1608 0 0 0 25 0 1 0 668226594 282865664 8303 33554432000 134512640 135204385 4292871216 18446744073709551615 134545851 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/11315/statm: 69059 8303 64 169 0 68889 0
[pid=11316] ppid=11298 vsize=421444 CPUtime=0.04 cores=0-7
/proc/11316/stat : 11316 (python) S 11298 11298 11001 0 -1 4202560 805 0 0 0 2 2 0 0 15 0 1 0 668226597 431558656 14808 33554432000 4194304 5683644 140736708680848 18446744073709551615 268272709190 0 0 16781312 514 9578188837184569603 0 0 17 1 0 0 0
/proc/11316/statm: 105361 14808 371 364 0 41267 0
[pid=11318] ppid=11316 vsize=276240 CPUtime=15.76 cores=0-7
/proc/11318/stat : 11318 (hybridGM3) R 11316 11318 11318 0 -1 4202496 8553 0 0 0 1575 1 0 0 25 0 1 0 668226597 282869760 8297 33554432000 134512640 135204385 4289662688 18446744073709551615 134546082 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/11318/statm: 69060 8297 64 169 0 68890 0

################
# More data... #
################

/proc/11405/stat : 11405 (hybridGM3) T 11403 11405 11405 0 -1 4202496 8582 0 0 0 29253 3 0 0 25 0 1 0 668285515 282869760 8301 33554432000 134512640 135204385 4288449088 18446744073709551615 134537403 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/11405/statm: 69060 8301 64 169 0 68890 0
[pid=11403/tid=11440] ppid=11298 vsize=606052 CPUtime=0 cores=0-7
/proc/11403/task/11440/stat : 11440 (python) S 11298 11298 11001 0 -1 4202560 87 0 0 0 0 0 0 0 15 0 2 0 668305240 620597248 15236 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 514 18446604469405461376 0 0 -1 4 0 0 0
[pid=11406] ppid=11298 vsize=616308 CPUtime=0.4 cores=0-7
/proc/11406/stat : 11406 (python) S 11298 11298 11001 0 -1 4202560 1193 0 0 0 22 18 0 0 19 0 2 0 668286210 631099392 15207 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/11406/statm: 154077 15207 407 364 0 89926 0
[pid=11408] ppid=11406 vsize=467024 CPUtime=99.4 cores=0-7
/proc/11408/stat : 11408 (TNM) T 11406 11408 11408 0 -1 4202496 6840 0 0 0 9940 0 0 0 25 0 1 0 668286211 478232576 6557 33554432000 134512640 135008748 4294199488 18446744073709551615 134529367 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/11408/statm: 116756 6557 46 122 0 115656 0
[pid=11406/tid=11426] ppid=11298 vsize=616308 CPUtime=0 cores=0-7
/proc/11406/task/11426/stat : 11426 (python) S 11298 11298 11001 0 -1 4202560 85 0 0 0 0 0 0 0 18 0 2 0 668296153 631099392 15207 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 514 18446604470214800640 0 0 -1 2 0 0 0
[pid=11410] ppid=11298 vsize=626564 CPUtime=2 cores=0-7
/proc/11410/stat : 11410 (python) S 11298 11298 11001 0 -1 4202560 1182 0 0 0 93 107 0 0 18 0 2 0 668286485 641601536 15253 33554432000 4194304 5683644 140736708680848 18446744073709551615 268272709190 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/11410/statm: 156641 15253 400 364 0 92487 0
[pid=11412] ppid=11410 vsize=467024 CPUtime=488.96 cores=0-7
/proc/11412/stat : 11412 (TNM) R 11410 11412 11412 0 -1 4202496 6841 0 0 0 48894 2 0 0 25 0 1 0 668286486 478232576 6557 33554432000 134512640 135008748 4287672832 18446744073709551615 134528974 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/11412/statm: 116756 6557 46 122 0 115656 0
[pid=11410/tid=11430] ppid=11298 vsize=626564 CPUtime=0 cores=0-7
/proc/11410/task/11430/stat : 11430 (python) S 11298 11298 11001 0 -1 4202560 83 0 0 0 0 0 0 0 19 0 2 0 668297128 641601536 15253 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 514 18446604461383357376 0 0 -1 2 0 0 0
[pid=11417] ppid=11298 vsize=636820 CPUtime=0.4 cores=0-7
/proc/11417/stat : 11417 (python) S 11298 11298 11001 0 -1 4202560 1174 0 0 0 21 19 0 0 19 0 2 0 668294746 652103680 15219 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/11417/statm: 159205 15219 400 364 0 95048 0
[pid=11419] ppid=11417 vsize=467024 CPUtime=99.8 cores=0-7
/proc/11419/stat : 11419 (TNM) T 11417 11419 11419 0 -1 4202496 6838 0 0 0 9978 2 0 0 25 0 1 0 668294746 478232576 6557 33554432000 134512640 135008748 4291818432 18446744073709551615 134527111 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/11419/statm: 116756 6557 46 122 0 115656 0
[pid=11417/tid=11436] ppid=11298 vsize=636820 CPUtime=0 cores=0-7
/proc/11417/task/11436/stat : 11436 (python) S 11298 11298 11001 0 -1 4202560 85 0 0 0 0 0 0 0 19 0 2 0 668304786 652103680 15219 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 514 18446604461383356480 0 0 -1 5 0 0 0
[pid=11422] ppid=11298 vsize=647076 CPUtime=0.79 cores=0-7
/proc/11422/stat : 11422 (python) S 11298 11298 11001 0 -1 4202560 1196 0 0 0 39 40 0 0 19 0 2 0 668295777 662605824 15199 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/11422/statm: 161769 15199 400 364 0 97609 0
[pid=11424] ppid=11422 vsize=434080 CPUtime=195.2 cores=0-7
/proc/11424/stat : 11424 (adaptg2wsat2009) T 11422 11424 11424 0 -1 4202496 7102 0 0 0 19518 2 0 0 25 0 1 0 668295778 444497920 6820 33554432000 134512640 135107411 4288746512 18446744073709551615 134532288 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/11424/statm: 108520 6820 59 146 0 107396 0
[pid=11422/tid=11450] ppid=11298 vsize=647076 CPUtime=0 cores=0-7
/proc/11422/task/11450/stat : 11450 (python) S 11298 11298 11001 0 -1 4202560 90 0 0 0 0 0 0 0 19 0 2 0 668315454 662605824 15199 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 514 18446604465144463488 0 0 -1 0 0 0 0
[pid=11427] ppid=11298 vsize=657332 CPUtime=0.4 cores=0-7
/proc/11427/stat : 11427 (python) S 11298 11298 11001 0 -1 4202560 1202 0 0 0 21 19 0 0 19 0 2 0 668296160 673107968 15208 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/11427/statm: 164333 15208 400 364 0 100170 0
[pid=11429] ppid=11427 vsize=467024 CPUtime=99.24 cores=0-7
/proc/11429/stat : 11429 (TNM) T 11427 11429 11429 0 -1 4202496 6842 0 0 0 9923 1 0 0 25 0 1 0 668296160 478232576 6559 33554432000 134512640 135008748 4287709136 18446744073709551615 134528940 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/11429/statm: 116756 6559 46 122 0 115656 0
[pid=11427/tid=11442] ppid=11298 vsize=657332 CPUtime=0 cores=0-7
/proc/11427/task/11442/stat : 11442 (python) S 11298 11298 11001 0 -1 4202560 91 0 0 0 0 0 0 0 19 0 2 0 668306100 673107968 15208 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 514 18446604465144464384 0 0 -1 2 0 0 0
[pid=11431] ppid=11298 vsize=667588 CPUtime=0.41 cores=0-7
/proc/11431/stat : 11431 (python) S 11298 11298 11001 0 -1 4202560 1234 0 0 0 21 20 0 0 19 0 2 0 668297135 683610112 15308 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/11431/statm: 166897 15308 407 364 0 102731 0
[pid=11433] ppid=11431 vsize=467024 CPUtime=99.09 cores=0-7
/proc/11433/stat : 11433 (TNM) T 11431 11433 11433 0 -1 4202496 6850 0 0 0 9908 1 0 0 25 0 1 0 668297135 478232576 6559 33554432000 134512640 135008748 4292060208 18446744073709551615 134527795 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/11433/statm: 116756 6559 46 122 0 115656 0
[pid=11431/tid=11443] ppid=11298 vsize=667588 CPUtime=0 cores=0-7
/proc/11431/task/11443/stat : 11443 (python) S 11298 11298 11001 0 -1 4202560 88 0 0 0 0 0 0 0 19 0 2 0 668307175 683610112 15308 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 514 18446604463165509824 0 0 -1 3 0 0 0
[pid=11437] ppid=11298 vsize=677844 CPUtime=0.41 cores=0-7
/proc/11437/stat : 11437 (python) S 11298 11298 11001 0 -1 4202560 1238 0 0 0 22 19 0 0 19 0 2 0 668304793 694112256 15265 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/11437/statm: 169461 15265 407 364 0 105292 0
[pid=11439] ppid=11437 vsize=467024 CPUtime=98.46 cores=0-7
/proc/11439/stat : 11439 (TNM) T 11437 11439 11439 0 -1 4202496 6845 0 0 0 9846 0 0 0 25 0 1 0 668304794 478232576 6559 33554432000 134512640 135008748 4288053440 18446744073709551615 134527773 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/11439/statm: 116756 6559 46 122 0 115656 0
[pid=11437/tid=11446] ppid=11298 vsize=677844 CPUtime=0 cores=0-7
/proc/11437/task/11446/stat : 11446 (python) S 11298 11298 11001 0 -1 4202560 88 0 0 0 0 0 0 0 19 0 2 0 668314739 694112256 15265 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 514 18446604463165510720 0 0 -1 5 0 0 0
[pid=11447] ppid=11298 vsize=688100 CPUtime=0.83 cores=0-7
/proc/11447/stat : 11447 (python) S 11298 11298 11001 0 -1 4202560 1260 0 0 0 41 42 0 0 19 0 2 0 668315392 704614400 15215 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284515787 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/11447/statm: 172025 15215 406 364 0 107853 0
[pid=11449] ppid=11447 vsize=434080 CPUtime=197.04 cores=0-7
/proc/11449/stat : 11449 (adaptg2wsat2009) T 11447 11449 11449 0 -1 4202496 7108 0 0 0 19702 2 0 0 25 0 1 0 668315393 444497920 6819 33554432000 134512640 135107411 4293682224 18446744073709551615 134532326 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/11449/statm: 108520 6819 59 146 0 107396 0
[pid=11447/tid=11460] ppid=11298 vsize=688100 CPUtime=0 cores=0-7
/proc/11447/task/11460/stat : 11460 (python) S 11298 11298 11001 0 -1 4202560 96 0 0 0 0 0 0 0 19 0 2 0 668336274 704614400 15215 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 514 18446604443720934720 0 0 -1 7 0 0 0
[pid=11456] ppid=11298 vsize=698356 CPUtime=0.38 cores=0-7
/proc/11456/stat : 11456 (python) S 11298 11298 11001 0 -1 4202560 1059 0 0 0 20 18 0 0 15 0 1 0 668335242 715116544 15197 33554432000 4194304 5683644 140736708680848 18446744073709551615 268272709190 0 0 16781312 514 9578188837184569602 0 0 17 4 0 0 0
/proc/11456/statm: 174589 15197 379 364 0 110414 0
[pid=11458] ppid=11456 vsize=276244 CPUtime=91.41 cores=0-7
/proc/11458/stat : 11458 (hybridGM3) R 11456 11458 11458 0 -1 4202496 8600 0 0 0 9140 1 0 0 25 0 1 0 668335243 282873856 8308 33554432000 134512640 135204385 4288112112 18446744073709551615 134537502 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/11458/statm: 69061 8308 64 169 0 68891 0
[pid=11461] ppid=11298 vsize=708612 CPUtime=0.33 cores=0-7
/proc/11461/stat : 11461 (python) S 11298 11298 11001 0 -1 4202560 1064 0 0 0 19 14 0 0 18 0 1 0 668336281 725618688 15305 33554432000 4194304 5683644 140736708680848 18446744073709551615 268272709190 0 0 16781312 514 9578188837184569603 0 0 17 4 0 0 0
/proc/11461/statm: 177153 15305 379 364 0 112975 0
[pid=11463] ppid=11461 vsize=262376 CPUtime=85.13 cores=0-7
/proc/11463/stat : 11463 (FH) R 11461 11463 11463 0 -1 4202496 7014 0 0 0 8512 1 0 0 25 0 1 0 668336281 268673024 6723 33554432000 134512640 134936988 4289466656 18446744073709551615 134554211 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/11463/statm: 65594 6723 41 104 0 65098 0
[pid=11466] ppid=11298 vsize=718868 CPUtime=0 cores=0-7
/proc/11466/stat : 11466 (python) S 11298 11298 11001 0 -1 4202560 916 0 0 0 0 0 0 0 19 0 1 0 668344839 736120832 15217 33554432000 4194304 5683644 140736708680848 18446744073709551615 268272709190 0 0 16781312 514 4562630211937501183 0 0 17 4 0 0 0
/proc/11466/statm: 179717 15217 346 364 0 115536 0
[pid=11468] ppid=11466 vsize=429720 CPUtime=0.47 cores=0-7
/proc/11468/stat : 11468 (adaptg2wsat2009) R 11466 11468 11468 0 -1 4202496 4688 0 0 0 46 1 0 0 21 0 1 0 668344840 440033280 4401 33554432000 134512640 135107411 4293960832 18446744073709551615 134523503 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/11468/statm: 107430 4401 49 146 0 106306 0
[pid=11298/tid=11305] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11305/stat : 11305 (python) S 11296 11298 11001 0 -1 4202560 46 937 0 0 0 0 0 0 15 0 35 0 668226584 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 0 0 0 0
[pid=11298/tid=11308] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11308/stat : 11308 (python) S 11296 11298 11001 0 -1 4202560 43 937 0 0 0 0 0 0 15 0 35 0 668226587 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 2 0 0 0
[pid=11298/tid=11311] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11311/stat : 11311 (python) S 11296 11298 11001 0 -1 4202560 101 937 0 0 0 0 0 0 15 0 35 0 668226591 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 0 0 0 0
[pid=11298/tid=11314] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11314/stat : 11314 (python) S 11296 11298 11001 0 -1 4202560 48 937 0 0 0 0 0 0 18 0 35 0 668226594 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 5 0 0 0
[pid=11298/tid=11317] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11317/stat : 11317 (python) S 11296 11298 11001 0 -1 4202560 50 937 0 0 0 0 0 0 18 0 35 0 668226597 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 2 0 0 0
[pid=11298/tid=11320] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11320/stat : 11320 (python) S 11296 11298 11001 0 -1 4202560 45 937 0 0 0 0 0 0 18 0 35 0 668226600 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 0 0 0 0
[pid=11298/tid=11323] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11323/stat : 11323 (python) S 11296 11298 11001 0 -1 4202560 42 937 0 0 0 0 0 0 18 0 35 0 668226604 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 0 0 0 0
[pid=11298/tid=11326] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11326/stat : 11326 (python) S 11296 11298 11001 0 -1 4202560 43 937 0 0 0 0 0 0 18 0 35 0 668226607 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 0 0 0 0
[pid=11298/tid=11340] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11340/stat : 11340 (python) S 11296 11298 11001 0 -1 4202560 53 937 0 0 0 0 0 0 18 0 35 0 668246207 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 0 0 0 0
[pid=11298/tid=11349] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11349/stat : 11349 (python) S 11296 11298 11001 0 -1 4202560 52 937 0 0 0 0 0 0 18 0 35 0 668256247 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 1 0 0 0
[pid=11298/tid=11352] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11352/stat : 11352 (python) S 11296 11298 11001 0 -1 4202560 40 937 0 0 0 0 0 0 18 0 35 0 668256343 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 0 0 0 0
[pid=11298/tid=11355] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11355/stat : 11355 (python) S 11296 11298 11001 0 -1 4202560 31 937 0 0 0 0 0 0 21 0 35 0 668256353 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 0 0 0 0
[pid=11298/tid=11358] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11358/stat : 11358 (python) S 11296 11298 11001 0 -1 4202560 37 937 0 0 0 0 0 0 20 0 35 0 668256459 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 2 0 0 0
[pid=11298/tid=11361] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11361/stat : 11361 (python) S 11296 11298 11001 0 -1 4202560 36 937 0 0 0 0 0 0 20 0 35 0 668256553 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 2 0 0 0
[pid=11298/tid=11367] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11367/stat : 11367 (python) S 11296 11298 11001 0 -1 4202560 37 937 0 0 0 0 0 0 20 0 35 0 668266000 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 0 0 0 0
[pid=11298/tid=11371] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11371/stat : 11371 (python) S 11296 11298 11001 0 -1 4202560 33 937 0 0 0 0 0 0 20 0 35 0 668266396 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 5 0 0 0
[pid=11298/tid=11375] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11375/stat : 11375 (python) S 11296 11298 11001 0 -1 4202560 34 937 0 0 0 0 0 0 21 0 35 0 668266406 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 5 0 0 0
[pid=11298/tid=11381] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11381/stat : 11381 (python) S 11296 11298 11001 0 -1 4202560 35 937 0 0 0 0 0 0 20 0 35 0 668275019 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 5 0 0 0
[pid=11298/tid=11387] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11387/stat : 11387 (python) S 11296 11298 11001 0 -1 4202560 52 937 0 0 0 0 0 0 18 0 35 0 668276041 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 0 0 0 0
[pid=11298/tid=11390] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11390/stat : 11390 (python) S 11296 11298 11001 0 -1 4202560 31 937 0 0 0 0 0 0 20 0 35 0 668276048 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 5 0 0 0
[pid=11298/tid=11395] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11395/stat : 11395 (python) S 11296 11298 11001 0 -1 4202560 40 937 0 0 0 0 0 0 18 0 35 0 668276352 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 5 0 0 0
[pid=11298/tid=11399] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11399/stat : 11399 (python) S 11296 11298 11001 0 -1 4202560 34 937 0 0 0 0 0 0 20 0 35 0 668276441 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 5 0 0 0
[pid=11298/tid=11404] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11404/stat : 11404 (python) S 11296 11298 11001 0 -1 4202560 39 937 0 0 0 0 0 0 18 0 35 0 668285514 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 5 0 0 0
[pid=11298/tid=11407] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11407/stat : 11407 (python) S 11296 11298 11001 0 -1 4202560 35 937 0 0 0 0 0 0 20 0 35 0 668286211 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 5 0 0 0
[pid=11298/tid=11411] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11411/stat : 11411 (python) S 11296 11298 11001 0 -1 4202560 39 937 0 0 0 0 0 0 18 0 35 0 668286486 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 5 0 0 0
[pid=11298/tid=11418] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11418/stat : 11418 (python) S 11296 11298 11001 0 -1 4202560 35 937 0 0 0 0 0 0 19 0 35 0 668294746 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 5 0 0 0
[pid=11298/tid=11423] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11423/stat : 11423 (python) S 11296 11298 11001 0 -1 4202560 35 937 0 0 0 0 0 0 20 0 35 0 668295778 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 3 0 0 0
[pid=11298/tid=11428] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11428/stat : 11428 (python) S 11296 11298 11001 0 -1 4202560 35 937 0 0 0 0 0 0 20 0 35 0 668296160 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 3 0 0 0
[pid=11298/tid=11432] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11432/stat : 11432 (python) S 11296 11298 11001 0 -1 4202560 34 937 0 0 0 0 0 0 20 0 35 0 668297135 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 3 0 0 0
[pid=11298/tid=11438] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11438/stat : 11438 (python) S 11296 11298 11001 0 -1 4202560 33 937 0 0 0 0 0 0 19 0 35 0 668304793 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 3 0 0 0
[pid=11298/tid=11448] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11448/stat : 11448 (python) S 11296 11298 11001 0 -1 4202560 35 937 0 0 0 0 0 0 20 0 35 0 668315393 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 1 0 0 0
[pid=11298/tid=11457] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11457/stat : 11457 (python) S 11296 11298 11001 0 -1 4202560 37 937 0 0 0 0 0 0 19 0 35 0 668335242 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 4 0 0 0
[pid=11298/tid=11462] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11462/stat : 11462 (python) S 11296 11298 11001 0 -1 4202560 32 937 0 0 0 0 0 0 20 0 35 0 668336281 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 4 0 0 0
[pid=11298/tid=11467] ppid=11296 vsize=729112 CPUtime=0 cores=0-7
/proc/11298/task/11467/stat : 11467 (python) S 11296 11298 11001 0 -1 4202560 34 937 0 0 0 0 0 0 20 0 35 0 668344839 746610688 16817 33554432000 4194304 5683644 140736708680848 18446744073709551615 268284512689 0 0 16781312 2 18446604450859169792 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 9423.43
Current children cumulated vsize (KiB) 32037976

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 11298 and gives
#  childrusage.ru_utime.tv_sec=2167
#  childrusage.ru_utime.tv_usec=259526
#  childrusage.ru_stime.tv_sec=18
#  childrusage.ru_stime.tv_usec=776145
# CPU time returned by wait4() is 2186.04
# while last known CPU time is 9423.43
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1192.53
CPU time (s): 9423.43
CPU user time (s): 9404.54
CPU system time (s): 18.89
CPU usage (%): 790.205
Max. virtual memory (cumulated for all children) (KiB): 32037976

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2167.26
system time used= 18.7761
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 163317
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 16572
involuntary context switches= 11923

runsolver used 13.245 second user time and 34.6027 second system time

The end

Launcher Data 

Begin job on node122 at 2011-05-16 19:17:48
IDJOB=3385405
IDBENCH=82670
IDSOLVER=1870
FILE ID=node122/3385405-1305566268
RUNJOBID= node122-1305564964-11015
PBS_JOBID= 13350128
Free space on /tmp= 73396 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/large/unif-k3-r4.2-v40000-c168000-S386300535-065-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3385405-1305566268/watcher-3385405-1305566268 -o /tmp/evaluation-result-3385405-1305566268/solver-3385405-1305566268 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385405-1305566268.cnf -seed 616990486 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 2ca6073d25d8d5f2648d590c0edc75c0
RANDOM SEED=616990486

node122.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.64
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.97
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      14121344 kB
Buffers:        330564 kB
Cached:        4393624 kB
SwapCached:      50104 kB
Active:       14072784 kB
Inactive:      4468436 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      14121344 kB
SwapTotal:    67111528 kB
SwapFree:     60199076 kB
Dirty:           35228 kB
Writeback:           0 kB
AnonPages:    13778188 kB
Mapped:          28404 kB
Slab:           137344 kB
PageTables:      76764 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 139590124 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73364 MiB
End job on node122 at 2011-05-16 19:37:43