Trace number 3385312

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)? (MO) 7159.52 909.818

General information on the benchmark

Namerandom/large/
unif-k3-r4.2-v50000-c210000-S746725327-020-UNKNOWN.cnf
MD5SUM0ad997eea0def3552b6d7e45aac8ca7c
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark1056.42
Satisfiable
(Un)Satisfiability was proved
Number of variables49999
Number of clauses210000
Sum of the clauses size630000
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3210000
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.34/0.67	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.34/0.67	c INFO: solving HOME/instance-3385312-1305565050.cnf
0.34/0.67	c INFO: computing task features
1.12/1.40	c INFO: computed variable-clause graph statistics
1.12/1.43	c INFO: computed clause balance statistics
5.13/5.48	c INFO: computed variable balance statistics
8.43/8.76	c INFO: computed small-clause counts
8.43/8.78	c INFO: computed Horn-clause counts
11.92/12.20	c INFO: computed variable graph statistics
11.92/12.20	c INFO: collected features for HOME/instance-3385312-1305565050.cnf in 11.50 s
11.92/12.23	c INFO: running hybridGM3@0 for 97 with 4988 remaining (b = 0.33)
11.92/12.26	c INFO: running hybridGM3@0 for 97 with 4988 remaining (b = 0.21)
11.92/12.29	c INFO: running hybridGM3@0 for 97 with 4988 remaining (b = 0.14)
12.02/12.33	c INFO: running hybridGM3@0 for 97 with 4988 remaining (b = 0.09)
12.02/12.37	c INFO: running hybridGM3@0 for 97 with 4988 remaining (b = 0.07)
12.12/12.40	c INFO: running hybridGM3@0 for 194 with 4988 remaining (b = 0.10)
12.12/12.43	c INFO: running hybridGM3@0 for 194 with 4988 remaining (b = 0.07)
12.12/12.46	c INFO: running hybridGM3@0 for 194 with 4988 remaining (b = 0.06)
797.21/110.76	c INFO: running hybridGM3@0 for 97 with 4889 remaining (b = 0.03)
812.38/112.64	c INFO: running hybridGM3@0 for 97 with 4888 remaining (b = 0.03)
812.38/112.69	c INFO: running hybridGM3@0 for 97 with 4888 remaining (b = 0.03)
813.11/112.72	c INFO: running hybridGM3@0 for 97 with 4887 remaining (b = 0.03)
813.11/112.77	c INFO: running hybridGM3@0 for 97 with 4887 remaining (b = 0.03)
1572.71/207.96	c INFO: running hybridGM3@100 for 97 with 4792 remaining (b = 0.03)
1572.71/207.99	c INFO: running hybridGM3@100 for 97 with 4792 remaining (b = 0.03)
1573.41/208.05	c INFO: running TNM@0 for 97 with 4792 remaining (b = 0.03)
1581.47/209.08	c INFO: running hybridGM3@94 for 97 with 4791 remaining (b = 0.02)
1582.16/209.11	c INFO: running hybridGM3@100 for 97 with 4791 remaining (b = 0.02)
1582.16/209.14	c INFO: running hybridGM3@0 for 97 with 4791 remaining (b = 0.02)
1620.36/213.93	c INFO: running hybridGM3@98 for 97 with 4786 remaining (b = 0.02)
1629.87/215.17	c INFO: running hybridGM3@100 for 97 with 4785 remaining (b = 0.02)
2373.41/308.35	c INFO: running hybridGM3@0 for 97 with 4692 remaining (b = 0.02)
2380.59/309.24	c INFO: running hybridGM3@0 for 194 with 4691 remaining (b = 0.04)
2380.59/309.29	c INFO: running TNM@0 for 97 with 4691 remaining (b = 0.02)
2381.32/309.37	c INFO: running hybridGM3@0 for 194 with 4691 remaining (b = 0.03)
2381.87/309.43	c INFO: running hybridGM3@296 for 97 with 4691 remaining (b = 0.02)
2381.87/309.47	c INFO: running TNM@0 for 97 with 4691 remaining (b = 0.01)
2390.04/310.44	c INFO: running hybridGM3@99 for 97 with 4690 remaining (b = 0.01)
2396.46/311.22	c INFO: running hybridGM3@296 for 97 with 4689 remaining (b = 0.01)
3141.93/404.72	c INFO: running hybridGM3@391 for 97 with 4595 remaining (b = 0.01)
3149.01/405.66	c INFO: running hybridGM3@100 for 97 with 4595 remaining (b = 0.01)
3149.81/405.74	c INFO: running TNM@0 for 97 with 4594 remaining (b = 0.01)
3156.12/406.52	c INFO: running hybridGM3@392 for 97 with 4594 remaining (b = 0.01)
3180.70/409.60	c INFO: running TNM@0 for 97 with 4591 remaining (b = 0.01)
3182.13/409.81	c INFO: running TNM@0 for 97 with 4590 remaining (b = 0.01)
3907.90/500.96	c INFO: running TNM@0 for 97 with 4499 remaining (b = 0.01)
3932.69/504.06	c INFO: running TNM@0 for 97 with 4496 remaining (b = 0.01)
3947.78/505.98	c INFO: running hybridGM3@98 for 97 with 4494 remaining (b = 0.01)
3948.61/506.08	c INFO: running TNM@0 for 97 with 4494 remaining (b = 0.01)
3949.24/506.14	c INFO: running hybridGM3@99 for 97 with 4494 remaining (b = 0.01)
3962.73/507.84	c INFO: running TNM@0 for 2037 with 4492 remaining (b = 0.19)
3979.42/509.94	c INFO: running hybridGM3@0 for 194 with 4490 remaining (b = 0.02)
4020.80/515.18	c INFO: running hybridGM3@0 for 194 with 4485 remaining (b = 0.02)
4707.52/601.31	c INFO: running hybridGM3@0 for 194 with 4399 remaining (b = 0.01)
4708.25/601.45	c INFO: running adaptg2wsat2009++@0 for 97 with 4399 remaining (b = 0.01)
4732.15/604.46	c INFO: running hybridGM3@0 for 194 with 4396 remaining (b = 0.01)
4747.99/606.48	c INFO: running adaptg2wsat2009++@0 for 194 with 4394 remaining (b = 0.01)
4755.14/607.35	c INFO: running TNM@0 for 97 with 4393 remaining (b = 0.01)
5507.18/701.84	c INFO: running TNM@0 for 97 with 4298 remaining (b = 0.01)
5546.85/706.81	c INFO: running hybridGM3@101 for 97 with 4293 remaining (b = 0.01)
5553.98/707.72	c INFO: running adaptg2wsat2009++@99 for 97 with 4293 remaining (b = 0.01)
5588.14/712.04	c INFO: running hybridGM3@95 for 97 with 4288 remaining (b = 0.01)
6273.28/798.18	c INFO: running hybridGM3@94 for 97 with 4202 remaining (b = 0.01)
6298.71/801.38	c INFO: running hybridGM3@100 for 97 with 4199 remaining (b = 0.01)
6328.15/805.07	c INFO: running hybridGM3@0 for 194 with 4195 remaining (b = 0.01)
6345.65/807.28	c INFO: running adaptg2wsat2009++@295 for 291 with 4193 remaining (b = 0.02)
6352.80/808.16	c INFO: running TNM@0 for 97 with 4192 remaining (b = 0.01)
6360.72/809.14	c INFO: running TNM@0 for 97 with 4191 remaining (b = 0.01)
6394.06/813.40	c INFO: running hybridGM3@291 for 194 with 4187 remaining (b = 0.01)
7031.47/893.53	c INFO: running hybridGM3@291 for 194 with 4107 remaining (b = 0.01)
7104.56/902.74	c INFO: running TNM@0 for 97 with 4097 remaining (b = 0.01)
7150.71/908.54	c INFO: running TNM@0 for 97 with 4092 remaining (b = 0.01)
7157.69/909.53	c INFO: running TNM@0 for 97 with 4091 remaining (b = 0.01)
7159.52/909.81	c INFO: running TNM@0 for 97 with 4090 remaining (b = 0.00)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3385312-1305565050/watcher-3385312-1305565050 -o /tmp/evaluation-result-3385312-1305565050/solver-3385312-1305565050 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385312-1305565050.cnf -seed 1978747136 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 5.98 1.90 0.67 1/348 14277
/proc/meminfo: memFree=19325388/32950928 swapFree=67042944/67111528
[pid=14277] ppid=14275 vsize=15380 CPUtime=0 cores=0-7
/proc/14277/stat : 14277 (python) R 14275 14277 14192 0 -1 4202496 807 0 0 0 0 0 0 0 20 0 1 0 668104754 15749120 693 33554432000 4194304 5683644 140737093460560 18446744073709551615 274473967248 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/14277/statm: 3845 693 332 364 0 332 0

[startup+0.0637119 s]
/proc/loadavg: 5.98 1.90 0.67 1/348 14277
/proc/meminfo: memFree=19325388/32950928 swapFree=67042944/67111528
[pid=14277] ppid=14275 vsize=29784 CPUtime=0.03 cores=0-7
/proc/14277/stat : 14277 (python) R 14275 14277 14192 0 -1 4202496 1459 0 0 0 3 0 0 0 20 0 1 0 668104754 30498816 1299 33554432000 4194304 5683644 140737093460560 18446744073709551615 4873215 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/14277/statm: 7446 1299 443 364 0 816 0
Current children cumulated CPU time (s) 0.03
Current children cumulated vsize (KiB) 29784

[startup+0.1007 s]
/proc/loadavg: 5.98 1.90 0.67 1/348 14277
/proc/meminfo: memFree=19325388/32950928 swapFree=67042944/67111528
[pid=14277] ppid=14275 vsize=87700 CPUtime=0.06 cores=0-7
/proc/14277/stat : 14277 (python) S 14275 14277 14192 0 -1 4202496 2636 0 0 0 4 2 0 0 18 0 1 0 668104754 89804800 2353 33554432000 4194304 5683644 140737093460560 18446744073709551615 274473965568 0 2147483391 16781312 2 18446744071699215550 0 0 17 4 0 0 0
/proc/14277/statm: 21925 2353 717 364 0 1621 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 87700

[startup+0.300669 s]
/proc/loadavg: 5.98 1.90 0.67 1/348 14277
/proc/meminfo: memFree=19325388/32950928 swapFree=67042944/67111528
[pid=14277] ppid=14275 vsize=135744 CPUtime=0.17 cores=0-7
/proc/14277/stat : 14277 (python) R 14275 14277 14192 0 -1 4202496 5965 938 1 0 13 4 0 0 18 0 1 0 668104754 139001856 4150 33554432000 4194304 5683644 140737093460560 18446744073709551615 274473963525 0 2147483391 16781312 2 0 0 0 17 4 0 0 0
/proc/14277/statm: 33936 4150 1032 364 0 3122 0
Current children cumulated CPU time (s) 0.17
Current children cumulated vsize (KiB) 135744

[startup+0.7006 s]
/proc/loadavg: 5.98 1.90 0.67 1/348 14277
/proc/meminfo: memFree=19325388/32950928 swapFree=67042944/67111528
[pid=14277] ppid=14275 vsize=283636 CPUtime=0.43 cores=0-7
/proc/14277/stat : 14277 (python) R 14275 14277 14192 0 -1 4202496 10560 938 1 0 32 11 0 0 17 0 1 0 668104754 290443264 8240 33554432000 4194304 5683644 140737093460560 18446744073709551615 4445115 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/14277/statm: 70909 8240 1761 364 0 6833 0
Current children cumulated CPU time (s) 0.43
Current children cumulated vsize (KiB) 283636

[startup+1.50038 s]
/proc/loadavg: 5.98 1.90 0.67 2/349 14282
/proc/meminfo: memFree=19279836/32950928 swapFree=67042944/67111528
[pid=14277] ppid=14275 vsize=327996 CPUtime=1.22 cores=0-7
/proc/14277/stat : 14277 (python) R 14275 14277 14192 0 -1 4202496 27454 938 1 0 109 13 0 0 18 0 1 0 668104754 335867904 19384 33554432000 4194304 5683644 140737093460560 18446744073709551615 4458886 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/14277/statm: 81999 19384 1826 364 0 17923 0
Current children cumulated CPU time (s) 1.22
Current children cumulated vsize (KiB) 327996

[startup+3.1001 s]
/proc/loadavg: 5.58 1.89 0.67 2/349 14282
/proc/meminfo: memFree=19254044/32950928 swapFree=67042944/67111528
[pid=14277] ppid=14275 vsize=327996 CPUtime=2.82 cores=0-7
/proc/14277/stat : 14277 (python) R 14275 14277 14192 0 -1 4202496 27454 938 1 0 269 13 0 0 21 0 1 0 668104754 335867904 19384 33554432000 4194304 5683644 140737093460560 18446744073709551615 274477376169 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/14277/statm: 81999 19384 1826 364 0 17923 0
Current children cumulated CPU time (s) 2.82
Current children cumulated vsize (KiB) 327996

[startup+6.30054 s]
/proc/loadavg: 5.58 1.89 0.67 2/349 14282
/proc/meminfo: memFree=19254424/32950928 swapFree=67042944/67111528
[pid=14277] ppid=14275 vsize=327996 CPUtime=6.02 cores=0-7
/proc/14277/stat : 14277 (python) R 14275 14277 14192 0 -1 4202496 27455 938 1 0 589 13 0 0 25 0 1 0 668104754 335867904 19385 33554432000 4194304 5683644 140737093460560 18446744073709551615 274477376181 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/14277/statm: 81999 19385 1827 364 0 17923 0
Current children cumulated CPU time (s) 6.02
Current children cumulated vsize (KiB) 327996

[startup+12.7004 s]
/proc/loadavg: 4.87 1.86 0.67 2/349 14282
/proc/meminfo: memFree=19254580/32950928 swapFree=67042944/67111528
[pid=14277] ppid=14275 vsize=471688 CPUtime=12.19 cores=0-7
/proc/14277/stat : 14277 (python) S 14275 14277 14192 0 -1 4202496 33512 938 1 0 1203 16 0 0 25 0 9 0 668104754 483008512 18657 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485794251 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/14277/statm: 117922 18657 1889 364 0 53819 0
Current children cumulated CPU time (s) 12.19
Current children cumulated vsize (KiB) 471688

[startup+25.5012 s]
/proc/loadavg: 5.35 2.06 0.75 9/373 14306
/proc/meminfo: memFree=18932208/32950928 swapFree=67042944/67111528
[pid=14277] ppid=14275 vsize=471688 CPUtime=12.19 cores=0-7
/proc/14277/stat : 14277 (python) S 14275 14277 14192 0 -1 4202496 33512 938 1 0 1203 16 0 0 25 0 9 0 668104754 483008512 18657 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485794251 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/14277/statm: 117922 18657 1889 364 0 53819 0
[pid=14283] ppid=14277 vsize=324116 CPUtime=0.03 cores=0-7
/proc/14283/stat : 14283 (python) S 14277 14277 14192 0 -1 4202560 909 0 0 0 2 1 0 0 15 0 1 0 668105977 331894784 16958 33554432000 4194304 5683644 140737093460560 18446744073709551615 274473987599 0 0 16781312 514 4267512490654400769 0 0 17 1 0 0 0
/proc/14283/statm: 81029 16958 376 364 0 16947 0
[pid=14285] ppid=14283 vsize=276244 CPUtime=13.25 cores=0-7
/proc/14285/stat : 14285 (hybridGM3) R 14283 14285 14285 0 -1 4202496 9511 0 0 0 1324 1 0 0 25 0 1 0 668105977 282873856 9251 33554432000 134512640 135204385 4290849152 18446744073709551615 134537292 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/14285/statm: 69061 9251 64 169 0 68891 0
[pid=14286] ppid=14277 vsize=334372 CPUtime=0.02 cores=0-7
/proc/14286/stat : 14286 (python) S 14277 14277 14192 0 -1 4202560 905 0 0 0 1 1 0 0 15 0 1 0 668105980 342396928 16955 33554432000 4194304 5683644 140737093460560 18446744073709551615 274473987654 0 0 16781312 514 4267512490654400771 0 0 17 4 0 0 0
/proc/14286/statm: 83593 16955 369 364 0 19508 0
[pid=14288] ppid=14286 vsize=276236 CPUtime=13.19 cores=0-7
/proc/14288/stat : 14288 (hybridGM3) R 14286 14288 14288 0 -1 4202496 9517 0 0 0 1318 1 0 0 25 0 1 0 668105980 282865664 9251 33554432000 134512640 135204385 4287307776 18446744073709551615 134536641 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/14288/statm: 69059 9251 64 169 0 68889 0
[pid=14289] ppid=14277 vsize=344628 CPUtime=0.02 cores=0-7
/proc/14289/stat : 14289 (python) S 14277 14277 14192 0 -1 4202560 914 0 0 0 1 1 0 0 15 0 1 0 668105983 352899072 16960 33554432000 4194304 5683644 140737093460560 18446744073709551615 274473987654 0 0 16781312 514 4267512490654400768 0 0 17 1 0 0 0
/proc/14289/statm: 86157 16960 370 364 0 22069 0
[pid=14291] ppid=14289 vsize=276236 CPUtime=13.19 cores=0-7
/proc/14291/stat : 14291 (hybridGM3) R 14289 14291 14291 0 -1 4202496 9519 0 0 0 1317 2 0 0 25 0 1 0 668105984 282865664 9253 33554432000 134512640 135204385 4294810400 18446744073709551615 134537357 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/14291/statm: 69059 9253 64 169 0 68889 0
[pid=14292] ppid=14277 vsize=354884 CPUtime=0.03 cores=0-7
/proc/14292/stat : 14292 (python) S 14277 14277 14192 0 -1 4202560 933 0 0 0 1 2 0 0 15 0 1 0 668105987 363401216 16964 33554432000 4194304 5683644 140737093460560 18446744073709551615 274473987654 0 0 16781312 514 4267512490654400769 0 0 17 7 0 0 0
/proc/14292/statm: 88721 16964 370 364 0 24630 0
[pid=14294] ppid=14292 vsize=276240 CPUtime=13.15 cores=0-7
/proc/14294/stat : 14294 (hybridGM3) R 14292 14294 14294 0 -1 4202496 9512 0 0 0 1314 1 0 0 25 0 1 0 668105987 282869760 9253 33554432000 134512640 135204385 4294563232 18446744073709551615 134536663 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/14294/statm: 69060 9253 64 169 0 68890 0
[pid=14295] ppid=14277 vsize=430676 CPUtime=0.03 cores=0-7
/proc/14295/stat : 14295 (python) S 14277 14277 14192 0 -1 4202560 799 0 0 0 1 2 0 0 15 0 1 0 668105991 441012224 17142 33554432000 4194304 5683644 140737093460560 18446744073709551615 274473987654 0 0 16781312 514 4267512490654400772 0 0 17 1 0 0 0
/proc/14295/statm: 107669 17142 371 364 0 43575 0
[pid=14297] ppid=14295 vsize=276244 CPUtime=13.12 cores=0-7
/proc/14297/stat : 14297 (hybridGM3) R 14295 14297 14297 0 -1 4202496 9502 0 0 0 1311 1 0 0 25 0 1 0 668105991 282873856 9247 33554432000 134512640 135204385 4288997456 18446744073709551615 134537483 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/14297/statm: 69061 9247 64 169 0 68891 0
[pid=14298] ppid=14277 vsize=440932 CPUtime=0.03 cores=0-7
/proc/14298/stat : 14298 (python) S 14277 14277 14192 0 -1 4202560 806 0 0 0 1 2 0 0 15 0 1 0 668105994 451514368 17110 33554432000 4194304 5683644 140737093460560 18446744073709551615 274473987654 0 0 16781312 514 4267512490654400771 0 0 17 1 0 0 0
/proc/14298/statm: 110233 17110 371 364 0 46136 0
[pid=14300] ppid=14298 vsize=276236 CPUtime=12.86 cores=0-7
/proc/14300/stat : 14300 (hybridGM3) R 14298 14300 14300 0 -1 4202496 9506 0 0 0 1284 2 0 0 25 0 1 0 668105994 282865664 9250 33554432000 134512640 135204385 4289498640 18446744073709551615 134546082 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/14300/statm: 69059 9250 64 169 0 68889 0
[pid=14301] ppid=14277 vsize=451188 CPUtime=0.03 cores=0-7
/proc/14301/stat : 14301 (python) S 14277 14277 14192 0 -1 4202560 825 0 0 0 1 2 0 0 15 0 1 0 668105997 462016512 17115 33554432000 4194304 5683644 140737093460560 18446744073709551615 274473987654 0 0 16781312 514 4267512490654400772 0 0 17 1 0 0 0
/proc/14301/statm: 112797 17115 371 364 0 48697 0
[pid=14303] ppid=14301 vsize=276240 CPUtime=12.8 cores=0-7
/proc/14303/stat : 14303 (hybridGM3) R 14301 14303 14303 0 -1 4202496 9518 0 0 0 1279 1 0 0 25 0 1 0 668106000 282869760 9259 33554432000 134512640 135204385 4287051200 18446744073709551615 134537565 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/14303/statm: 69060 9259 64 169 0 68890 0
[pid=14304] ppid=14277 vsize=461444 CPUtime=0.03 cores=0-7
/proc/14304/stat : 14304 (python) S 14277 14277 14192 0 -1 4202560 865 0 0 0 1 2 0 0 15 0 1 0 668106000 472518656 17141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274473987654 0 0 16781312 514 4267512490654400770 0 0 17 1 0 0 0
/proc/14304/statm: 115361 17141 377 364 0 51258 0
[pid=14306] ppid=14304 vsize=276240 CPUtime=13 cores=0-7
/proc/14306/stat : 14306 (hybridGM3) R 14304 14306 14306 0 -1 4202496 9516 0 0 0 1299 1 0 0 25 0 1 0 668106002 282869760 9252 33554432000 134512640 135204385 4290974320 18446744073709551615 134547654 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/14306/statm: 69060 9252 64 169 0 68890 0
[pid=14277/tid=14284] ppid=14275 vsize=471688 CPUtime=0 cores=0-7
/proc/14277/task/14284/stat : 14284 (python) S 14275 14277 14192 0 -1 4202560 27 938 0 0 0 0 0 0 25 0 9 0 668105977 483008512 18657 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 0 0 0 0
[pid=14277/tid=14287] ppid=14275 vsize=471688 CPUtime=0 cores=0-7
/proc/14277/task/14287/stat : 14287 (python) S 14275 14277 14192 0 -1 4202560 22 938 0 0 0 0 0 0 18 0 9 0 668105980 483008512 18657 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 4 0 0 0
[pid=14277/tid=14290] ppid=14275 vsize=471688 CPUtime=0 cores=0-7
/proc/14277/task/14290/stat : 14290 (python) S 14275 14277 14192 0 -1 4202560 23 938 0 0 0 0 0 0 21 0 9 0 668105984 483008512 18657 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 6 0 0 0
[pid=14277/tid=14293] ppid=14275 vsize=471688 CPUtime=0 cores=0-7
/proc/14277/task/14293/stat : 14293 (python) S 14275 14277 14192 0 -1 4202560 25 938 0 0 0 0 0 0 18 0 9 0 668105987 483008512 18657 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 5 0 0 0
[pid=14277/tid=14296] ppid=14275 vsize=471688 CPUtime=0 cores=0-7
/proc/14277/task/14296/stat : 14296 (python) S 14275 14277 14192 0 -1 4202560 93 938 0 0 0 0 0 0 23 0 9 0 668105991 483008512 18657 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 2 0 0 0
[pid=14277/tid=14299] ppid=14275 vsize=471688 CPUtime=0 cores=0-7
/proc/14277/task/14299/stat : 14299 (python) S 14275 14277 14192 0 -1 4202560 31 938 0 0 0 0 0 0 24 0 9 0 668105994 483008512 18657 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 7 0 0 0
[pid=14277/tid=14302] ppid=14275 vsize=471688 CPUtime=0 cores=0-7
/proc/14277/task/14302/stat : 14302 (python) S 14275 14277 14192 0 -1 4202560 34 938 0 0 0 0 0 0 25 0 9 0 668105997 483008512 18657 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 1 0 0 0
[pid=14277/tid=14305] ppid=14275 vsize=471688 CPUtime=0 cores=0-7
/proc/14277/task/14305/stat : 14305 (python) S 14275 14277 14192 0 -1 4202560 33 938 0 0 0 0 0 0 25 0 9 0 668106001 483008512 18657 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 116.97
Current children cumulated vsize (KiB) 5823844

[startup+51.1017 s]
/proc/loadavg: 6.34 2.55 0.95 9/373 14307
/proc/meminfo: memFree=18932200/32950928 swapFree=67042944/67111528
[pid=14277] ppid=14275 vsize=471688 CPUtime=12.19 cores=0-7
/proc/14277/stat : 14277 (python) S 14275 14277 14192 0 -1 4202496 33512 938 1 0 1203 16 0 0 25 0 9 0 668104754 483008512 18657 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485794251 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/14277/statm: 117922 18657 1889 364 0 53819 0
[pid=14283] ppid=14277 vsize=324116 CPUtime=0.1 cores=0-7
/proc/14283/stat : 14283 (python) S 14277 14277 14192 0 -1 4202560 915 0 0 0 5 5 0 0 15 0 1 0 668105977 331894784 16958 33554432000 4194304 5683644 140737093460560 18446744073709551615 274473987599 0 0 16781312 514 4267512490654400769 0 0 17 1 0 0 0
/proc/14283/statm: 81029 16958 376 364 0 16947 0
[pid=14285] ppid=14283 vsize=276244 CPUtime=38.85 cores=0-7
/proc/14285/stat : 14285 (hybridGM3) R 14283 14285 14285 0 -1 4202496 9512 0 0 0 3884 1 0 0 25 0 1 0 668105977 282873856 9252 33554432000 134512640 135204385 4290849152 18446744073709551615 134536698 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/14285/statm: 69061 9252 64 169 0 68891 0
[pid=14286] ppid=14277 vsize=334372 CPUtime=0.09 cores=0-7
/proc/14286/stat : 14286 (python) S 14277 14277 14192 0 -1 4202560 916 0 0 0 5 4 0 0 15 0 1 0 668105980 342396928 16956 33554432000 4194304 5683644 140737093460560 18446744073709551615 274473987654 0 0 16781312 514 4267512490654400771 0 0 17 4 0 0 0
/proc/14286/statm: 83593 16956 370 364 0 19508 0
[pid=14288] ppid=14286 vsize=276236 CPUtime=38.73 cores=0-7
/proc/14288/stat : 14288 (hybridGM3) R 14286 14288 14288 0 -1 4202496 9517 0 0 0 3872 1 0 0 25 0 1 0 668105980 282865664 9251 33554432000 134512640 135204385 4287307776 18446744073709551615 134547654 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/14288/statm: 69059 9251 64 169 0 68889 0
[pid=14289] ppid=14277 vsize=344628 CPUtime=0.09 cores=0-7
/proc/14289/stat : 14289 (python) S 14277 14277 14192 0 -1 4202560 922 0 0 0 5 4 0 0 15 0 1 0 668105983 352899072 16960 33554432000 4194304 5683644 140737093460560 18446744073709551615 274473987654 0 0 16781312 514 4267512490654400768 0 0 17 1 0 0 0
/proc/14289/statm: 86157 16960 370 364 0 22069 0
[pid=14291] ppid=14289 vsize=276236 CPUtime=38.78 cores=0-7
/proc/14291/stat : 14291 (hybridGM3) R 14289 14291 14291 0 -1 4202496 9519 0 0 0 3876 2 0 0 25 0 1 0 668105984 282865664 9253 33554432000 134512640 135204385 4294810400 18446744073709551615 134536698 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/14291/statm: 69059 9253 64 169 0 68889 0
[pid=14292] ppid=14277 vsize=354884 CPUtime=0.09 cores=0-7
/proc/14292/stat : 14292 (python) S 14277 14277 14192 0 -1 4202560 945 0 0 0 4 5 0 0 15 0 1 0 668105987 363401216 16964 33554432000 4194304 5683644 140737093460560 18446744073709551615 274473987654 0 0 16781312 514 4267512490654400769 0 0 17 7 0 0 0
/proc/14292/statm: 88721 16964 370 364 0 24630 0

################
# More data... #
################

/proc/14393/task/14423/stat : 14423 (python) S 14277 14277 14192 0 -1 4202560 84 0 0 0 0 0 0 0 15 0 2 0 668184565 630050816 17505 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 514 18446604454463664576 0 0 -1 1 0 0 0
[pid=14396] ppid=14277 vsize=625540 CPUtime=0.91 cores=0-7
/proc/14396/stat : 14396 (python) S 14277 14277 14192 0 -1 4202560 1189 0 0 0 41 50 0 0 18 0 2 0 668164899 640552960 17554 33554432000 4194304 5683644 140737093460560 18446744073709551615 274473987654 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/14396/statm: 156385 17554 406 364 0 92234 0
[pid=14398] ppid=14396 vsize=438604 CPUtime=299.57 cores=0-7
/proc/14398/stat : 14398 (adaptg2wsat2009) R 14396 14398 14398 0 -1 4202496 8781 0 0 0 29955 2 0 0 25 0 1 0 668164900 449130496 8497 33554432000 134512640 135107411 4287851024 18446744073709551615 134531428 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/14398/statm: 109651 8497 59 146 0 108527 0
[pid=14396/tid=14413] ppid=14277 vsize=625540 CPUtime=0 cores=0-7
/proc/14396/task/14413/stat : 14413 (python) S 14277 14277 14192 0 -1 4202560 87 0 0 0 0 0 0 0 18 0 2 0 668174930 640552960 17554 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 514 18446604454463665472 0 0 -1 4 0 0 0
[pid=14400] ppid=14277 vsize=635796 CPUtime=0.89 cores=0-7
/proc/14400/stat : 14400 (python) S 14277 14277 14192 0 -1 4202560 1200 0 0 0 44 45 0 0 16 0 2 0 668165200 651055104 17523 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485794251 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/14400/statm: 158949 17523 406 364 0 94795 0
[pid=14402] ppid=14400 vsize=276240 CPUtime=296.17 cores=0-7
/proc/14402/stat : 14402 (hybridGM3) T 14400 14402 14402 0 -1 4202496 9547 0 0 0 29615 2 0 0 25 0 1 0 668165201 282869760 9262 33554432000 134512640 135204385 4292600592 18446744073709551615 134546082 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/14402/statm: 69060 9262 64 169 0 68890 0
[pid=14400/tid=14424] ppid=14277 vsize=635796 CPUtime=0 cores=0-7
/proc/14400/task/14424/stat : 14424 (python) S 14277 14277 14192 0 -1 4202560 85 0 0 0 0 0 0 0 15 0 2 0 668184885 651055104 17523 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 514 18446604458174697664 0 0 -1 1 0 0 0
[pid=14404] ppid=14277 vsize=646052 CPUtime=0.6 cores=0-7
/proc/14404/stat : 14404 (python) S 14277 14277 14192 0 -1 4202560 1184 0 0 0 33 27 0 0 19 0 2 0 668165402 661557248 17572 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485794251 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/14404/statm: 161513 17572 401 364 0 97356 0
[pid=14406] ppid=14404 vsize=438604 CPUtime=198.17 cores=0-7
/proc/14406/stat : 14406 (adaptg2wsat2009) T 14404 14406 14406 0 -1 4202496 8784 0 0 0 19815 2 0 0 25 0 1 0 668165402 449130496 8498 33554432000 134512640 135107411 4293309616 18446744073709551615 134531239 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/14406/statm: 109651 8498 59 146 0 108527 0
[pid=14404/tid=14432] ppid=14277 vsize=646052 CPUtime=0 cores=0-7
/proc/14404/task/14432/stat : 14432 (python) S 14277 14277 14192 0 -1 4202560 87 0 0 0 0 0 0 0 19 0 2 0 668185661 661557248 17572 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 514 18446604470243937088 0 0 -1 5 0 0 0
[pid=14408] ppid=14277 vsize=656308 CPUtime=0.3 cores=0-7
/proc/14408/stat : 14408 (python) S 14277 14277 14192 0 -1 4202560 1188 0 0 0 18 12 0 0 19 0 2 0 668165489 672059392 17536 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485794251 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/14408/statm: 164077 17536 400 364 0 99917 0
[pid=14410] ppid=14408 vsize=471116 CPUtime=98.57 cores=0-7
/proc/14410/stat : 14410 (TNM) T 14408 14410 14410 0 -1 4202496 8452 0 0 0 9856 1 0 0 25 0 1 0 668165490 482422784 8172 33554432000 134512640 135008748 4293026784 18446744073709551615 134527542 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/14410/statm: 117779 8172 46 122 0 116679 0
[pid=14408/tid=14419] ppid=14277 vsize=656308 CPUtime=0 cores=0-7
/proc/14408/task/14419/stat : 14419 (python) S 14277 14277 14192 0 -1 4202560 86 0 0 0 0 0 0 0 19 0 2 0 668175519 672059392 17536 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 514 18446604467719475520 0 0 -1 0 0 0 0
[pid=14414] ppid=14277 vsize=666564 CPUtime=0.32 cores=0-7
/proc/14414/stat : 14414 (python) S 14277 14277 14192 0 -1 4202560 1202 0 0 0 17 15 0 0 19 0 2 0 668174938 682561536 17595 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485794251 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/14414/statm: 166641 17595 401 364 0 102478 0
[pid=14416] ppid=14414 vsize=471116 CPUtime=100.92 cores=0-7
/proc/14416/stat : 14416 (TNM) T 14414 14416 14416 0 -1 4202496 8453 0 0 0 10090 2 0 0 25 0 1 0 668174939 482422784 8170 33554432000 134512640 135008748 4294455184 18446744073709551615 134524819 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/14416/statm: 117779 8170 46 122 0 116679 0
[pid=14414/tid=14428] ppid=14277 vsize=666564 CPUtime=0 cores=0-7
/proc/14414/task/14428/stat : 14428 (python) S 14277 14277 14192 0 -1 4202560 87 0 0 0 0 0 0 0 19 0 2 0 668185475 682561536 17595 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 514 18446604468260510720 0 0 -1 7 0 0 0
[pid=14425] ppid=14277 vsize=676820 CPUtime=0.3 cores=0-7
/proc/14425/stat : 14425 (python) S 14277 14277 14192 0 -1 4202560 1090 0 0 0 15 15 0 0 15 0 1 0 668185261 693063680 17543 33554432000 4194304 5683644 140737093460560 18446744073709551615 274473987654 0 0 16781312 514 4267512490654400773 0 0 17 4 0 0 0
/proc/14425/statm: 169205 17543 378 364 0 105039 0
[pid=14427] ppid=14425 vsize=276236 CPUtime=104.46 cores=0-7
/proc/14427/stat : 14427 (hybridGM3) R 14425 14427 14427 0 -1 4202496 9538 0 0 0 10444 2 0 0 25 0 1 0 668185262 282865664 9253 33554432000 134512640 135204385 4289280976 18446744073709551615 134536993 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/14427/statm: 69059 9253 64 169 0 68889 0
[pid=14429] ppid=14277 vsize=687076 CPUtime=0.31 cores=0-7
/proc/14429/stat : 14429 (python) S 14277 14277 14192 0 -1 4202560 1243 0 0 0 15 16 0 0 19 0 2 0 668185570 703565824 17694 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485794251 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/14429/statm: 171769 17694 407 364 0 107600 0
[pid=14431] ppid=14429 vsize=471116 CPUtime=100.29 cores=0-7
/proc/14431/stat : 14431 (TNM) T 14429 14431 14431 0 -1 4202496 8461 0 0 0 10027 2 0 0 25 0 1 0 668185571 482422784 8172 33554432000 134512640 135008748 4293877296 18446744073709551615 134527089 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/14431/statm: 117779 8172 46 122 0 116679 0
[pid=14429/tid=14442] ppid=14277 vsize=687076 CPUtime=0 cores=0-7
/proc/14429/task/14442/stat : 14442 (python) S 14277 14277 14192 0 -1 4202560 90 0 0 0 0 0 0 0 19 0 2 0 668195601 703565824 17694 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 514 18446604466789848384 0 0 -1 5 0 0 0
[pid=14433] ppid=14277 vsize=697332 CPUtime=0.31 cores=0-7
/proc/14433/stat : 14433 (python) S 14277 14277 14192 0 -1 4202560 1254 0 0 0 17 14 0 0 19 0 2 0 668185668 714067968 17643 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485794251 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/14433/statm: 174333 17643 407 364 0 110161 0
[pid=14435] ppid=14433 vsize=471116 CPUtime=100.16 cores=0-7
/proc/14435/stat : 14435 (TNM) T 14433 14435 14435 0 -1 4202496 8460 0 0 0 10015 1 0 0 25 0 1 0 668185668 482422784 8170 33554432000 134512640 135008748 4287699264 18446744073709551615 134527542 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/14435/statm: 117779 8170 46 122 0 116679 0
[pid=14433/tid=14446] ppid=14277 vsize=697332 CPUtime=0 cores=0-7
/proc/14433/task/14446/stat : 14446 (python) S 14277 14277 14192 0 -1 4202560 89 0 0 0 0 0 0 0 19 0 2 0 668195699 714067968 17643 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 514 18446604466789847488 0 0 -1 6 0 0 0
[pid=14439] ppid=14277 vsize=707588 CPUtime=0.01 cores=0-7
/proc/14439/stat : 14439 (python) S 14277 14277 14192 0 -1 4202560 1065 0 0 0 0 1 0 0 18 0 1 0 668195028 724570112 17562 33554432000 4194304 5683644 140737093460560 18446744073709551615 274473987654 0 0 16781312 514 650145352967487751 0 0 17 1 0 0 0
/proc/14439/statm: 176897 17562 379 364 0 112722 0
[pid=14441] ppid=14439 vsize=471116 CPUtime=6.77 cores=0-7
/proc/14441/stat : 14441 (TNM) R 14439 14441 14441 0 -1 4202496 8465 0 0 0 676 1 0 0 25 0 1 0 668195029 482422784 8172 33554432000 134512640 135008748 4294318464 18446744073709551615 134528942 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/14441/statm: 117779 8172 46 122 0 116679 0
[pid=14443] ppid=14277 vsize=717844 CPUtime=0 cores=0-7
/proc/14443/stat : 14443 (python) S 14277 14277 14192 0 -1 4202560 955 0 0 0 0 0 0 0 18 0 1 0 668195608 735072256 17670 33554432000 4194304 5683644 140737093460560 18446744073709551615 274473987654 0 0 16781312 514 2343899859725058047 0 0 17 1 0 0 0
/proc/14443/statm: 179461 17670 355 364 0 115283 0
[pid=14445] ppid=14443 vsize=471116 CPUtime=1.01 cores=0-7
/proc/14445/stat : 14445 (TNM) R 14443 14445 14445 0 -1 4202496 8462 0 0 0 100 1 0 0 24 0 1 0 668195609 482422784 8172 33554432000 134512640 135008748 4290189392 18446744073709551615 134528786 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/14445/statm: 117779 8172 46 122 0 116679 0
[pid=14447] ppid=14277 vsize=728100 CPUtime=0 cores=0-7
/proc/14447/stat : 14447 (python) S 14277 14277 14192 0 -1 4202560 922 0 0 0 0 0 0 0 19 0 1 0 668195707 745574400 17541 33554432000 4194304 5683644 140737093460560 18446744073709551615 274473987654 0 0 16781312 514 4587835691370545151 0 0 17 1 0 0 0
/proc/14447/statm: 182025 17541 346 364 0 117844 0
[pid=14449] ppid=14447 vsize=453300 CPUtime=0.08 cores=0-7
/proc/14449/stat : 14449 (TNM) R 14447 14449 14449 0 -1 4202496 1255 0 0 0 8 0 0 0 21 0 1 0 668195708 464179200 968 33554432000 134512640 135008748 4290957456 18446744073709551615 134580187 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/14449/statm: 113325 968 45 122 0 112225 0
[pid=14277/tid=14284] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14284/stat : 14284 (python) S 14275 14277 14192 0 -1 4202560 37 938 0 0 0 0 0 0 15 0 35 0 668105977 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 0 0 0 0
[pid=14277/tid=14287] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14287/stat : 14287 (python) S 14275 14277 14192 0 -1 4202560 28 938 0 0 0 0 0 0 15 0 35 0 668105980 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 4 0 0 0
[pid=14277/tid=14290] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14290/stat : 14290 (python) S 14275 14277 14192 0 -1 4202560 28 938 0 0 0 0 0 0 18 0 35 0 668105984 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 6 0 0 0
[pid=14277/tid=14293] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14293/stat : 14293 (python) S 14275 14277 14192 0 -1 4202560 49 938 0 0 0 0 0 0 15 0 35 0 668105987 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 5 0 0 0
[pid=14277/tid=14296] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14296/stat : 14296 (python) S 14275 14277 14192 0 -1 4202560 103 938 0 0 0 0 0 0 17 0 35 0 668105991 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 0 0 0 0
[pid=14277/tid=14299] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14299/stat : 14299 (python) S 14275 14277 14192 0 -1 4202560 43 938 0 0 0 0 0 0 18 0 35 0 668105994 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 7 0 0 0
[pid=14277/tid=14302] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14302/stat : 14302 (python) S 14275 14277 14192 0 -1 4202560 45 938 0 0 0 0 0 0 18 0 35 0 668105997 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 1 0 0 0
[pid=14277/tid=14305] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14305/stat : 14305 (python) S 14275 14277 14192 0 -1 4202560 51 938 0 0 0 0 0 0 18 0 35 0 668106001 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 1 0 0 0
[pid=14277/tid=14318] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14318/stat : 14318 (python) S 14275 14277 14192 0 -1 4202560 45 938 0 0 0 0 0 0 20 0 35 0 668125559 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 1 0 0 0
[pid=14277/tid=14337] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14337/stat : 14337 (python) S 14275 14277 14192 0 -1 4202560 51 938 0 0 0 0 0 0 18 0 35 0 668135679 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 6 0 0 0
[pid=14277/tid=14340] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14340/stat : 14340 (python) S 14275 14277 14192 0 -1 4202560 34 938 0 0 0 0 0 0 21 0 35 0 668135684 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 6 0 0 0
[pid=14277/tid=14343] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14343/stat : 14343 (python) S 14275 14277 14192 0 -1 4202560 41 938 0 0 0 0 0 0 18 0 35 0 668135691 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 6 0 0 0
[pid=14277/tid=14346] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14346/stat : 14346 (python) S 14275 14277 14192 0 -1 4202560 37 938 0 0 0 0 0 0 23 0 35 0 668135701 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 1 0 0 0
[pid=14277/tid=14354] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14354/stat : 14354 (python) S 14275 14277 14192 0 -1 4202560 41 938 0 0 0 0 0 0 20 0 35 0 668145328 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 1 0 0 0
[pid=14277/tid=14359] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14359/stat : 14359 (python) S 14275 14277 14192 0 -1 4202560 37 938 0 0 0 0 0 0 20 0 35 0 668145715 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 1 0 0 0
[pid=14277/tid=14363] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14363/stat : 14363 (python) S 14275 14277 14192 0 -1 4202560 34 938 0 0 0 0 0 0 20 0 35 0 668145735 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 1 0 0 0
[pid=14277/tid=14368] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14368/stat : 14368 (python) S 14275 14277 14192 0 -1 4202560 35 938 0 0 0 0 0 0 20 0 35 0 668154851 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 0 0 0 0
[pid=14277/tid=14371] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14371/stat : 14371 (python) S 14275 14277 14192 0 -1 4202560 35 938 0 0 0 0 0 0 19 0 35 0 668155160 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 1 0 0 0
[pid=14277/tid=14377] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14377/stat : 14377 (python) S 14275 14277 14192 0 -1 4202560 35 938 0 0 0 0 0 0 19 0 35 0 668155362 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 0 0 0 0
[pid=14277/tid=14380] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14380/stat : 14380 (python) S 14275 14277 14192 0 -1 4202560 37 938 0 0 0 0 0 0 20 0 35 0 668155539 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 1 0 0 0
[pid=14277/tid=14384] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14384/stat : 14384 (python) S 14275 14277 14192 0 -1 4202560 42 938 0 0 0 0 0 0 18 0 35 0 668155749 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 1 0 0 0
[pid=14277/tid=14388] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14388/stat : 14388 (python) S 14275 14277 14192 0 -1 4202560 48 938 0 0 0 0 0 0 18 0 35 0 668156272 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 1 0 0 0
[pid=14277/tid=14394] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14394/stat : 14394 (python) S 14275 14277 14192 0 -1 4202560 41 938 0 0 0 0 0 0 18 0 35 0 668164886 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 0 0 0 0
[pid=14277/tid=14397] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14397/stat : 14397 (python) S 14275 14277 14192 0 -1 4202560 44 938 0 0 0 0 0 0 18 0 35 0 668164900 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 1 0 0 0
[pid=14277/tid=14401] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14401/stat : 14401 (python) S 14275 14277 14192 0 -1 4202560 33 938 0 0 0 0 0 0 18 0 35 0 668165201 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 1 0 0 0
[pid=14277/tid=14405] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14405/stat : 14405 (python) S 14275 14277 14192 0 -1 4202560 34 938 0 0 0 0 0 0 19 0 35 0 668165402 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 1 0 0 0
[pid=14277/tid=14409] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14409/stat : 14409 (python) S 14275 14277 14192 0 -1 4202560 33 938 0 0 0 0 0 0 19 0 35 0 668165489 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 1 0 0 0
[pid=14277/tid=14415] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14415/stat : 14415 (python) S 14275 14277 14192 0 -1 4202560 34 938 0 0 0 0 0 0 20 0 35 0 668174939 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 0 0 0 0
[pid=14277/tid=14426] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14426/stat : 14426 (python) S 14275 14277 14192 0 -1 4202560 38 938 0 0 0 0 0 0 20 0 35 0 668185262 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 1 0 0 0
[pid=14277/tid=14430] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14430/stat : 14430 (python) S 14275 14277 14192 0 -1 4202560 34 938 0 0 0 0 0 0 20 0 35 0 668185571 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 1 0 0 0
[pid=14277/tid=14434] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14434/stat : 14434 (python) S 14275 14277 14192 0 -1 4202560 32 938 0 0 0 0 0 0 20 0 35 0 668185668 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 1 0 0 0
[pid=14277/tid=14440] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14440/stat : 14440 (python) S 14275 14277 14192 0 -1 4202560 37 938 0 0 0 0 0 0 20 0 35 0 668195028 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 0 0 0 0
[pid=14277/tid=14444] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14444/stat : 14444 (python) S 14275 14277 14192 0 -1 4202560 34 938 0 0 0 0 0 0 20 0 35 0 668195609 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 1 0 0 0
[pid=14277/tid=14448] ppid=14275 vsize=738344 CPUtime=0 cores=0-7
/proc/14277/task/14448/stat : 14448 (python) S 14275 14277 14192 0 -1 4202560 34 938 0 0 0 0 0 0 20 0 35 0 668195707 756064256 19141 33554432000 4194304 5683644 140737093460560 18446744073709551615 274485791153 0 0 16781312 2 18446604449906486272 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 7159.52
Current children cumulated vsize (KiB) 32565716

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 14277 and gives
#  childrusage.ru_utime.tv_sec=1226
#  childrusage.ru_utime.tv_usec=781500
#  childrusage.ru_stime.tv_sec=11
#  childrusage.ru_stime.tv_usec=175301
# CPU time returned by wait4() is 1237.96
# while last known CPU time is 7159.52
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 909.818
CPU time (s): 7159.52
CPU user time (s): 7148.26
CPU system time (s): 11.26
CPU usage (%): 786.917
Max. virtual memory (cumulated for all children) (KiB): 32565716

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1226.78
system time used= 11.1753
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 168350
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 11756
involuntary context switches= 6611

runsolver used 8.06177 second user time and 20.5549 second system time

The end

Launcher Data 

Begin job on node125 at 2011-05-16 18:57:31
IDJOB=3385312
IDBENCH=82577
IDSOLVER=1870
FILE ID=node125/3385312-1305565050
RUNJOBID= node125-1305564963-14206
PBS_JOBID= 13350125
Free space on /tmp= 73252 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/large/unif-k3-r4.2-v50000-c210000-S746725327-020-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3385312-1305565050/watcher-3385312-1305565050 -o /tmp/evaluation-result-3385312-1305565050/solver-3385312-1305565050 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385312-1305565050.cnf -seed 1978747136 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 0ad997eea0def3552b6d7e45aac8ca7c
RANDOM SEED=1978747136

node125.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5266.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      19325668 kB
Buffers:        199612 kB
Cached:        1924112 kB
SwapCached:      55652 kB
Active:        6353368 kB
Inactive:      6944484 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      19325668 kB
SwapTotal:    67111528 kB
SwapFree:     67042944 kB
Dirty:           36104 kB
Writeback:           0 kB
AnonPages:    11118424 kB
Mapped:          35336 kB
Slab:           216332 kB
PageTables:      42888 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 79240136 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73220 MiB
End job on node125 at 2011-05-16 19:12:43