Trace number 3386457

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-seq-random (fixed)? (MO) 3550.33 3539.83

General information on the benchmark

Namerandom/large/
unif-k5-r20-v2000-c40000-S1143439968-082-UNKNOWN.cnf
MD5SUM4baebda329e22fa39a89c4e2a8acff70
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark2363.84
Satisfiable
(Un)Satisfiability was proved
Number of variables2000
Number of clauses40000
Sum of the clauses size200000
Maximum clause length5
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 540000
Number of clauses of size over 50

Solver Data 

0.40/1.11	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.40/1.11	c INFO: solving HOME/instance-3386457-1305586144.cnf
0.40/1.11	c INFO: computing task features
0.62/1.32	c INFO: computed variable-clause graph statistics
0.62/1.32	c INFO: computed clause balance statistics
1.62/2.38	c INFO: computed variable balance statistics
2.22/2.97	c INFO: computed small-clause counts
2.22/2.97	c INFO: computed Horn-clause counts
2.32/3.02	c INFO: computed variable graph statistics
2.32/3.02	c INFO: collected features for HOME/instance-3386457-1305586144.cnf in 1.90 s
2.32/3.04	c INFO: running gnovelty+2@0 for 97 with 4998 remaining (b = 0.38)
101.99/102.40	c INFO: running gnovelty+2@0 for 97 with 4898 remaining (b = 0.13)
201.54/201.77	c INFO: running adaptg2wsat2009++@0 for 97 with 4799 remaining (b = 0.08)
302.26/302.11	c INFO: running adaptg2wsat2009++@0 for 97 with 4699 remaining (b = 0.07)
401.84/401.46	c INFO: running adaptg2wsat2009++@0 for 97 with 4599 remaining (b = 0.05)
501.53/500.82	c INFO: running gnovelty+2@0 for 97 with 4500 remaining (b = 0.05)
602.23/601.20	c INFO: running FH@0 for 97 with 4400 remaining (b = 0.03)
701.80/700.57	c INFO: running adaptg2wsat2009++@0 for 97 with 4300 remaining (b = 0.03)
800.52/798.92	c INFO: running adaptg2wsat2009++@0 for 97 with 4202 remaining (b = 0.02)
900.11/898.29	c INFO: running FH@0 for 97 with 4103 remaining (b = 0.02)
1000.82/998.64	c INFO: running hybridGM3@0 for 194 with 4002 remaining (b = 0.04)
1196.11/1193.41	c INFO: running gnovelty+2@0 for 97 with 3807 remaining (b = 0.02)
1295.94/1292.82	c INFO: running hybridGM3@0 for 194 with 3708 remaining (b = 0.03)
1491.34/1487.61	c INFO: running adaptg2wsat2009++@0 for 97 with 3513 remaining (b = 0.01)
1591.95/1587.99	c INFO: running hybridGM3@0 for 194 with 3413 remaining (b = 0.03)
1788.38/1783.77	c INFO: running hybridGM3@0 for 194 with 3217 remaining (b = 0.02)
1988.83/1983.59	c INFO: running TNM@0 for 1552 with 3017 remaining (b = 0.19)
3549.38/3538.84	c INFO: running FH@0 for 97 with 1462 remaining (b = 0.01)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3386457-1305586144/watcher-3386457-1305586144 -o /tmp/evaluation-result-3386457-1305586144/solver-3386457-1305586144 -C 5000 -W 5100 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3386457-1305586144.cnf -seed 700887766 -budget 5000 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 5030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.00 1.04 2.51 4/392 10289
/proc/meminfo: memFree=21505000/32951124 swapFree=67057896/67111528
[pid=10289] ppid=10287 vsize=15760 CPUtime=0 cores=1,3,5,7
/proc/10289/stat : 10289 (python) S 10287 10289 10255 0 -1 4202496 843 0 0 0 0 0 0 0 17 0 1 0 651029772 16138240 727 33554432000 4194304 5683644 140733828325024 18446744073709551615 254261614592 0 2147483391 16781312 2 18446744071699207358 0 0 17 3 0 0 0
/proc/10289/statm: 3940 727 334 364 0 427 0

[startup+0.036404 s]
/proc/loadavg: 1.00 1.04 2.51 4/392 10289
/proc/meminfo: memFree=21505000/32951124 swapFree=67057896/67111528
[pid=10289] ppid=10287 vsize=16392 CPUtime=0.01 cores=1,3,5,7
/proc/10289/stat : 10289 (python) R 10287 10289 10255 0 -1 4202496 1163 0 0 0 1 0 0 0 17 0 1 0 651029772 16785408 1029 33554432000 4194304 5683644 140733828325024 18446744073709551615 254261614592 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/10289/statm: 4098 1029 414 364 0 585 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 16392

[startup+0.100395 s]
/proc/loadavg: 1.00 1.04 2.51 4/392 10289
/proc/meminfo: memFree=21505000/32951124 swapFree=67057896/67111528
[pid=10289] ppid=10287 vsize=50332 CPUtime=0.06 cores=1,3,5,7
/proc/10289/stat : 10289 (python) R 10287 10289 10255 0 -1 4202496 2002 0 0 0 4 2 0 0 17 0 1 0 651029772 51539968 1764 33554432000 4194304 5683644 140733828325024 18446744073709551615 4509088 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/10289/statm: 12583 1764 480 364 0 1283 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 50332

[startup+0.300372 s]
/proc/loadavg: 1.00 1.04 2.51 4/392 10289
/proc/meminfo: memFree=21505000/32951124 swapFree=67057896/67111528
[pid=10289] ppid=10287 vsize=131292 CPUtime=0.19 cores=1,3,5,7
/proc/10289/stat : 10289 (python) R 10287 10289 10255 0 -1 4202496 5672 930 1 0 12 7 0 0 17 0 1 0 651029772 134443008 4074 33554432000 4194304 5683644 140733828325024 18446744073709551615 254261616384 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/10289/statm: 32823 4079 1015 364 0 3084 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 131292

[startup+0.700328 s]
/proc/loadavg: 1.00 1.04 2.51 4/392 10289
/proc/meminfo: memFree=21505000/32951124 swapFree=67057896/67111528
[pid=10289] ppid=10287 vsize=135748 CPUtime=0.24 cores=1,3,5,7
/proc/10289/stat : 10289 (python) S 10287 10289 10255 0 -1 4202496 5973 930 1 0 15 9 0 0 16 0 1 0 651029772 139005952 4150 33554432000 4194304 5683644 140733828325024 18446744073709551615 254261612549 0 2147483391 16781312 2 18446744071699207358 0 0 17 3 0 0 0
/proc/10289/statm: 33937 4150 1032 364 0 3123 0
Current children cumulated CPU time (s) 0.24
Current children cumulated vsize (KiB) 135748

[startup+1.50016 s]
/proc/loadavg: 1.00 1.04 2.51 4/398 10303
/proc/meminfo: memFree=21464736/32951124 swapFree=67057896/67111528
[pid=10289] ppid=10287 vsize=290796 CPUtime=0.82 cores=1,3,5,7
/proc/10289/stat : 10289 (python) R 10287 10289 10255 0 -1 4202496 14020 930 1 0 67 15 0 0 16 0 1 0 651029772 297775104 10084 33554432000 4194304 5683644 140733828325024 18446744073709551615 47499056711491 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/10289/statm: 72699 10084 1826 364 0 8623 0
Current children cumulated CPU time (s) 0.82
Current children cumulated vsize (KiB) 290796

[startup+3.10086 s]
/proc/loadavg: 1.00 1.04 2.51 4/398 10303
/proc/meminfo: memFree=21437208/32951124 swapFree=67057896/67111528
[pid=10289] ppid=10287 vsize=300280 CPUtime=2.37 cores=1,3,5,7
/proc/10289/stat : 10289 (python) S 10287 10289 10255 0 -1 4202496 14324 930 1 0 222 15 0 0 20 0 2 0 651029772 307486720 9943 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273443275 0 0 16781312 2 18446744073709551615 0 0 17 3 0 0 0
/proc/10289/statm: 75070 9943 1877 364 0 10988 0
Current children cumulated CPU time (s) 2.37
Current children cumulated vsize (KiB) 300280
heavy processes:

[startup+6.30029 s]
/proc/loadavg: 1.16 1.07 2.51 4/404 10309
/proc/meminfo: memFree=21431844/32951124 swapFree=67057896/67111528
[pid=10289] ppid=10287 vsize=300280 CPUtime=2.37 cores=1,3,5,7
/proc/10289/stat : 10289 (python) S 10287 10289 10255 0 -1 4202496 14324 930 1 0 222 15 0 0 20 0 2 0 651029772 307486720 9943 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273443275 0 0 16781312 2 18446744073709551615 0 0 17 3 0 0 0
/proc/10289/statm: 75070 9943 1877 364 0 10988 0
[pid=10307] ppid=10289 vsize=290036 CPUtime=0 cores=1,3,5,7
/proc/10307/stat : 10307 (python) S 10289 10289 10255 0 -1 4202560 736 0 0 0 0 0 0 0 18 0 1 0 651030077 296996864 8415 33554432000 4194304 5683644 140733828325024 18446744073709551615 254261636623 0 0 16781312 514 15546786557791797508 0 0 17 3 0 0 0
/proc/10307/statm: 72509 8415 353 364 0 8427 0
[pid=10309] ppid=10307 vsize=1020308 CPUtime=3.24 cores=1,3,5,7
/proc/10309/stat : 10309 (gnovelty+2) R 10307 10309 10309 0 -1 4202496 1154 0 0 0 324 0 0 0 25 0 1 0 651030077 1044795392 891 33554432000 134512640 135663704 4289232080 18446744073709551615 134525181 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/10309/statm: 255077 891 113 282 0 254793 0
[pid=10289/tid=10308] ppid=10287 vsize=300280 CPUtime=0 cores=1,3,5,7
/proc/10289/task/10308/stat : 10308 (python) S 10287 10289 10255 0 -1 4202560 29 930 0 0 0 0 0 0 20 0 2 0 651030077 307486720 9943 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 2 18446604463253787904 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 5.61
Current children cumulated vsize (KiB) 1610624

[startup+12.7012 s]
/proc/loadavg: 1.31 1.10 2.51 4/404 10309
/proc/meminfo: memFree=21430456/32951124 swapFree=67057896/67111528
[pid=10289] ppid=10287 vsize=300280 CPUtime=2.37 cores=1,3,5,7
/proc/10289/stat : 10289 (python) S 10287 10289 10255 0 -1 4202496 14324 930 1 0 222 15 0 0 20 0 2 0 651029772 307486720 9943 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273443275 0 0 16781312 2 18446744073709551615 0 0 17 3 0 0 0
/proc/10289/statm: 75070 9943 1877 364 0 10988 0
[pid=10307] ppid=10289 vsize=290036 CPUtime=0.01 cores=1,3,5,7
/proc/10307/stat : 10307 (python) S 10289 10289 10255 0 -1 4202560 886 0 0 0 1 0 0 0 18 0 1 0 651030077 296996864 8438 33554432000 4194304 5683644 140733828325024 18446744073709551615 254261636623 0 0 16781312 514 15546786557791797508 0 0 17 3 0 0 0
/proc/10307/statm: 72509 8438 376 364 0 8427 0
[pid=10309] ppid=10307 vsize=1020308 CPUtime=9.64 cores=1,3,5,7
/proc/10309/stat : 10309 (gnovelty+2) R 10307 10309 10309 0 -1 4202496 1154 0 0 0 964 0 0 0 25 0 1 0 651030077 1044795392 891 33554432000 134512640 135663704 4289232080 18446744073709551615 134545619 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/10309/statm: 255077 891 113 282 0 254793 0
[pid=10289/tid=10308] ppid=10287 vsize=300280 CPUtime=0 cores=1,3,5,7
/proc/10289/task/10308/stat : 10308 (python) S 10287 10289 10255 0 -1 4202560 29 930 0 0 0 0 0 0 20 0 2 0 651030077 307486720 9943 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 2 18446604463253787904 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 12.02
Current children cumulated vsize (KiB) 1610624

[startup+25.5009 s]
/proc/loadavg: 1.68 1.20 2.52 4/404 10309
/proc/meminfo: memFree=21431552/32951124 swapFree=67057896/67111528
[pid=10289] ppid=10287 vsize=300280 CPUtime=2.37 cores=1,3,5,7
/proc/10289/stat : 10289 (python) S 10287 10289 10255 0 -1 4202496 14324 930 1 0 222 15 0 0 20 0 2 0 651029772 307486720 9943 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273443275 0 0 16781312 2 18446744073709551615 0 0 17 3 0 0 0
/proc/10289/statm: 75070 9943 1877 364 0 10988 0
[pid=10307] ppid=10289 vsize=290036 CPUtime=0.06 cores=1,3,5,7
/proc/10307/stat : 10307 (python) S 10289 10289 10255 0 -1 4202560 893 0 0 0 3 3 0 0 18 0 1 0 651030077 296996864 8438 33554432000 4194304 5683644 140733828325024 18446744073709551615 254261636623 0 0 16781312 514 4132422094019330308 0 0 17 3 0 0 0
/proc/10307/statm: 72509 8438 376 364 0 8427 0
[pid=10309] ppid=10307 vsize=1020308 CPUtime=22.44 cores=1,3,5,7
/proc/10309/stat : 10309 (gnovelty+2) R 10307 10309 10309 0 -1 4202496 1154 0 0 0 2244 0 0 0 25 0 1 0 651030077 1044795392 891 33554432000 134512640 135663704 4289232080 18446744073709551615 134546755 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/10309/statm: 255077 891 113 282 0 254793 0
[pid=10289/tid=10308] ppid=10287 vsize=300280 CPUtime=0 cores=1,3,5,7
/proc/10289/task/10308/stat : 10308 (python) S 10287 10289 10255 0 -1 4202560 29 930 0 0 0 0 0 0 20 0 2 0 651030077 307486720 9943 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 2 18446604463253787904 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 24.87
Current children cumulated vsize (KiB) 1610624

[startup+51.1003 s]
/proc/loadavg: 2.13 1.34 2.53 4/404 10310
/proc/meminfo: memFree=21431416/32951124 swapFree=67057896/67111528
[pid=10289] ppid=10287 vsize=300280 CPUtime=2.37 cores=1,3,5,7
/proc/10289/stat : 10289 (python) S 10287 10289 10255 0 -1 4202496 14324 930 1 0 222 15 0 0 20 0 2 0 651029772 307486720 9943 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273443275 0 0 16781312 2 18446744073709551615 0 0 17 3 0 0 0
/proc/10289/statm: 75070 9943 1877 364 0 10988 0
[pid=10307] ppid=10289 vsize=290036 CPUtime=0.13 cores=1,3,5,7
/proc/10307/stat : 10307 (python) S 10289 10289 10255 0 -1 4202560 904 0 0 0 6 7 0 0 18 0 1 0 651030077 296996864 8438 33554432000 4194304 5683644 140733828325024 18446744073709551615 254261636623 0 0 16781312 514 4132422094019330308 0 0 17 7 0 0 0
/proc/10307/statm: 72509 8438 376 364 0 8427 0
[pid=10309] ppid=10307 vsize=1020308 CPUtime=48.04 cores=1,3,5,7
/proc/10309/stat : 10309 (gnovelty+2) R 10307 10309 10309 0 -1 4202496 1154 0 0 0 4804 0 0 0 25 0 1 0 651030077 1044795392 891 33554432000 134512640 135663704 4289232080 18446744073709551615 134545635 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/10309/statm: 255077 891 113 282 0 254793 0
[pid=10289/tid=10308] ppid=10287 vsize=300280 CPUtime=0 cores=1,3,5,7
/proc/10289/task/10308/stat : 10308 (python) S 10287 10289 10255 0 -1 4202560 29 930 0 0 0 0 0 0 20 0 2 0 651030077 307486720 9943 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 2 18446604463253787904 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 50.54
Current children cumulated vsize (KiB) 1610624

[startup+102.315 s]
/proc/loadavg: 2.67 1.61 2.56 4/408 10315
/proc/meminfo: memFree=21425728/32951124 swapFree=67057896/67111528
[pid=10289] ppid=10287 vsize=300280 CPUtime=2.37 cores=1,3,5,7
/proc/10289/stat : 10289 (python) S 10287 10289 10255 0 -1 4202496 14324 930 1 0 222 15 0 0 20 0 2 0 651029772 307486720 9943 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273443275 0 0 16781312 2 18446744073709551615 0 0 17 3 0 0 0
/proc/10289/statm: 75070 9943 1877 364 0 10988 0
[pid=10307] ppid=10289 vsize=290036 CPUtime=0.26 cores=1,3,5,7
/proc/10307/stat : 10307 (python) S 10289 10289 10255 0 -1 4202560 921 0 0 0 13 13 0 0 18 0 1 0 651030077 296996864 8438 33554432000 4194304 5683644 140733828325024 18446744073709551615 254261636623 0 0 16781312 514 4132422094019330308 0 0 17 7 0 0 0
/proc/10307/statm: 72509 8438 376 364 0 8427 0
[pid=10309] ppid=10307 vsize=1020308 CPUtime=99.26 cores=1,3,5,7
/proc/10309/stat : 10309 (gnovelty+2) R 10307 10309 10309 0 -1 4202496 1154 0 0 0 9926 0 0 0 25 0 1 0 651030077 1044795392 891 33554432000 134512640 135663704 4289232080 18446744073709551615 134524498 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/10309/statm: 255077 891 113 282 0 254793 0
[pid=10289/tid=10308] ppid=10287 vsize=300280 CPUtime=0 cores=1,3,5,7
/proc/10289/task/10308/stat : 10308 (python) S 10287 10289 10255 0 -1 4202560 29 930 0 0 0 0 0 0 20 0 2 0 651030077 307486720 9943 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 2 18446604463253787904 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 101.89
Current children cumulated vsize (KiB) 1610624

[startup+162.301 s]
/proc/loadavg: 2.98 1.89 2.59 4/412 10321
/proc/meminfo: memFree=21417212/32951124 swapFree=67057896/67111528
[pid=10289] ppid=10287 vsize=310536 CPUtime=2.4 cores=1,3,5,7
/proc/10289/stat : 10289 (python) S 10287 10289 10255 0 -1 4202496 15193 930 1 0 224 16 0 0 20 0 3 0 651029772 317988864 9950 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273443275 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/10289/statm: 77634 9950 1880 364 0 13549 0
[pid=10307] ppid=10289 vsize=300280 CPUtime=0.28 cores=1,3,5,7
/proc/10307/stat : 10307 (python) S 10289 10289 10255 0 -1 4202560 1080 0 0 0 14 14 0 0 19 0 2 0 651030077 307486720 8476 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273443275 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/10307/statm: 75070 8476 409 364 0 10988 0
[pid=10309] ppid=10307 vsize=1020308 CPUtime=99.32 cores=1,3,5,7
/proc/10309/stat : 10309 (gnovelty+2) T 10307 10309 10309 0 -1 4202496 1154 0 0 0 9932 0 0 0 25 0 1 0 651030077 1044795392 891 33554432000 134512640 135663704 4289232080 18446744073709551615 134525255 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/10309/statm: 255077 891 113 282 0 254793 0
[pid=10307/tid=10316] ppid=10289 vsize=300280 CPUtime=0 cores=1,3,5,7
/proc/10307/task/10316/stat : 10316 (python) S 10289 10289 10255 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 651040010 307486720 8476 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 514 18446604460827428864 0 0 -1 7 0 0 0
[pid=10317] ppid=10289 vsize=300292 CPUtime=0.16 cores=1,3,5,7
/proc/10317/stat : 10317 (python) S 10289 10289 10255 0 -1 4202560 916 0 0 0 8 8 0 0 18 0 1 0 651040013 307499008 8436 33554432000 4194304 5683644 140733828325024 18446744073709551615 254261636678 0 0 16781312 514 4132422094019330312 0 0 17 3 0 0 0
/proc/10317/statm: 75073 8436 370 364 0 10988 0
[pid=10319] ppid=10317 vsize=1020308 CPUtime=59.88 cores=1,3,5,7
/proc/10319/stat : 10319 (gnovelty+2) R 10317 10319 10319 0 -1 4202496 1158 0 0 0 5988 0 0 0 25 0 1 0 651040013 1044795392 892 33554432000 134512640 135663704 4290991728 18446744073709551615 134525286 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/10319/statm: 255077 892 113 282 0 254793 0
[pid=10289/tid=10308] ppid=10287 vsize=310536 CPUtime=0 cores=1,3,5,7
/proc/10289/task/10308/stat : 10308 (python) S 10287 10289 10255 0 -1 4202560 29 930 0 0 0 0 0 0 20 0 3 0 651030077 317988864 9950 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 2 18446604463253787904 0 0 -1 7 0 0 0
[pid=10289/tid=10318] ppid=10287 vsize=310536 CPUtime=0 cores=1,3,5,7
/proc/10289/task/10318/stat : 10318 (python) S 10287 10289 10255 0 -1 4202560 28 930 0 0 0 0 0 0 20 0 3 0 651040013 317988864 9950 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 2 18446604463253787904 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 162.04
Current children cumulated vsize (KiB) 2951724

[startup+222.301 s]
/proc/loadavg: 2.99 2.09 2.62 4/420 10330
/proc/meminfo: memFree=21392780/32951124 swapFree=67057896/67111528
[pid=10289] ppid=10287 vsize=320792 CPUtime=2.43 cores=1,3,5,7
/proc/10289/stat : 10289 (python) S 10287 10289 10255 0 -1 4202496 16029 930 1 0 227 16 0 0 20 0 4 0 651029772 328491008 9956 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273443275 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0

################
# More data... #
################

[pid=10339] ppid=10289 vsize=320804 CPUtime=0.3 cores=1,3,5,7
/proc/10339/stat : 10339 (python) S 10289 10289 10255 0 -1 4202560 1103 0 0 0 15 15 0 0 19 0 2 0 651059984 328503296 8474 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273443275 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/10339/statm: 80201 8474 399 364 0 16110 0
[pid=10341] ppid=10339 vsize=423400 CPUtime=99.3 cores=1,3,5,7
/proc/10341/stat : 10341 (adaptg2wsat2009) T 10339 10341 10341 0 -1 4202496 2620 0 0 0 9930 0 0 0 25 0 1 0 651059984 433561600 2356 33554432000 134512640 135107411 4286941504 18446744073709551615 134531896 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/10341/statm: 105850 2356 59 146 0 104726 0
[pid=10339/tid=10348] ppid=10289 vsize=320804 CPUtime=0 cores=1,3,5,7
/proc/10339/task/10348/stat : 10348 (python) S 10289 10289 10255 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 651069915 328503296 8474 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 514 18446604464129140032 0 0 -1 7 0 0 0
[pid=10349] ppid=10289 vsize=331060 CPUtime=0.3 cores=1,3,5,7
/proc/10349/stat : 10349 (python) S 10289 10289 10255 0 -1 4202560 1109 0 0 0 16 14 0 0 19 0 2 0 651069919 339005440 8478 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273443275 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/10349/statm: 82765 8478 399 364 0 18671 0
[pid=10351] ppid=10349 vsize=423400 CPUtime=99.31 cores=1,3,5,7
/proc/10351/stat : 10351 (adaptg2wsat2009) T 10349 10351 10351 0 -1 4202496 2623 0 0 0 9931 0 0 0 25 0 1 0 651069919 433561600 2357 33554432000 134512640 135107411 4293239296 18446744073709551615 134531600 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/10351/statm: 105850 2357 59 146 0 104726 0
[pid=10349/tid=10359] ppid=10289 vsize=331060 CPUtime=0 cores=1,3,5,7
/proc/10349/task/10359/stat : 10359 (python) S 10289 10289 10255 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 651079850 339005440 8478 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 514 18446604458139758656 0 0 -1 3 0 0 0
[pid=10360] ppid=10289 vsize=341316 CPUtime=0.3 cores=1,3,5,7
/proc/10360/stat : 10360 (python) S 10289 10289 10255 0 -1 4202560 1125 0 0 0 15 15 0 0 19 0 2 0 651079854 349507584 8482 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273443275 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/10360/statm: 85329 8482 399 364 0 21232 0
[pid=10362] ppid=10360 vsize=1020308 CPUtime=100.34 cores=1,3,5,7
/proc/10362/stat : 10362 (gnovelty+2) T 10360 10362 10362 0 -1 4202496 1155 0 0 0 10034 0 0 0 25 0 1 0 651079854 1044795392 891 33554432000 134512640 135663704 4290957776 18446744073709551615 134544196 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/10362/statm: 255077 891 113 282 0 254793 0
[pid=10360/tid=10370] ppid=10289 vsize=341316 CPUtime=0 cores=1,3,5,7
/proc/10360/task/10370/stat : 10370 (python) S 10289 10289 10255 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 651089889 349507584 8482 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 514 18446604469049945152 0 0 -1 7 0 0 0
[pid=10371] ppid=10289 vsize=351572 CPUtime=0.3 cores=1,3,5,7
/proc/10371/stat : 10371 (python) S 10289 10289 10255 0 -1 4202560 1179 0 0 0 16 14 0 0 19 0 2 0 651089893 360009728 8492 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273443275 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/10371/statm: 87893 8492 405 364 0 23793 0
[pid=10373] ppid=10371 vsize=253032 CPUtime=99.32 cores=1,3,5,7
/proc/10373/stat : 10373 (FH) T 10371 10373 10373 0 -1 4202496 2503 0 0 0 9932 0 0 0 25 0 1 0 651089893 259104768 2234 33554432000 134512640 134936988 4286948352 18446744073709551615 134553505 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/10373/statm: 63258 2234 41 104 0 62762 0
[pid=10371/tid=10380] ppid=10289 vsize=351572 CPUtime=0 cores=1,3,5,7
/proc/10371/task/10380/stat : 10380 (python) S 10289 10289 10255 0 -1 4202560 94 0 0 0 0 0 0 0 19 0 2 0 651099825 360009728 8492 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 514 18446604471162828608 0 0 -1 7 0 0 0
[pid=10381] ppid=10289 vsize=361828 CPUtime=0.31 cores=1,3,5,7
/proc/10381/stat : 10381 (python) S 10289 10289 10255 0 -1 4202560 1155 0 0 0 17 14 0 0 19 0 2 0 651099829 370511872 8496 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273443275 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/10381/statm: 90457 8496 405 364 0 26354 0
[pid=10383] ppid=10381 vsize=423400 CPUtime=98.3 cores=1,3,5,7
/proc/10383/stat : 10383 (adaptg2wsat2009) T 10381 10383 10383 0 -1 4202496 2630 0 0 0 9830 0 0 0 25 0 1 0 651099829 433561600 2357 33554432000 134512640 135107411 4289299056 18446744073709551615 134537756 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/10383/statm: 105850 2357 59 146 0 104726 0
[pid=10381/tid=10396] ppid=10289 vsize=361828 CPUtime=0 cores=1,3,5,7
/proc/10381/task/10396/stat : 10396 (python) S 10289 10289 10255 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 651109661 370511872 8496 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 514 18446604456718625472 0 0 -1 7 0 0 0
[pid=10397] ppid=10289 vsize=372084 CPUtime=0.31 cores=1,3,5,7
/proc/10397/stat : 10397 (python) S 10289 10289 10255 0 -1 4202560 1145 0 0 0 17 14 0 0 19 0 2 0 651109664 381014016 8494 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273443275 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/10397/statm: 93021 8494 399 364 0 28915 0
[pid=10399] ppid=10397 vsize=423400 CPUtime=99.32 cores=1,3,5,7
/proc/10399/stat : 10399 (adaptg2wsat2009) T 10397 10399 10399 0 -1 4202496 2630 0 0 0 9932 0 0 0 25 0 1 0 651109665 433561600 2357 33554432000 134512640 135107411 4287612464 18446744073709551615 134532371 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/10399/statm: 105850 2357 59 146 0 104726 0
[pid=10397/tid=10406] ppid=10289 vsize=372084 CPUtime=0 cores=1,3,5,7
/proc/10397/task/10406/stat : 10406 (python) S 10289 10289 10255 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 651119597 381014016 8494 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 514 18446604464809334720 0 0 -1 7 0 0 0
[pid=10407] ppid=10289 vsize=382340 CPUtime=0.32 cores=1,3,5,7
/proc/10407/stat : 10407 (python) S 10289 10289 10255 0 -1 4202560 1145 0 0 0 17 15 0 0 19 0 2 0 651119601 391516160 8498 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273443275 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/10407/statm: 95585 8498 399 364 0 31476 0
[pid=10409] ppid=10407 vsize=253028 CPUtime=100.3 cores=1,3,5,7
/proc/10409/stat : 10409 (FH) T 10407 10409 10409 0 -1 4202496 2504 0 0 0 10030 0 0 0 25 0 1 0 651119602 259100672 2233 33554432000 134512640 134936988 4288463552 18446744073709551615 134552486 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/10409/statm: 63257 2233 41 104 0 62761 0
[pid=10407/tid=10417] ppid=10289 vsize=382340 CPUtime=0 cores=1,3,5,7
/proc/10407/task/10417/stat : 10417 (python) S 10289 10289 10255 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 651129633 391516160 8498 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 514 18446604461729875264 0 0 -1 7 0 0 0
[pid=10418] ppid=10289 vsize=392596 CPUtime=0.62 cores=1,3,5,7
/proc/10418/stat : 10418 (python) S 10289 10289 10255 0 -1 4202560 1155 0 0 0 30 32 0 0 19 0 2 0 651129637 402018304 8502 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273443275 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/10418/statm: 98149 8502 399 364 0 34037 0
[pid=10420] ppid=10418 vsize=268424 CPUtime=194.72 cores=1,3,5,7
/proc/10420/stat : 10420 (hybridGM3) T 10418 10420 10420 0 -1 4202496 6193 0 0 0 19472 0 0 0 25 0 1 0 651129637 274866176 5920 33554432000 134512640 135204385 4288439440 18446744073709551615 134538284 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/10420/statm: 67106 5920 63 169 0 66936 0
[pid=10418/tid=10430] ppid=10289 vsize=392596 CPUtime=0 cores=1,3,5,7
/proc/10418/task/10430/stat : 10430 (python) S 10289 10289 10255 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 651149109 402018304 8502 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 514 18446604463769425216 0 0 -1 7 0 0 0
[pid=10431] ppid=10289 vsize=402852 CPUtime=0.31 cores=1,3,5,7
/proc/10431/stat : 10431 (python) S 10289 10289 10255 0 -1 4202560 1147 0 0 0 15 16 0 0 19 0 2 0 651149113 412520448 8506 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273443275 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/10431/statm: 100713 8506 399 364 0 36598 0
[pid=10433] ppid=10431 vsize=1020308 CPUtime=99.37 cores=1,3,5,7
/proc/10433/stat : 10433 (gnovelty+2) T 10431 10433 10433 0 -1 4202496 1165 0 0 0 9937 0 0 0 25 0 1 0 651149114 1044795392 891 33554432000 134512640 135663704 4287369952 18446744073709551615 134545203 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/10433/statm: 255077 891 113 282 0 254793 0
[pid=10431/tid=10441] ppid=10289 vsize=402852 CPUtime=0 cores=1,3,5,7
/proc/10431/task/10441/stat : 10441 (python) S 10289 10289 10255 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 651159051 412520448 8506 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 514 18446604469258916032 0 0 -1 7 0 0 0
[pid=10442] ppid=10289 vsize=413108 CPUtime=0.62 cores=1,3,5,7
/proc/10442/stat : 10442 (python) S 10289 10289 10255 0 -1 4202560 1160 0 0 0 31 31 0 0 19 0 2 0 651159055 423022592 8510 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273443275 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/10442/statm: 103277 8510 399 364 0 39159 0
[pid=10444] ppid=10442 vsize=268428 CPUtime=194.73 cores=1,3,5,7
/proc/10444/stat : 10444 (hybridGM3) T 10442 10444 10444 0 -1 4202496 6191 0 0 0 19473 0 0 0 25 0 1 0 651159055 274870272 5919 33554432000 134512640 135204385 4290877424 18446744073709551615 134537792 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/10444/statm: 67107 5919 63 169 0 66937 0
[pid=10442/tid=10454] ppid=10289 vsize=413108 CPUtime=0 cores=1,3,5,7
/proc/10442/task/10454/stat : 10454 (python) S 10289 10289 10255 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 651178529 423022592 8510 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 514 18446604469258917824 0 0 -1 7 0 0 0
[pid=10455] ppid=10289 vsize=423364 CPUtime=0.32 cores=1,3,5,7
/proc/10455/stat : 10455 (python) S 10289 10289 10255 0 -1 4202560 1142 0 0 0 17 15 0 0 19 0 2 0 651178533 433524736 8514 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273443275 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/10455/statm: 105841 8514 399 364 0 41720 0
[pid=10457] ppid=10455 vsize=423400 CPUtime=100.33 cores=1,3,5,7
/proc/10457/stat : 10457 (adaptg2wsat2009) T 10455 10457 10457 0 -1 4202496 2630 0 0 0 10033 0 0 0 25 0 1 0 651178534 433561600 2357 33554432000 134512640 135107411 4294230336 18446744073709551615 134539290 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/10457/statm: 105850 2357 59 146 0 104726 0
[pid=10455/tid=10465] ppid=10289 vsize=423364 CPUtime=0 cores=1,3,5,7
/proc/10455/task/10465/stat : 10465 (python) S 10289 10289 10255 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 651188567 433524736 8514 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 514 18446604469115998464 0 0 -1 7 0 0 0
[pid=10466] ppid=10289 vsize=433620 CPUtime=0.65 cores=1,3,5,7
/proc/10466/stat : 10466 (python) S 10289 10289 10255 0 -1 4202560 1202 0 0 0 31 34 0 0 19 0 2 0 651188571 444026880 8525 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273443275 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/10466/statm: 108405 8525 406 364 0 44281 0
[pid=10468] ppid=10466 vsize=268420 CPUtime=195.73 cores=1,3,5,7
/proc/10468/stat : 10468 (hybridGM3) T 10466 10468 10468 0 -1 4202496 6193 0 0 0 19573 0 0 0 25 0 1 0 651188571 274862080 5918 33554432000 134512640 135204385 4294323968 18446744073709551615 134537724 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/10468/statm: 67105 5918 63 169 0 66935 0
[pid=10466/tid=10478] ppid=10289 vsize=433620 CPUtime=0 cores=1,3,5,7
/proc/10466/task/10478/stat : 10478 (python) S 10289 10289 10255 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 651208146 444026880 8525 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 514 18446604469116000256 0 0 -1 7 0 0 0
[pid=10479] ppid=10289 vsize=443876 CPUtime=0.68 cores=1,3,5,7
/proc/10479/stat : 10479 (python) S 10289 10289 10255 0 -1 4202560 1211 0 0 0 34 34 0 0 19 0 2 0 651208149 454529024 8529 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273443275 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/10479/statm: 110969 8529 406 364 0 46842 0
[pid=10481] ppid=10479 vsize=268420 CPUtime=199.77 cores=1,3,5,7
/proc/10481/stat : 10481 (hybridGM3) T 10479 10481 10481 0 -1 4202496 6192 0 0 0 19977 0 0 0 25 0 1 0 651208150 274862080 5919 33554432000 134512640 135204385 4294452224 18446744073709551615 134538432 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/10481/statm: 67105 5919 63 169 0 66935 0
[pid=10479/tid=10487] ppid=10289 vsize=443876 CPUtime=0 cores=1,3,5,7
/proc/10479/task/10487/stat : 10487 (python) S 10289 10289 10255 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 651228128 454529024 8529 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 514 18446604464301736256 0 0 -1 7 0 0 0
[pid=10488] ppid=10289 vsize=454132 CPUtime=5.22 cores=1,3,5,7
/proc/10488/stat : 10488 (python) S 10289 10289 10255 0 -1 4202560 1196 0 0 0 266 256 0 0 19 0 2 0 651228131 465031168 8527 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273443275 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/10488/statm: 113533 8527 400 364 0 49403 0
[pid=10490] ppid=10488 vsize=461508 CPUtime=1555.26 cores=1,3,5,7
/proc/10490/stat : 10490 (TNM) T 10488 10490 10490 0 -1 4202496 2471 0 0 0 155526 0 0 0 25 0 1 0 651228132 472584192 2195 33554432000 134512640 135008748 4292804224 18446744073709551615 134528657 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/10490/statm: 115377 2195 46 122 0 114277 0
[pid=10488/tid=10689] ppid=10289 vsize=454132 CPUtime=0 cores=1,3,5,7
/proc/10488/task/10689/stat : 10689 (python) S 10289 10289 10255 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 651383655 465031168 8527 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 514 18446604464301738048 0 0 -1 7 0 0 0
[pid=10690] ppid=10289 vsize=464388 CPUtime=0 cores=1,3,5,7
/proc/10690/stat : 10690 (python) S 10289 10289 10255 0 -1 4202560 823 0 0 0 0 0 0 0 19 0 1 0 651383657 475533312 8474 33554432000 4194304 5683644 140733828325024 18446744073709551615 254261636678 0 0 16781312 514 12442409485010468863 0 0 17 7 0 0 0
/proc/10690/statm: 116097 8474 344 364 0 51964 0
[pid=10692] ppid=10690 vsize=253036 CPUtime=0.87 cores=1,3,5,7
/proc/10692/stat : 10692 (FH) R 10690 10692 10692 0 -1 4202496 2506 0 0 0 87 0 0 0 23 0 1 0 651383657 259108864 2233 33554432000 134512640 134936988 4292069584 18446744073709551615 134552472 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/10692/statm: 63259 2233 41 104 0 62763 0
[pid=10289/tid=10308] ppid=10287 vsize=474632 CPUtime=0 cores=1,3,5,7
/proc/10289/task/10308/stat : 10308 (python) S 10287 10289 10255 0 -1 4202560 29 930 0 0 0 0 0 0 20 0 19 0 651030077 486023168 10046 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 2 18446604463253787904 0 0 -1 7 0 0 0
[pid=10289/tid=10318] ppid=10287 vsize=474632 CPUtime=0 cores=1,3,5,7
/proc/10289/task/10318/stat : 10318 (python) S 10287 10289 10255 0 -1 4202560 28 930 0 0 0 0 0 0 20 0 19 0 651040013 486023168 10046 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 2 18446604463253787904 0 0 -1 7 0 0 0
[pid=10289/tid=10329] ppid=10287 vsize=474632 CPUtime=0 cores=1,3,5,7
/proc/10289/task/10329/stat : 10329 (python) S 10287 10289 10255 0 -1 4202560 25 930 0 0 0 0 0 0 20 0 19 0 651049950 486023168 10046 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 2 18446604463253787904 0 0 -1 7 0 0 0
[pid=10289/tid=10340] ppid=10287 vsize=474632 CPUtime=0 cores=1,3,5,7
/proc/10289/task/10340/stat : 10340 (python) S 10287 10289 10255 0 -1 4202560 28 930 0 0 0 0 0 0 20 0 19 0 651059984 486023168 10046 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 2 18446604463253787904 0 0 -1 1 0 0 0
[pid=10289/tid=10350] ppid=10287 vsize=474632 CPUtime=0 cores=1,3,5,7
/proc/10289/task/10350/stat : 10350 (python) S 10287 10289 10255 0 -1 4202560 30 930 0 0 0 0 0 0 20 0 19 0 651069919 486023168 10046 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 2 18446604463253787904 0 0 -1 7 0 0 0
[pid=10289/tid=10361] ppid=10287 vsize=474632 CPUtime=0 cores=1,3,5,7
/proc/10289/task/10361/stat : 10361 (python) S 10287 10289 10255 0 -1 4202560 29 930 0 0 0 0 0 0 20 0 19 0 651079854 486023168 10046 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 2 18446604463253787904 0 0 -1 3 0 0 0
[pid=10289/tid=10372] ppid=10287 vsize=474632 CPUtime=0 cores=1,3,5,7
/proc/10289/task/10372/stat : 10372 (python) S 10287 10289 10255 0 -1 4202560 30 930 0 0 0 0 0 0 20 0 19 0 651089893 486023168 10046 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 2 18446604463253787904 0 0 -1 7 0 0 0
[pid=10289/tid=10382] ppid=10287 vsize=474632 CPUtime=0 cores=1,3,5,7
/proc/10289/task/10382/stat : 10382 (python) S 10287 10289 10255 0 -1 4202560 30 930 0 0 0 0 0 0 20 0 19 0 651099829 486023168 10046 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 2 18446604463253787904 0 0 -1 3 0 0 0
[pid=10289/tid=10398] ppid=10287 vsize=474632 CPUtime=0 cores=1,3,5,7
/proc/10289/task/10398/stat : 10398 (python) S 10287 10289 10255 0 -1 4202560 30 930 0 0 0 0 0 0 20 0 19 0 651109664 486023168 10046 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 2 18446604463253787904 0 0 -1 3 0 0 0
[pid=10289/tid=10408] ppid=10287 vsize=474632 CPUtime=0 cores=1,3,5,7
/proc/10289/task/10408/stat : 10408 (python) S 10287 10289 10255 0 -1 4202560 30 930 0 0 0 0 0 0 20 0 19 0 651119601 486023168 10046 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 2 18446604463253787904 0 0 -1 3 0 0 0
[pid=10289/tid=10419] ppid=10287 vsize=474632 CPUtime=0 cores=1,3,5,7
/proc/10289/task/10419/stat : 10419 (python) S 10287 10289 10255 0 -1 4202560 29 930 0 0 0 0 0 0 20 0 19 0 651129637 486023168 10046 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 2 18446604463253787904 0 0 -1 5 0 0 0
[pid=10289/tid=10432] ppid=10287 vsize=474632 CPUtime=0 cores=1,3,5,7
/proc/10289/task/10432/stat : 10432 (python) S 10287 10289 10255 0 -1 4202560 30 930 0 0 0 0 0 0 20 0 19 0 651149113 486023168 10046 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 2 18446604463253787904 0 0 -1 3 0 0 0
[pid=10289/tid=10443] ppid=10287 vsize=474632 CPUtime=0 cores=1,3,5,7
/proc/10289/task/10443/stat : 10443 (python) S 10287 10289 10255 0 -1 4202560 28 930 0 0 0 0 0 0 20 0 19 0 651159055 486023168 10046 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 2 18446604463253787904 0 0 -1 3 0 0 0
[pid=10289/tid=10456] ppid=10287 vsize=474632 CPUtime=0 cores=1,3,5,7
/proc/10289/task/10456/stat : 10456 (python) S 10287 10289 10255 0 -1 4202560 26 930 0 0 0 0 0 0 20 0 19 0 651178533 486023168 10046 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 2 18446604463253787904 0 0 -1 3 0 0 0
[pid=10289/tid=10467] ppid=10287 vsize=474632 CPUtime=0 cores=1,3,5,7
/proc/10289/task/10467/stat : 10467 (python) S 10287 10289 10255 0 -1 4202560 27 930 0 0 0 0 0 0 20 0 19 0 651188571 486023168 10046 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 2 18446604463253787904 0 0 -1 3 0 0 0
[pid=10289/tid=10480] ppid=10287 vsize=474632 CPUtime=0 cores=1,3,5,7
/proc/10289/task/10480/stat : 10480 (python) S 10287 10289 10255 0 -1 4202560 24 930 0 0 0 0 0 0 20 0 19 0 651208150 486023168 10046 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 2 18446604463253787904 0 0 -1 3 0 0 0
[pid=10289/tid=10489] ppid=10287 vsize=474632 CPUtime=0 cores=1,3,5,7
/proc/10289/task/10489/stat : 10489 (python) S 10287 10289 10255 0 -1 4202560 25 930 0 0 0 0 0 0 20 0 19 0 651228132 486023168 10046 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 2 18446604463253787904 0 0 -1 3 0 0 0
[pid=10289/tid=10691] ppid=10287 vsize=474632 CPUtime=0 cores=1,3,5,7
/proc/10289/task/10691/stat : 10691 (python) S 10287 10289 10255 0 -1 4202560 29 930 0 0 0 0 0 0 20 0 19 0 651383657 486023168 10046 33554432000 4194304 5683644 140733828325024 18446744073709551615 254273440177 0 0 16781312 2 18446604463253787904 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3550.33
Current children cumulated vsize (KiB) 16190620

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 10289 and gives
#  childrusage.ru_utime.tv_sec=2
#  childrusage.ru_utime.tv_usec=802573
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=209968
# CPU time returned by wait4() is 3.01254
# while last known CPU time is 3550.33
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 3539.83
CPU time (s): 3550.33
CPU user time (s): 3544.52
CPU system time (s): 5.81
CPU usage (%): 100.296
Max. virtual memory (cumulated for all children) (KiB): 16190620

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2.80257
system time used= 0.209968
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 31762
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 9594
involuntary context switches= 43

runsolver used 29.6355 second user time and 65.919 second system time

The end

Launcher Data 

Begin job on node140 at 2011-05-17 00:49:04
IDJOB=3386457
IDBENCH=82522
IDSOLVER=1872
FILE ID=node140/3386457-1305586144
RUNJOBID= node140-1305586143-10274
PBS_JOBID= 13350338
Free space on /tmp= 73424 MiB

SOLVER NAME= borg-sat 11.03.02-seq-random (fixed)
BENCH NAME= SAT11/random/large/unif-k5-r20-v2000-c40000-S1143439968-082-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3386457-1305586144/watcher-3386457-1305586144 -o /tmp/evaluation-result-3386457-1305586144/solver-3386457-1305586144 -C 5000 -W 5100 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3386457-1305586144.cnf -seed 700887766 -budget 5000

TIME LIMIT= 5000 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 4baebda329e22fa39a89c4e2a8acff70
RANDOM SEED=700887766

node140.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.14
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      21505288 kB
Buffers:        335452 kB
Cached:        5022628 kB
SwapCached:       8088 kB
Active:        7669888 kB
Inactive:      3438108 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      21505288 kB
SwapTotal:    67111528 kB
SwapFree:     67057896 kB
Dirty:           65608 kB
Writeback:           0 kB
AnonPages:     5746760 kB
Mapped:          30244 kB
Slab:           228820 kB
PageTables:      41104 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 102020508 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73356 MiB
End job on node140 at 2011-05-17 01:48:06