Trace number 3386955

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-seq-random (fixed)? (MO) 4546.61 4528.05

General information on the benchmark

Namerandom/medium/
unif-k7-r89-v90-c8010-S423984545-074.UNKNOWN.cnf
MD5SUM2ef214e0162b22d1093a804ea2af97a1
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables90
Number of clauses8010
Sum of the clauses size56070
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 58010

Solver Data 

0.32/0.48	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.32/0.48	c INFO: solving HOME/instance-3386955-1305595121.cnf
0.32/0.48	c INFO: computing task features
0.40/0.53	c INFO: computed variable-clause graph statistics
0.40/0.54	c INFO: computed clause balance statistics
0.70/0.80	c INFO: computed variable balance statistics
0.80/0.91	c INFO: computed small-clause counts
0.80/0.91	c INFO: computed Horn-clause counts
0.80/0.92	c INFO: computed variable graph statistics
0.80/0.92	c INFO: collected features for HOME/instance-3386955-1305595121.cnf in 0.44 s
0.80/0.95	c INFO: running gnovelty+2@0 for 97 with 4999 remaining (b = 0.47)
101.47/101.39	c INFO: running gnovelty+2@0 for 97 with 4899 remaining (b = 0.11)
204.40/203.89	c INFO: running adaptg2wsat2009++@0 for 97 with 4796 remaining (b = 0.06)
302.31/301.37	c INFO: running adaptg2wsat2009++@0 for 97 with 4699 remaining (b = 0.05)
403.12/401.82	c INFO: running adaptg2wsat2009++@0 for 97 with 4598 remaining (b = 0.04)
503.90/502.29	c INFO: running gnovelty+2@0 for 97 with 4498 remaining (b = 0.03)
604.74/602.75	c INFO: running FH@0 for 97 with 4398 remaining (b = 0.02)
705.51/703.20	c INFO: running adaptg2wsat2009++@0 for 97 with 4297 remaining (b = 0.02)
806.31/803.63	c INFO: running adaptg2wsat2009++@0 for 97 with 4197 remaining (b = 0.02)
909.22/906.13	c INFO: running hybridGM3@0 for 194 with 4095 remaining (b = 0.03)
1107.82/1104.02	c INFO: running FH@0 for 97 with 3897 remaining (b = 0.01)
1210.77/1206.53	c INFO: running gnovelty+2@0 for 97 with 3794 remaining (b = 0.01)
1308.67/1304.00	c INFO: running hybridGM3@0 for 194 with 3697 remaining (b = 0.02)
1504.40/1498.93	c INFO: running adaptg2wsat2009++@97 for 97 with 3502 remaining (b = 0.01)
1607.23/1601.39	c INFO: running hybridGM3@0 for 194 with 3399 remaining (b = 0.02)
1805.11/1798.33	c INFO: running march_hi@0 for 388 with 3203 remaining (b = 0.04)
2196.36/2187.82	c INFO: running march_hi@389 for 97 with 2813 remaining (b = 0.02)
2295.59/2286.62	c INFO: running march_hi@488 for 97 with 2714 remaining (b = 0.01)
2391.31/2381.91	c INFO: running march_hi@583 for 97 with 2619 remaining (b = 0.02)
2488.72/2478.97	c INFO: running march_hi@680 for 97 with 2522 remaining (b = 0.01)
2589.19/2578.94	c INFO: running hybridGM3@0 for 194 with 2422 remaining (b = 0.02)
2786.69/2775.67	c INFO: running TNM@0 for 1455 with 2225 remaining (b = 0.13)
4251.41/4234.02	c INFO: running TNM@1458 for 97 with 767 remaining (b = 0.01)
4347.20/4329.44	c INFO: running adaptg2wsat2009++@100 for 97 with 671 remaining (b = 0.01)
4445.05/4426.87	c INFO: running FH@0 for 97 with 574 remaining (b = 0.01)
4545.97/4527.34	c INFO: running march_hi@0 for 474 with 474 remaining (b = 0.04)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3386955-1305595121/watcher-3386955-1305595121 -o /tmp/evaluation-result-3386955-1305595121/solver-3386955-1305595121 -C 5000 -W 5100 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3386955-1305595121.cnf -seed 2080514635 -budget 5000 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 5030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 5.09 5.07 5.08 6/581 10087
/proc/meminfo: memFree=20576176/32950928 swapFree=67056568/67111528
[pid=10087] ppid=10085 vsize=15752 CPUtime=0 cores=1,3,5,7
/proc/10087/stat : 10087 (python) S 10085 10087 9113 0 -1 4202496 837 0 0 0 0 0 0 0 25 0 1 0 671109228 16130048 722 33554432000 4194304 5683644 140735766553424 18446744073709551615 239476692992 0 2147483391 16781312 2 18446744071699207358 0 0 17 3 0 0 0
/proc/10087/statm: 3938 722 334 364 0 425 0

[startup+0.07699 s]
/proc/loadavg: 5.09 5.07 5.08 6/581 10087
/proc/meminfo: memFree=20576176/32950928 swapFree=67056568/67111528
[pid=10087] ppid=10085 vsize=56640 CPUtime=0.05 cores=1,3,5,7
/proc/10087/stat : 10087 (python) R 10085 10087 9113 0 -1 4202496 2087 0 0 0 4 1 0 0 17 0 1 0 671109228 57999360 1843 33554432000 4194304 5683644 140735766553424 18446744073709551615 4509104 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/10087/statm: 14160 1850 499 364 0 1283 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 56640

[startup+0.100985 s]
/proc/loadavg: 5.09 5.07 5.08 6/581 10087
/proc/meminfo: memFree=20576176/32950928 swapFree=67056568/67111528
[pid=10087] ppid=10085 vsize=101744 CPUtime=0.07 cores=1,3,5,7
/proc/10087/stat : 10087 (python) R 10085 10087 9113 0 -1 4202496 2930 0 0 0 5 2 0 0 17 0 1 0 671109228 104185856 2627 33554432000 4194304 5683644 140735766553424 18446744073709551615 239476694784 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/10087/statm: 25470 2645 837 364 0 1770 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 101744

[startup+0.302326 s]
/proc/loadavg: 5.09 5.07 5.08 6/581 10087
/proc/meminfo: memFree=20576176/32950928 swapFree=67056568/67111528
[pid=10087] ppid=10085 vsize=157376 CPUtime=0.22 cores=1,3,5,7
/proc/10087/stat : 10087 (python) S 10085 10087 9113 0 -1 4202496 6600 933 1 0 16 6 0 0 17 0 1 0 671109228 161153024 4694 33554432000 4194304 5683644 140735766553424 18446744073709551615 239476692992 0 2147483391 16781312 2 18446744071699207358 0 0 17 7 0 0 0
/proc/10087/statm: 39344 4694 1257 364 0 3452 0
Current children cumulated CPU time (s) 0.22
Current children cumulated vsize (KiB) 157376

[startup+0.700869 s]
/proc/loadavg: 5.09 5.07 5.08 6/581 10087
/proc/meminfo: memFree=20576176/32950928 swapFree=67056568/67111528
[pid=10087] ppid=10085 vsize=280224 CPUtime=0.6 cores=1,3,5,7
/proc/10087/stat : 10087 (python) R 10085 10087 9113 0 -1 4202496 10004 933 1 0 51 9 0 0 17 0 1 0 671109228 286949376 7441 33554432000 4194304 5683644 140735766553424 18446744073709551615 239476349945 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/10087/statm: 70056 7441 1826 364 0 5980 0
Current children cumulated CPU time (s) 0.6
Current children cumulated vsize (KiB) 280224

[startup+1.50072 s]
/proc/loadavg: 5.09 5.07 5.08 6/585 10095
/proc/meminfo: memFree=20549608/32950928 swapFree=67056568/67111528
[pid=10087] ppid=10085 vsize=289648 CPUtime=0.85 cores=1,3,5,7
/proc/10087/stat : 10087 (python) S 10085 10087 9113 0 -1 4202496 10440 933 1 0 76 9 0 0 20 0 2 0 671109228 296599552 7317 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488521675 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/10087/statm: 72412 7317 1877 364 0 8330 0
[pid=10093] ppid=10087 vsize=279404 CPUtime=0 cores=1,3,5,7
/proc/10093/stat : 10093 (python) S 10087 10087 9113 0 -1 4202560 734 0 0 0 0 0 0 0 20 0 1 0 671109323 286109696 5790 33554432000 4194304 5683644 140735766553424 18446744073709551615 239476715023 0 0 16781312 514 8770609920383909887 0 0 17 3 0 0 0
/proc/10093/statm: 69851 5790 354 364 0 5769 0
[pid=10095] ppid=10093 vsize=1018092 CPUtime=0.53 cores=1,3,5,7
/proc/10095/stat : 10095 (gnovelty+2) R 10093 10095 10095 0 -1 4202496 573 0 0 0 53 0 0 0 23 0 1 0 671109323 1042526208 307 33554432000 134512640 135663704 4293514576 18446744073709551615 134525678 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/10095/statm: 254523 307 113 282 0 254239 0
[pid=10087/tid=10094] ppid=10085 vsize=289648 CPUtime=0 cores=1,3,5,7
/proc/10087/task/10094/stat : 10094 (python) S 10085 10087 9113 0 -1 4202560 27 933 0 0 0 0 0 0 20 0 2 0 671109323 296599552 7317 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 2 18446604468912035968 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 1.38
Current children cumulated vsize (KiB) 1587144

[startup+3.10041 s]
/proc/loadavg: 5.09 5.07 5.08 6/585 10095
/proc/meminfo: memFree=20548616/32950928 swapFree=67056568/67111528
[pid=10087] ppid=10085 vsize=289648 CPUtime=0.85 cores=1,3,5,7
/proc/10087/stat : 10087 (python) S 10085 10087 9113 0 -1 4202496 10440 933 1 0 76 9 0 0 20 0 2 0 671109228 296599552 7317 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488521675 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/10087/statm: 72412 7317 1877 364 0 8330 0
[pid=10093] ppid=10087 vsize=279404 CPUtime=0 cores=1,3,5,7
/proc/10093/stat : 10093 (python) S 10087 10087 9113 0 -1 4202560 734 0 0 0 0 0 0 0 18 0 1 0 671109323 286109696 5790 33554432000 4194304 5683644 140735766553424 18446744073709551615 239476715023 0 0 16781312 514 8770609920383877381 0 0 17 3 0 0 0
/proc/10093/statm: 69851 5790 354 364 0 5769 0
[pid=10095] ppid=10093 vsize=1018092 CPUtime=2.13 cores=1,3,5,7
/proc/10095/stat : 10095 (gnovelty+2) R 10093 10095 10095 0 -1 4202496 573 0 0 0 213 0 0 0 25 0 1 0 671109323 1042526208 307 33554432000 134512640 135663704 4293514576 18446744073709551615 134524548 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/10095/statm: 254523 307 113 282 0 254239 0
[pid=10087/tid=10094] ppid=10085 vsize=289648 CPUtime=0 cores=1,3,5,7
/proc/10087/task/10094/stat : 10094 (python) S 10085 10087 9113 0 -1 4202560 27 933 0 0 0 0 0 0 20 0 2 0 671109323 296599552 7317 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 2 18446604468912035968 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 2.98
Current children cumulated vsize (KiB) 1587144

[startup+6.30079 s]
/proc/loadavg: 5.16 5.08 5.08 6/585 10095
/proc/meminfo: memFree=20547396/32950928 swapFree=67056568/67111528
[pid=10087] ppid=10085 vsize=289648 CPUtime=0.85 cores=1,3,5,7
/proc/10087/stat : 10087 (python) S 10085 10087 9113 0 -1 4202496 10440 933 1 0 76 9 0 0 20 0 2 0 671109228 296599552 7317 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488521675 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/10087/statm: 72412 7317 1877 364 0 8330 0
[pid=10093] ppid=10087 vsize=279404 CPUtime=0.02 cores=1,3,5,7
/proc/10093/stat : 10093 (python) S 10087 10087 9113 0 -1 4202560 884 0 0 0 1 1 0 0 18 0 1 0 671109323 286109696 5813 33554432000 4194304 5683644 140735766553424 18446744073709551615 239476715023 0 0 16781312 514 8770609920383877381 0 0 17 3 0 0 0
/proc/10093/statm: 69851 5813 377 364 0 5769 0
[pid=10095] ppid=10093 vsize=1018092 CPUtime=5.33 cores=1,3,5,7
/proc/10095/stat : 10095 (gnovelty+2) R 10093 10095 10095 0 -1 4202496 573 0 0 0 533 0 0 0 25 0 1 0 671109323 1042526208 307 33554432000 134512640 135663704 4293514576 18446744073709551615 134525684 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/10095/statm: 254523 307 113 282 0 254239 0
[pid=10087/tid=10094] ppid=10085 vsize=289648 CPUtime=0 cores=1,3,5,7
/proc/10087/task/10094/stat : 10094 (python) S 10085 10087 9113 0 -1 4202560 27 933 0 0 0 0 0 0 20 0 2 0 671109323 296599552 7317 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 2 18446604468912035968 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 6.2
Current children cumulated vsize (KiB) 1587144

[startup+12.7006 s]
/proc/loadavg: 5.15 5.08 5.08 6/585 10095
/proc/meminfo: memFree=20547808/32950928 swapFree=67056568/67111528
[pid=10087] ppid=10085 vsize=289648 CPUtime=0.85 cores=1,3,5,7
/proc/10087/stat : 10087 (python) S 10085 10087 9113 0 -1 4202496 10440 933 1 0 76 9 0 0 20 0 2 0 671109228 296599552 7317 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488521675 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/10087/statm: 72412 7317 1877 364 0 8330 0
[pid=10093] ppid=10087 vsize=279404 CPUtime=0.04 cores=1,3,5,7
/proc/10093/stat : 10093 (python) S 10087 10087 9113 0 -1 4202560 887 0 0 0 2 2 0 0 18 0 1 0 671109323 286109696 5813 33554432000 4194304 5683644 140735766553424 18446744073709551615 239476715023 0 0 16781312 514 2976122889691955461 0 0 17 3 0 0 0
/proc/10093/statm: 69851 5813 377 364 0 5769 0
[pid=10095] ppid=10093 vsize=1018092 CPUtime=11.72 cores=1,3,5,7
/proc/10095/stat : 10095 (gnovelty+2) R 10093 10095 10095 0 -1 4202496 573 0 0 0 1172 0 0 0 25 0 1 0 671109323 1042526208 307 33554432000 134512640 135663704 4293514576 18446744073709551615 134545631 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/10095/statm: 254523 307 113 282 0 254239 0
[pid=10087/tid=10094] ppid=10085 vsize=289648 CPUtime=0 cores=1,3,5,7
/proc/10087/task/10094/stat : 10094 (python) S 10085 10087 9113 0 -1 4202560 27 933 0 0 0 0 0 0 20 0 2 0 671109323 296599552 7317 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 2 18446604468912035968 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 12.61
Current children cumulated vsize (KiB) 1587144

[startup+25.5011 s]
/proc/loadavg: 5.25 5.11 5.09 6/585 10095
/proc/meminfo: memFree=20548320/32950928 swapFree=67056568/67111528
[pid=10087] ppid=10085 vsize=289648 CPUtime=0.85 cores=1,3,5,7
/proc/10087/stat : 10087 (python) S 10085 10087 9113 0 -1 4202496 10440 933 1 0 76 9 0 0 20 0 2 0 671109228 296599552 7317 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488521675 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/10087/statm: 72412 7317 1877 364 0 8330 0
[pid=10093] ppid=10087 vsize=279404 CPUtime=0.09 cores=1,3,5,7
/proc/10093/stat : 10093 (python) S 10087 10087 9113 0 -1 4202560 893 0 0 0 4 5 0 0 18 0 1 0 671109323 286109696 5813 33554432000 4194304 5683644 140735766553424 18446744073709551615 239476715023 0 0 16781312 514 2976122889691955461 0 0 17 3 0 0 0
/proc/10093/statm: 69851 5813 377 364 0 5769 0
[pid=10095] ppid=10093 vsize=1018092 CPUtime=24.51 cores=1,3,5,7
/proc/10095/stat : 10095 (gnovelty+2) R 10093 10095 10095 0 -1 4202496 573 0 0 0 2451 0 0 0 25 0 1 0 671109323 1042526208 307 33554432000 134512640 135663704 4293514576 18446744073709551615 134525002 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/10095/statm: 254523 307 113 282 0 254239 0
[pid=10087/tid=10094] ppid=10085 vsize=289648 CPUtime=0 cores=1,3,5,7
/proc/10087/task/10094/stat : 10094 (python) S 10085 10087 9113 0 -1 4202560 27 933 0 0 0 0 0 0 20 0 2 0 671109323 296599552 7317 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 2 18446604468912035968 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 25.45
Current children cumulated vsize (KiB) 1587144

[startup+51.1012 s]
/proc/loadavg: 5.16 5.10 5.09 6/585 10096
/proc/meminfo: memFree=20548580/32950928 swapFree=67056568/67111528
[pid=10087] ppid=10085 vsize=289648 CPUtime=0.85 cores=1,3,5,7
/proc/10087/stat : 10087 (python) S 10085 10087 9113 0 -1 4202496 10440 933 1 0 76 9 0 0 20 0 2 0 671109228 296599552 7317 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488521675 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/10087/statm: 72412 7317 1877 364 0 8330 0
[pid=10093] ppid=10087 vsize=279404 CPUtime=0.19 cores=1,3,5,7
/proc/10093/stat : 10093 (python) S 10087 10087 9113 0 -1 4202560 904 0 0 0 10 9 0 0 18 0 1 0 671109323 286109696 5813 33554432000 4194304 5683644 140735766553424 18446744073709551615 239476715023 0 0 16781312 514 2976122889691955461 0 0 17 3 0 0 0
/proc/10093/statm: 69851 5813 377 364 0 5769 0
[pid=10095] ppid=10093 vsize=1018092 CPUtime=50.08 cores=1,3,5,7
/proc/10095/stat : 10095 (gnovelty+2) R 10093 10095 10095 0 -1 4202496 573 0 0 0 5008 0 0 0 25 0 1 0 671109323 1042526208 307 33554432000 134512640 135663704 4293514576 18446744073709551615 134525237 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/10095/statm: 254523 307 113 282 0 254239 0
[pid=10087/tid=10094] ppid=10085 vsize=289648 CPUtime=0 cores=1,3,5,7
/proc/10087/task/10094/stat : 10094 (python) S 10085 10087 9113 0 -1 4202560 27 933 0 0 0 0 0 0 20 0 2 0 671109323 296599552 7317 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 2 18446604468912035968 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 51.12
Current children cumulated vsize (KiB) 1587144

[startup+102.32 s]
/proc/loadavg: 5.12 5.09 5.09 6/589 10101
/proc/meminfo: memFree=20543356/32950928 swapFree=67056568/67111528
[pid=10087] ppid=10085 vsize=300224 CPUtime=0.89 cores=1,3,5,7
/proc/10087/stat : 10087 (python) S 10085 10087 9113 0 -1 4202496 11350 933 1 0 79 10 0 0 18 0 3 0 671109228 307429376 7372 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488521675 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/10087/statm: 75056 7372 1880 364 0 10971 0
[pid=10093] ppid=10087 vsize=289648 CPUtime=0.4 cores=1,3,5,7
/proc/10093/stat : 10093 (python) S 10087 10087 9113 0 -1 4202560 1079 0 0 0 21 19 0 0 19 0 2 0 671109323 296599552 5851 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488521675 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/10093/statm: 72412 5851 410 364 0 8330 0
[pid=10095] ppid=10093 vsize=1018092 CPUtime=100.3 cores=1,3,5,7
/proc/10095/stat : 10095 (gnovelty+2) T 10093 10095 10095 0 -1 4202496 573 0 0 0 10030 0 0 0 25 0 1 0 671109323 1042526208 307 33554432000 134512640 135663704 4293514576 18446744073709551615 134525278 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/10095/statm: 254523 307 113 282 0 254239 0
[pid=10093/tid=10098] ppid=10087 vsize=289648 CPUtime=0 cores=1,3,5,7
/proc/10093/task/10098/stat : 10098 (python) S 10087 10087 9113 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 671119364 296599552 5851 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 514 18446604454954760000 0 0 -1 3 0 0 0
[pid=10099] ppid=10087 vsize=289980 CPUtime=0 cores=1,3,5,7
/proc/10099/stat : 10099 (python) S 10087 10087 9113 0 -1 4202560 745 0 0 0 0 0 0 0 19 0 1 0 671119367 296939520 5836 33554432000 4194304 5683644 140735766553424 18446744073709551615 239476715078 0 0 16781312 514 2776157463022403583 0 0 17 7 0 0 0
/proc/10099/statm: 72495 5836 348 364 0 8410 0
[pid=10101] ppid=10099 vsize=1018092 CPUtime=0.92 cores=1,3,5,7
/proc/10101/stat : 10101 (gnovelty+2) R 10099 10101 10101 0 -1 4202496 570 0 0 0 92 0 0 0 23 0 1 0 671119367 1042526208 307 33554432000 134512640 135663704 4292203264 18446744073709551615 134525224 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/10101/statm: 254523 307 113 282 0 254239 0
[pid=10087/tid=10094] ppid=10085 vsize=300224 CPUtime=0 cores=1,3,5,7
/proc/10087/task/10094/stat : 10094 (python) S 10085 10087 9113 0 -1 4202560 27 933 0 0 0 0 0 0 20 0 3 0 671109323 307429376 7372 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 2 18446604468912035968 0 0 -1 7 0 0 0
[pid=10087/tid=10100] ppid=10085 vsize=300224 CPUtime=0 cores=1,3,5,7
/proc/10087/task/10100/stat : 10100 (python) S 10085 10087 9113 0 -1 4202560 28 933 0 0 0 0 0 0 20 0 3 0 671119367 307429376 7372 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 2 18446604468912035968 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 102.51
Current children cumulated vsize (KiB) 2916036

[startup+162.301 s]
/proc/loadavg: 5.18 5.12 5.09 6/589 10103
/proc/meminfo: memFree=20541472/32950928 swapFree=67056568/67111528
[pid=10087] ppid=10085 vsize=300224 CPUtime=0.89 cores=1,3,5,7
/proc/10087/stat : 10087 (python) S 10085 10087 9113 0 -1 4202496 11350 933 1 0 79 10 0 0 18 0 3 0 671109228 307429376 7372 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488521675 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/10087/statm: 75056 7372 1880 364 0 10971 0
[pid=10093] ppid=10087 vsize=289648 CPUtime=0.4 cores=1,3,5,7

################
# More data... #
################

/proc/10119/statm: 80187 5901 400 364 0 16093 0
[pid=10121] ppid=10119 vsize=417192 CPUtime=100.43 cores=1,3,5,7
/proc/10121/stat : 10121 (adaptg2wsat2009) T 10119 10121 10121 0 -1 4202496 763 0 0 0 10043 0 0 0 25 0 1 0 671149410 427204608 495 33554432000 134512640 135107411 4288711872 18446744073709551615 134531664 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/10121/statm: 104298 495 59 146 0 103174 0
[pid=10119/tid=10126] ppid=10087 vsize=320748 CPUtime=0 cores=1,3,5,7
/proc/10119/task/10126/stat : 10126 (python) S 10087 10087 9113 0 -1 4202560 51 0 0 0 0 0 0 0 19 0 2 0 671159454 328445952 5901 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 514 18446604456156705088 0 0 -1 3 0 0 0
[pid=10127] ppid=10087 vsize=331004 CPUtime=0.41 cores=1,3,5,7
/proc/10127/stat : 10127 (python) S 10087 10087 9113 0 -1 4202560 1126 0 0 0 20 21 0 0 19 0 2 0 671159457 338948096 5905 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488521675 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/10127/statm: 82751 5905 400 364 0 18654 0
[pid=10129] ppid=10127 vsize=1018092 CPUtime=100.3 cores=1,3,5,7
/proc/10129/stat : 10129 (gnovelty+2) T 10127 10129 10129 0 -1 4202496 572 0 0 0 10030 0 0 0 25 0 1 0 671159458 1042526208 306 33554432000 134512640 135663704 4290003968 18446744073709551615 134525255 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/10129/statm: 254523 306 113 282 0 254239 0
[pid=10127/tid=10136] ppid=10087 vsize=331004 CPUtime=0 cores=1,3,5,7
/proc/10127/task/10136/stat : 10136 (python) S 10087 10087 9113 0 -1 4202560 51 0 0 0 0 0 0 0 19 0 2 0 671169500 338948096 5905 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 514 18446604464052269440 0 0 -1 7 0 0 0
[pid=10137] ppid=10087 vsize=406796 CPUtime=0.42 cores=1,3,5,7
/proc/10137/stat : 10137 (python) S 10087 10087 9113 0 -1 4202560 1181 0 0 0 23 19 0 0 18 0 2 0 671169503 416559104 5916 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488521675 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/10137/statm: 101699 5916 406 364 0 37599 0
[pid=10139] ppid=10137 vsize=247348 CPUtime=100.28 cores=1,3,5,7
/proc/10139/stat : 10139 (FH) T 10137 10139 10139 0 -1 4202496 711 0 0 0 10028 0 0 0 25 0 1 0 671169503 253284352 444 33554432000 134512640 134936988 4289831824 18446744073709551615 134553238 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/10139/statm: 61837 444 41 104 0 61341 0
[pid=10137/tid=10182] ppid=10087 vsize=406796 CPUtime=0 cores=1,3,5,7
/proc/10137/task/10182/stat : 10182 (python) S 10087 10087 9113 0 -1 4202560 98 0 0 0 0 0 0 0 19 0 2 0 671179544 416559104 5916 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 514 18446604456156706880 0 0 -1 1 0 0 0
[pid=10183] ppid=10087 vsize=351516 CPUtime=0.41 cores=1,3,5,7
/proc/10183/stat : 10183 (python) S 10087 10087 9113 0 -1 4202560 1172 0 0 0 23 18 0 0 18 0 2 0 671179548 359952384 5919 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488521675 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/10183/statm: 87879 5919 406 364 0 23776 0
[pid=10185] ppid=10183 vsize=417192 CPUtime=100.39 cores=1,3,5,7
/proc/10185/stat : 10185 (adaptg2wsat2009) T 10183 10185 10185 0 -1 4202496 765 0 0 0 10039 0 0 0 25 0 1 0 671179548 427204608 495 33554432000 134512640 135107411 4288710192 18446744073709551615 134532306 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/10185/statm: 104298 495 59 146 0 103174 0
[pid=10183/tid=10193] ppid=10087 vsize=351516 CPUtime=0 cores=1,3,5,7
/proc/10183/task/10193/stat : 10193 (python) S 10087 10087 9113 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 671189588 359952384 5919 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 514 18446604470545409024 0 0 -1 7 0 0 0
[pid=10194] ppid=10087 vsize=361772 CPUtime=0.44 cores=1,3,5,7
/proc/10194/stat : 10194 (python) S 10087 10087 9113 0 -1 4202560 1148 0 0 0 23 21 0 0 19 0 2 0 671189591 370454528 5947 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488521675 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/10194/statm: 90443 5947 400 364 0 26337 0
[pid=10196] ppid=10194 vsize=417192 CPUtime=102.43 cores=1,3,5,7
/proc/10196/stat : 10196 (adaptg2wsat2009) T 10194 10196 10196 0 -1 4202496 769 0 0 0 10243 0 0 0 25 0 1 0 671189592 427204608 496 33554432000 134512640 135107411 4289393568 18446744073709551615 134531928 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/10196/statm: 104298 496 59 146 0 103174 0
[pid=10194/tid=10200] ppid=10087 vsize=361772 CPUtime=0 cores=1,3,5,7
/proc/10194/task/10200/stat : 10200 (python) S 10087 10087 9113 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 671199837 370454528 5947 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 514 18446604461265830336 0 0 -1 3 0 0 0
[pid=10201] ppid=10087 vsize=372028 CPUtime=0.81 cores=1,3,5,7
/proc/10201/stat : 10201 (python) S 10087 10087 9113 0 -1 4202560 1162 0 0 0 41 40 0 0 16 0 2 0 671199841 380956672 5951 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488521675 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/10201/statm: 93007 5951 400 364 0 28898 0
[pid=10203] ppid=10201 vsize=268424 CPUtime=197.75 cores=1,3,5,7
/proc/10203/stat : 10203 (hybridGM3) T 10201 10203 10203 0 -1 4202496 4943 0 0 0 19775 0 0 0 25 0 1 0 671199841 274866176 4673 33554432000 134512640 135204385 4287071696 18446744073709551615 134537457 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/10203/statm: 67106 4673 64 169 0 66936 0
[pid=10201/tid=10209] ppid=10087 vsize=372028 CPUtime=0 cores=1,3,5,7
/proc/10201/task/10209/stat : 10209 (python) S 10087 10087 9113 0 -1 4202560 45 0 0 0 0 0 0 0 17 0 2 0 671219626 380956672 5951 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 514 18446604461265832128 0 0 -1 7 0 0 0
[pid=10210] ppid=10087 vsize=382552 CPUtime=0.44 cores=1,3,5,7
/proc/10210/stat : 10210 (python) S 10087 10087 9113 0 -1 4202560 1154 0 0 0 22 22 0 0 19 0 2 0 671219630 391733248 5992 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488521675 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/10210/statm: 95638 5992 400 364 0 31526 0
[pid=10212] ppid=10210 vsize=247348 CPUtime=102.46 cores=1,3,5,7
/proc/10212/stat : 10212 (FH) T 10210 10212 10212 0 -1 4202496 715 0 0 0 10246 0 0 0 25 0 1 0 671219631 253284352 444 33554432000 134512640 134936988 4287672704 18446744073709551615 134552486 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/10212/statm: 61837 444 41 104 0 61341 0
[pid=10210/tid=10220] ppid=10087 vsize=382552 CPUtime=0 cores=1,3,5,7
/proc/10210/task/10220/stat : 10220 (python) S 10087 10087 9113 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 671229877 391733248 5992 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 514 18446604455695667456 0 0 -1 7 0 0 0
[pid=10221] ppid=10087 vsize=392808 CPUtime=0.42 cores=1,3,5,7
/proc/10221/stat : 10221 (python) S 10087 10087 9113 0 -1 4202560 1156 0 0 0 22 20 0 0 19 0 2 0 671229881 402235392 5996 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488521675 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/10221/statm: 98202 5996 400 364 0 34087 0
[pid=10223] ppid=10221 vsize=1018092 CPUtime=97.43 cores=1,3,5,7
/proc/10223/stat : 10223 (gnovelty+2) T 10221 10223 10223 0 -1 4202496 580 0 0 0 9743 0 0 0 25 0 1 0 671229881 1042526208 307 33554432000 134512640 135663704 4288768240 18446744073709551615 134524897 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/10223/statm: 254523 307 113 282 0 254239 0
[pid=10221/tid=10227] ppid=10087 vsize=392808 CPUtime=0 cores=1,3,5,7
/proc/10221/task/10227/stat : 10227 (python) S 10087 10087 9113 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 671239625 402235392 5996 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 514 18446604456560764736 0 0 -1 1 0 0 0
[pid=10228] ppid=10087 vsize=403064 CPUtime=0.85 cores=1,3,5,7
/proc/10228/stat : 10228 (python) S 10087 10087 9113 0 -1 4202560 1169 0 0 0 45 40 0 0 19 0 2 0 671239629 412737536 6008 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488521675 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/10228/statm: 100766 6008 400 364 0 36648 0
[pid=10230] ppid=10228 vsize=268428 CPUtime=194.87 cores=1,3,5,7
/proc/10230/stat : 10230 (hybridGM3) T 10228 10230 10230 0 -1 4202496 4942 0 0 0 19487 0 0 0 25 0 1 0 671239629 274870272 4673 33554432000 134512640 135204385 4288345632 18446744073709551615 134537502 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/10230/statm: 67107 4673 64 169 0 66937 0
[pid=10228/tid=10237] ppid=10087 vsize=403064 CPUtime=0 cores=1,3,5,7
/proc/10228/task/10237/stat : 10237 (python) S 10087 10087 9113 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 671259117 412737536 6008 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 514 18446604462047441024 0 0 -1 7 0 0 0
[pid=10243] ppid=10087 vsize=413320 CPUtime=0.85 cores=1,3,5,7
/proc/10243/stat : 10243 (python) S 10087 10087 9113 0 -1 4202560 1171 0 0 0 44 41 0 0 19 0 2 0 671269367 423239680 6019 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488521675 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/10243/statm: 103330 6019 400 364 0 39209 0
[pid=10245] ppid=10243 vsize=268424 CPUtime=196.89 cores=1,3,5,7
/proc/10245/stat : 10245 (hybridGM3) T 10243 10245 10245 0 -1 4202496 4941 0 0 0 19689 0 0 0 25 0 1 0 671269368 274866176 4672 33554432000 134512640 135204385 4290159184 18446744073709551615 134537363 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/10245/statm: 67106 4672 64 169 0 66936 0
[pid=10243/tid=10253] ppid=10087 vsize=413320 CPUtime=0 cores=1,3,5,7
/proc/10243/task/10253/stat : 10253 (python) S 10087 10087 9113 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 671289057 423239680 6019 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 514 18446604462047442816 0 0 -1 7 0 0 0
[pid=10254] ppid=10087 vsize=423576 CPUtime=3.45 cores=1,3,5,7
/proc/10254/stat : 10254 (python) S 10087 10087 9113 0 -1 4202560 1244 0 0 0 180 165 0 0 15 0 2 0 671289061 433741824 6030 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488521675 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/10254/statm: 105894 6030 407 364 0 41770 0
[pid=10256] ppid=10254 vsize=93920 CPUtime=780.5 cores=1,3,5,7
/proc/10256/stat : 10256 (march_hi) T 10254 10256 10256 0 -1 4202496 24078 0 0 0 78046 4 0 0 25 0 1 0 671289061 96174080 22516 33554432000 4194304 4955518 140736542733728 18446744073709551615 4237786 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/10256/statm: 23480 22516 106 186 0 23293 0
[pid=10254/tid=10271] ppid=10087 vsize=423576 CPUtime=0 cores=1,3,5,7
/proc/10254/task/10271/stat : 10271 (python) S 10087 10087 9113 0 -1 4202560 47 0 0 0 0 0 0 0 15 0 2 0 671328007 433741824 6030 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 514 18446604455695669248 0 0 -1 1 0 0 0
[pid=10291] ppid=10087 vsize=434080 CPUtime=0.86 cores=1,3,5,7
/proc/10291/stat : 10291 (python) S 10087 10087 9113 0 -1 4202560 1227 0 0 0 45 41 0 0 18 0 2 0 671367122 444497920 6090 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488521675 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/10291/statm: 108520 6090 416 364 0 44393 0
[pid=10293] ppid=10291 vsize=268428 CPUtime=196.68 cores=1,3,5,7
/proc/10293/stat : 10293 (hybridGM3) T 10291 10293 10293 0 -1 4202496 4949 0 0 0 19668 0 0 0 25 0 1 0 671367123 274870272 4672 33554432000 134512640 135204385 4288178832 18446744073709551615 134537502 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/10293/statm: 67107 4672 64 169 0 66937 0
[pid=10291/tid=10779] ppid=10087 vsize=434080 CPUtime=0 cores=1,3,5,7
/proc/10291/task/10779/stat : 10779 (python) S 10087 10087 9113 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 671386793 444497920 6090 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 514 18446604454954758208 0 0 -1 7 0 0 0
[pid=10780] ppid=10087 vsize=444336 CPUtime=6.72 cores=1,3,5,7
/proc/10780/stat : 10780 (python) S 10087 10087 9113 0 -1 4202560 1222 0 0 0 348 324 0 0 18 0 2 0 671386795 455000064 6085 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488521675 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/10780/statm: 111084 6085 407 364 0 46954 0
[pid=10782] ppid=10780 vsize=455824 CPUtime=1553.72 cores=1,3,5,7
/proc/10782/stat : 10782 (TNM) T 10780 10782 10782 0 -1 4202496 709 0 0 0 155372 0 0 0 25 0 1 0 671386796 466763776 435 33554432000 134512640 135008748 4293502128 18446744073709551615 134529648 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/10782/statm: 113956 435 46 122 0 112856 0
[pid=10780/tid=10830] ppid=10087 vsize=444336 CPUtime=0 cores=1,3,5,7
/proc/10780/task/10830/stat : 10830 (python) S 10087 10087 9113 0 -1 4202560 45 0 0 0 0 0 0 0 18 0 2 0 671532628 455000064 6085 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 514 18446604461255215232 0 0 -1 7 0 0 0
[pid=10836] ppid=10087 vsize=454592 CPUtime=0.44 cores=1,3,5,7
/proc/10836/stat : 10836 (python) S 10087 10087 9113 0 -1 4202560 1175 0 0 0 24 20 0 0 19 0 2 0 671551915 465502208 6091 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488521675 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/10836/statm: 113648 6091 401 364 0 49515 0
[pid=10838] ppid=10836 vsize=247344 CPUtime=100.44 cores=1,3,5,7
/proc/10838/stat : 10838 (FH) T 10836 10838 10838 0 -1 4202496 718 0 0 0 10044 0 0 0 25 0 1 0 671551916 253280256 443 33554432000 134512640 134936988 4288081856 18446744073709551615 134545118 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/10838/statm: 61836 443 41 104 0 61340 0
[pid=10836/tid=10841] ppid=10087 vsize=454592 CPUtime=0 cores=1,3,5,7
/proc/10836/task/10841/stat : 10841 (python) S 10087 10087 9113 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 671561960 465502208 6091 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 514 18446604455733014656 0 0 -1 7 0 0 0
[pid=10842] ppid=10087 vsize=464848 CPUtime=0 cores=1,3,5,7
/proc/10842/stat : 10842 (python) S 10087 10087 9113 0 -1 4202560 855 0 0 0 0 0 0 0 15 0 1 0 671561962 476004352 6042 33554432000 4194304 5683644 140735766553424 18446744073709551615 239476715078 0 0 16781312 514 5607507802713981191 0 0 17 1 0 0 0
/proc/10842/statm: 116212 6042 349 364 0 52076 0
[pid=10844] ppid=10842 vsize=3032 CPUtime=0.58 cores=1,3,5,7
/proc/10844/stat : 10844 (march_hi) R 10842 10844 10844 0 -1 4202496 2077 0 0 0 58 0 0 0 22 0 1 0 671561962 3104768 520 33554432000 4194304 4955518 140737204679216 18446744073709551615 4237810 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/10844/statm: 758 520 106 186 0 571 0
[pid=10087/tid=10094] ppid=10085 vsize=475092 CPUtime=0 cores=1,3,5,7
/proc/10087/task/10094/stat : 10094 (python) S 10085 10087 9113 0 -1 4202560 27 933 0 0 0 0 0 0 20 0 20 0 671109323 486494208 7611 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 2 18446604468912035968 0 0 -1 7 0 0 0
[pid=10087/tid=10100] ppid=10085 vsize=475092 CPUtime=0 cores=1,3,5,7
/proc/10087/task/10100/stat : 10100 (python) S 10085 10087 9113 0 -1 4202560 28 933 0 0 0 0 0 0 20 0 20 0 671119367 486494208 7611 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 2 18446604468912035968 0 0 -1 3 0 0 0
[pid=10087/tid=10107] ppid=10085 vsize=475092 CPUtime=0 cores=1,3,5,7
/proc/10087/task/10107/stat : 10107 (python) S 10085 10087 9113 0 -1 4202560 33 933 0 0 0 0 0 0 18 0 20 0 671129618 486494208 7611 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 2 18446604468912035968 0 0 -1 3 0 0 0
[pid=10087/tid=10114] ppid=10085 vsize=475092 CPUtime=0 cores=1,3,5,7
/proc/10087/task/10114/stat : 10114 (python) S 10085 10087 9113 0 -1 4202560 32 933 0 0 0 0 0 0 18 0 20 0 671139366 486494208 7611 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 2 18446604468912035968 0 0 -1 7 0 0 0
[pid=10087/tid=10120] ppid=10085 vsize=475092 CPUtime=0 cores=1,3,5,7
/proc/10087/task/10120/stat : 10120 (python) S 10085 10087 9113 0 -1 4202560 28 933 0 0 0 0 0 0 20 0 20 0 671149410 486494208 7611 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 2 18446604468912035968 0 0 -1 7 0 0 0
[pid=10087/tid=10128] ppid=10085 vsize=475092 CPUtime=0 cores=1,3,5,7
/proc/10087/task/10128/stat : 10128 (python) S 10085 10087 9113 0 -1 4202560 25 933 0 0 0 0 0 0 20 0 20 0 671159457 486494208 7611 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 2 18446604468912035968 0 0 -1 3 0 0 0
[pid=10087/tid=10138] ppid=10085 vsize=475092 CPUtime=0 cores=1,3,5,7
/proc/10087/task/10138/stat : 10138 (python) S 10085 10087 9113 0 -1 4202560 30 933 0 0 0 0 0 0 20 0 20 0 671169503 486494208 7611 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 2 18446604468912035968 0 0 -1 7 0 0 0
[pid=10087/tid=10184] ppid=10085 vsize=475092 CPUtime=0 cores=1,3,5,7
/proc/10087/task/10184/stat : 10184 (python) S 10085 10087 9113 0 -1 4202560 29 933 0 0 0 0 0 0 20 0 20 0 671179548 486494208 7611 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 2 18446604468912035968 0 0 -1 1 0 0 0
[pid=10087/tid=10195] ppid=10085 vsize=475092 CPUtime=0 cores=1,3,5,7
/proc/10087/task/10195/stat : 10195 (python) S 10085 10087 9113 0 -1 4202560 27 933 0 0 0 0 0 0 20 0 20 0 671189592 486494208 7611 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 2 18446604468912035968 0 0 -1 7 0 0 0
[pid=10087/tid=10202] ppid=10085 vsize=475092 CPUtime=0 cores=1,3,5,7
/proc/10087/task/10202/stat : 10202 (python) S 10085 10087 9113 0 -1 4202560 27 933 0 0 0 0 0 0 20 0 20 0 671199841 486494208 7611 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 2 18446604468912035968 0 0 -1 3 0 0 0
[pid=10087/tid=10211] ppid=10085 vsize=475092 CPUtime=0 cores=1,3,5,7
/proc/10087/task/10211/stat : 10211 (python) S 10085 10087 9113 0 -1 4202560 28 933 0 0 0 0 0 0 20 0 20 0 671219630 486494208 7611 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 2 18446604468912035968 0 0 -1 3 0 0 0
[pid=10087/tid=10222] ppid=10085 vsize=475092 CPUtime=0 cores=1,3,5,7
/proc/10087/task/10222/stat : 10222 (python) S 10085 10087 9113 0 -1 4202560 25 933 0 0 0 0 0 0 20 0 20 0 671229881 486494208 7611 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 2 18446604468912035968 0 0 -1 7 0 0 0
[pid=10087/tid=10229] ppid=10085 vsize=475092 CPUtime=0 cores=1,3,5,7
/proc/10087/task/10229/stat : 10229 (python) S 10085 10087 9113 0 -1 4202560 26 933 0 0 0 0 0 0 20 0 20 0 671239629 486494208 7611 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 2 18446604468912035968 0 0 -1 1 0 0 0
[pid=10087/tid=10244] ppid=10085 vsize=475092 CPUtime=0 cores=1,3,5,7
/proc/10087/task/10244/stat : 10244 (python) S 10085 10087 9113 0 -1 4202560 30 933 0 0 0 0 0 0 20 0 20 0 671269368 486494208 7611 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 2 18446604468912035968 0 0 -1 1 0 0 0
[pid=10087/tid=10255] ppid=10085 vsize=475092 CPUtime=0 cores=1,3,5,7
/proc/10087/task/10255/stat : 10255 (python) S 10085 10087 9113 0 -1 4202560 29 933 0 0 0 0 0 0 18 0 20 0 671289061 486494208 7611 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 2 18446604468912035968 0 0 -1 1 0 0 0
[pid=10087/tid=10292] ppid=10085 vsize=475092 CPUtime=0 cores=1,3,5,7
/proc/10087/task/10292/stat : 10292 (python) S 10085 10087 9113 0 -1 4202560 27 933 0 0 0 0 0 0 20 0 20 0 671367123 486494208 7611 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 2 18446604468912035968 0 0 -1 1 0 0 0
[pid=10087/tid=10781] ppid=10085 vsize=475092 CPUtime=0 cores=1,3,5,7
/proc/10087/task/10781/stat : 10781 (python) S 10085 10087 9113 0 -1 4202560 26 933 0 0 0 0 0 0 18 0 20 0 671386795 486494208 7611 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 2 18446604468912035968 0 0 -1 1 0 0 0
[pid=10087/tid=10837] ppid=10085 vsize=475092 CPUtime=0 cores=1,3,5,7
/proc/10087/task/10837/stat : 10837 (python) S 10085 10087 9113 0 -1 4202560 28 933 0 0 0 0 0 0 20 0 20 0 671551915 486494208 7611 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 2 18446604468912035968 0 0 -1 1 0 0 0
[pid=10087/tid=10843] ppid=10085 vsize=475092 CPUtime=0 cores=1,3,5,7
/proc/10087/task/10843/stat : 10843 (python) S 10085 10087 9113 0 -1 4202560 28 933 0 0 0 0 0 0 20 0 20 0 671561962 486494208 7611 33554432000 4194304 5683644 140735766553424 18446744073709551615 239488518577 0 0 16781312 2 18446604468912035968 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4546.61
Current children cumulated vsize (KiB) 16149336

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 10087 and gives
#  childrusage.ru_utime.tv_sec=1
#  childrusage.ru_utime.tv_usec=476775
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=148977
# CPU time returned by wait4() is 1.62575
# while last known CPU time is 4546.61
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 4528.05
CPU time (s): 4546.61
CPU user time (s): 4537.04
CPU system time (s): 9.57
CPU usage (%): 100.41
Max. virtual memory (cumulated for all children) (KiB): 16149336

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1.47677
system time used= 0.148977
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 29757
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1062
involuntary context switches= 46

runsolver used 43.9173 second user time and 97.5782 second system time

The end

Launcher Data 

Begin job on node148 at 2011-05-17 03:18:41
IDJOB=3386955
IDBENCH=83020
IDSOLVER=1872
FILE ID=node148/3386955-1305595121
RUNJOBID= node148-1305586142-9132
PBS_JOBID= 13350330
Free space on /tmp= 72836 MiB

SOLVER NAME= borg-sat 11.03.02-seq-random (fixed)
BENCH NAME= SAT11/random/medium/unif-k7-r89-v90-c8010-S423984545-074.UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3386955-1305595121/watcher-3386955-1305595121 -o /tmp/evaluation-result-3386955-1305595121/solver-3386955-1305595121 -C 5000 -W 5100 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3386955-1305595121.cnf -seed 2080514635 -budget 5000

TIME LIMIT= 5000 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 2ef214e0162b22d1093a804ea2af97a1
RANDOM SEED=2080514635

node148.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      20576456 kB
Buffers:        359416 kB
Cached:        5099336 kB
SwapCached:       8488 kB
Active:        8767684 kB
Inactive:      3215824 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      20576456 kB
SwapTotal:    67111528 kB
SwapFree:     67056568 kB
Dirty:           32384 kB
Writeback:           0 kB
AnonPages:     6522672 kB
Mapped:          46976 kB
Slab:           254704 kB
PageTables:      65464 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 162251768 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72960 MiB
End job on node148 at 2011-05-17 04:34:12