[Version Information]
Phase2 (IST-Java-ICRA) WALi-OpenNWA version: 9042782a9a5ed8987eef2dfeec4e384ee950f24c (2017-05-01 17:41:53 -0500) "Adding benchmarks for model of malware analyzer."
Phase2 (IST-Java-ICRA) duet version: 9c9e95ed58abc89f69fc775b6f484eb2863d8b6e (2017-05-09 22:35:28 -0400) "Added extra bytecode costs for thread.sleep"

Phase3 WALi-OpenNWA version: 9042782a9a5ed8987eef2dfeec4e384ee950f24c (2017-05-01 17:41:53 -0500) "Adding benchmarks for model of malware analyzer."
Phase3 duet version: 0fa8064dfa94888565bc782359017482a22e6f46 (2017-05-16 08:41:47 -0400) "Fixed build after merge"
# Installed packages for 4.02.1+PIC:
apron              20160125  APRON numerical abstract domain library
base-bigarray          base  Bigarray library distributed with the OCaml compile
base-bytes             base  Bytes library distributed with the OCaml compiler
base-ocamlbuild        base  OCamlbuild binary and libraries distributed with th
base-threads           base  Threads library distributed with the OCaml compiler
base-unix              base  Unix library distributed with the OCaml compiler
batteries             2.5.3  a community-maintained standard library extension
camlidl                1.05  Stub code generator for OCaml
camlp4               4.02+7  Camlp4 is a system for writing extensible parsers f
cil                20150825  A front-end for the C programming language that fac
conf-gmp                  1  Virtual package relying on a GMP lib system install
conf-m4                   1  Virtual package relying on m4
conf-mathsat              1  Virtual package relying on a MathSAT system install
conf-mpfr                 1  Virtual package relying on library MPFR installatio
conf-perl                 1  Virtual package relying on perl
conf-python-2-7         1.0  Virtual package relying on Python-2.7 installation.
conf-which                1  Virtual package relying on which
cppo                  1.4.1  Equivalent of the C preprocessor for OCaml programs
deriving           20140904  Extension to OCaml for deriving functions from type
mathsat            20161206  MathSAT 5 SMT solver
menhir             20170101  LR(1) parser generator
mlgmpidl              1.2.4  OCaml interface to the GMP library
oasis                 0.4.8  Tooling for building OCaml libraries and applicatio
ocamlbuild                0  Build system distributed with the OCaml compiler si
ocamlfind             1.7.1  A library manager for OCaml
ocamlgraph            1.8.7  A generic graph library for OCaml
ocamlify              0.0.1  Include files in OCaml code
ocamlmod              0.0.8  Generate OCaml modules from source files
OCRS               20170323  Recurrence solver based on operational calculus
optcomp                 1.6  Optional compilation with cpp-like directives
ounit                 2.0.0  Unit testing framework loosely based on HUnit. It i
ppx_deriving            4.1  Type-driven code generation for OCaml >=4.02
ppx_tools        5.0+4.02.0  Tools for authors of ppx rewriters and other syntac
result                  1.2  Compatibility Result module
Z3                 20161217  Z3 SMT solver

Test Name Output No. of Rounds Result Time (Prev.)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/c4b
gcd.c [OCRS][without] 1 FAIL
0.354
kmp.c [OCRS][without] 1 PASS
1.366 (1.192)
qsort.c [OCRS][without] 5 PASS
3.579 (2.046)
speed_pldi09_fig1.c [OCRS][without] 1 PASS
0.387
speed_pldi09_fig4_2.c [OCRS][without] 1 FAIL
0.404
speed_pldi09_fig4_4.c [OCRS][without] 1 PASS
0.449
speed_pldi09_fig4_5.c [OCRS][without] 1 PASS
0.365
speed_pldi10_ex1.c [OCRS][without] 1 PASS
3.854 (0.822)
speed_pldi10_ex3.c [OCRS][without] 1 PASS
0.446
speed_pldi10_ex4.c [OCRS][without] EXCEPTION
0.418
speed_popl10_fig2_1.c [OCRS][without] 1 PASS
0.394
speed_popl10_fig2_2.c [OCRS][without] 1 FAIL
0.362
speed_popl10_nested_multiple.c [OCRS][without] 1 PASS
0.426
speed_popl10_nested_single.c [OCRS][without] 1 PASS
0.4
speed_popl10_sequential_single.c [OCRS][without] 1 PASS
0.362
speed_popl10_simple_multiple.c [OCRS][without] 1 PASS
0.405
speed_popl10_simple_single_2.c [OCRS][without] 1 PASS
0.423
speed_popl10_simple_single.c [OCRS][without] 1 PASS
0.359
t07.c [OCRS][without] 1 PASS
0.419 (0.368)
t08.c [OCRS][without] 1 PASS
0.361
t09.c [OCRS][without] 1 PASS
0.352
t10.c [OCRS][without] 1 PASS
0.347
t11.c [OCRS][without] 1 PASS
0.4
t13.c [OCRS][without] 1 FAIL
0.403
t15.c [OCRS][without] 1 PASS
0.372
t16.c [OCRS][without] 1 PASS
0.367
t19.c [OCRS][without] 1 PASS
0.363
t20.c [OCRS][without] 1 PASS
0.348
t27.c [OCRS][without] 1 PASS
0.439
t28.c [OCRS][without] 1 PASS
0.397
t30.c [OCRS][without] 1 FAIL
0.371
t37.c [OCRS][without] 2 PASS
0.495
t39.c [OCRS][without] 2 PASS
0.498
t46.c [OCRS][without] 1 PASS
0.361
t47.c [OCRS][without] 1 PASS
0.372
Total Below Time = 21.418 (was 16.525)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/misc-recursive
mult-rec.c [OCRS][without] 2 PASS
PASS
0.389
qsort_full.c [OCRS][without] 5 TIMEOUT
300.009 (43.093)
rec1-param_safe.c [OCRS][without] 2 PASS
PASS
PASS
0.34
rec1-param_unsafe.c [OCRS][without] 2 PASS
0.349
rec1_safe.c [OCRS][without] 2 PASS
PASS
PASS
0.353
rec1_unsafe.c [OCRS][without] 2 PASS
0.339
rec2-param_safe.c [OCRS][without] 2 PASS
PASS
PASS
0.351
rec2-param_unsafe.c [OCRS][without] 2 PASS
0.346
rec2_safe.c [OCRS][without] 2 PASS
PASS
PASS
0.334
rec2_unsafe.c [OCRS][without] 2 PASS
0.344
rec-test.c [OCRS][without] 2 PASS
0.347
sas03_bothbranches.c [OCRS][without] 7 PASS
1.304
sas03.c [OCRS][without] 2 PASS
0.815
simulated_lese_recursive.c [OCRS][without] 2 PASS
0.395
Total Below Time = 306.015 (was 49.059)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/duet
count_up_down_safe.c [OCRS][without] 1 PASS
0.341
count_up_down_unsafe.c [OCRS][without] 1 PASS
0.331
divide.c [OCRS][without] 1 PASS
0.33
factor_unsafe.c [OCRS][without] 1 PASS
0.326 (300.004)
for_infinite_loop_1_safe.c [OCRS][without] 1 PASS
0.317
for_infinite_loop_2_safe.c [OCRS][without] 1 PASS
0.325
interproc.c [OCRS][without] 1 PASS
0.318
mult.c [OCRS][without] 1 PASS
PASS
0.328
pointer_arith.c [OCRS][without] 1 PASS
PASS
PASS
PASS
0.335
quotient.c [OCRS][without] 3 PASS
0.429
subtract.c [OCRS][without] 1 PASS
0.32
sum01_bug02_sum01_bug02_base.case_unsafe.c [OCRS][without] 1 PASS
0.343
sum01_bug02_unsafe.c [OCRS][without] 1 PASS
0.384
sum01_real_safe.c [OCRS][without] 1 PASS
0.322
sum01_safe.c [OCRS][without] 1 PASS
0.344
sum01_unsafe.c [OCRS][without] 1 PASS
0.375
sum02_safe.c [OCRS][without] 1 PASS
0.336
sum03_safe.c [OCRS][without] 1 PASS
0.342
sum03_unsafe.c [OCRS][without] 1 PASS
0.416
sum04_safe.c [OCRS][without] 1 PASS
0.323
sum04_unsafe.c [OCRS][without] 1 PASS
0.396
terminator_02_safe.c [OCRS][without] 1 PASS
0.334
terminator_02_unsafe.c [OCRS][without] 1 PASS
0.355
terminator_03_safe.c [OCRS][without] 1 PASS
0.339
terminator_03_unsafe.c [OCRS][without] 1 PASS
0.336
token_ring01_safe.c [OCRS][without] TIMEOUT
300.007
token_ring01_unsafe.c [OCRS][without] TIMEOUT
300.006
toy_safe.c [OCRS][without] EXCEPTION
178.473
trex01_safe.c [OCRS][without] 1 PASS
0.722
trex01_unsafe.c [OCRS][without] 1 PASS
0.38
trex02_safe2.c [OCRS][without] 3 PASS
0.399
trex02_safe.c [OCRS][without] 3 PASS
0.399
trex02_unsafe.c [OCRS][without] 3 PASS
0.411
trex03_safe.c [OCRS][without] 1 PASS
0.478
trex03_unsafe.c [OCRS][without] 1 PASS
0.463
trex04_safe.c [OCRS][without] 1 PASS
0.347
while_infinite_loop_1_safe.c [OCRS][without] 1 PASS
0.325
while_infinite_loop_2_safe.c [OCRS][without] 1 PASS
0.315
while_infinite_loop_3_safe.c [OCRS][without] 1 PASS
0.377 (0.325)
Total Below Time = 791.747 (was 1094.058)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests
above_litmus.c [OCRS][without] 3 PASS
0.473
blogger_simplified1.c [OCRS][without] 3 PASS
1.006
divided_difference2.c [OCRS][without] TIMEOUT
300.006
mult-proc.c [OCRS][without] 3 PASS
PASS
0.359
mult-proc-params.c [OCRS][without] 3 PASS
PASS
0.404
popall.c [OCRS][without] 1 PASS
PASS
PASS
PASS
PASS
0.412
simulated_scc.c [OCRS][without] 1 PASS
PASS
0.543
Total Below Time = 303.203 (was 303.164)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/STAC/polynomial/assert
degree1_monomial.c [OCRS][without] 1 PASS
0.349
degree2_binomial.c [OCRS][without] 1 PASS
0.469
degree2_monomial.c [OCRS][without] 1 PASS
0.373
degree3_binomial.c [OCRS][without] 1 PASS
0.896
degree3_monomial.c [OCRS][without] 1 PASS
0.433
degree4_binomial.c [OCRS][without] 1 PASS
1.164
degree4_monomial.c [OCRS][without] 1 PASS
0.477
degree5_binomial.c [OCRS][without] 1 PASS
1.74
degree5_monomial.c [OCRS][without] 1 PASS
0.576
Total Below Time = 6.477 (was 6.503)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/snlee/snlee_tests
cubic_with_square.c [OCRS][without] 1 PASS
0.344
cubic_with_square_interproc.c [OCRS][without] 2 PASS
0.403
cubic_with_square_nonlinear.c [OCRS][without] 1 PASS
0.345
cubic_with_square_nonlinear_interproc.c [OCRS][without] 2 PASS
0.393
cubic_with_square_nonlinear_unsafe.c [OCRS][without] 1 PASS
0.366
cubic_with_square_unsafe.c [OCRS][without] 1 PASS
0.345
multi-input.c [OCRS][without] 1 PASS
0.415
multi-input_unsafe.c [OCRS][without] 1 PASS
0.44 (0.5)
nondet_loop_bound.c [OCRS][without] 1 PASS
0.365
nondet_loop_bound_quartic.c [OCRS][without] 1 PASS
0.368
nondet_loop_bound_quartic_unsafe.c [OCRS][without] 1 PASS
0.367
nondet_loop_bound_unsafe.c [OCRS][without] 1 PASS
0.399
nonlinear_stratified.c [OCRS][without] 1 FAIL
0.361
nonlinear_stratified_unsafe.c [OCRS][without] 1 PASS
0.381
quartic_with_cube.c [OCRS][without] 1 PASS
0.347
quartic_with_cube_nonlinear.c [OCRS][without] 1 PASS
0.365
quartic_with_cube_nonlinear_unsafe.c [OCRS][without] 1 PASS
0.388
quartic_with_cube_unsafe.c [OCRS][without] 1 PASS
0.358
quintic_with_quartic.c [OCRS][without] 1 PASS
0.372
quintic_with_quartic_nonlinear.c [OCRS][without] 1 PASS
0.397
quintic_with_quartic_nonlinear_unsafe.c [OCRS][without] 1 PASS
0.384 (0.465)
quintic_with_quartic_unsafe.c [OCRS][without] 1 PASS
0.357
Total Below Time = 8.26 (was 8.314)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/STAC/FiniteDifferencing
degree2.c [OCRS][without] 1 PASS
0.517
degree2_FD_AllAux_AllAssm.c [OCRS][without] 1 PASS
3.586 (3.15)
degree2_FD_AllAux_FewAssm.c [OCRS][without] 1 PASS
5.547 (3.154)
degree2_FD_FewAux_FewAssm.c [OCRS][without] 1 PASS
0.715
degree2_FD_FewAux_FewAssm_NonLinAsrt.c [OCRS][without] 1 PASS
0.72
degree3.c [OCRS][without] 1 PASS
0.564
degree3_FD.c [OCRS][without] 1 PASS
0.745
degree4.c [OCRS][without] 1 PASS
0.397
Total Below Time = 12.791 (was 9.921)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/STAC/LESE
loop.c [OCRS][without] 1 PASS
0.397
loop_unsafe.c [OCRS][without] 1 PASS
0.421
simulated_lese_parser.c [OCRS][without] 1 PASS
0.333
simulated_lese_sentinel.c [OCRS][without] 1 PASS
0.344
Total Below Time = 1.495 (was 1.469)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/STAC/LowerBound
lower_bound.c [OCRS][without] 1 PASS
PASS
PASS
0.342
Total Below Time = 0.342 (was 0.346)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/STAC/LZ
lz77.c [OCRS][without] TIMEOUT
300.005
Total Below Time = 300.005 (was 300.009)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/sv-benchmarks/loop-acceleration
array_false-unreach-call1.c [OCRS][without] 1 PASS
0.354
array_false-unreach-call2.c [OCRS][without] 1 PASS
0.361
array_false-unreach-call3.c [OCRS][without] 1 PASS
0.359
array_true-unreach-call1.c [OCRS][without] 1 FAIL
0.344
array_true-unreach-call2.c [OCRS][without] 1 FAIL
0.371
array_true-unreach-call3.c [OCRS][without] 1 PASS
0.341
array_true-unreach-call4.c [OCRS][without] 1 FAIL
0.336
const_false-unreach-call1.c [OCRS][without] 1 PASS
0.325
const_true-unreach-call1.c [OCRS][without] 1 PASS
0.327
diamond_false-unreach-call1.c [OCRS][without] 1 PASS
0.345
diamond_true-unreach-call1.c [OCRS][without] 1 PASS
0.337
diamond_true-unreach-call2.c [OCRS][without] 1 PASS
0.377
functions_false-unreach-call1.c [OCRS][without] 3 PASS
0.362
functions_true-unreach-call1.c [OCRS][without] 3 PASS
0.356
multivar_false-unreach-call1.c [OCRS][without] 1 PASS
0.327
multivar_true-unreach-call1.c [OCRS][without] 1 PASS
0.317
nested_false-unreach-call1.c [OCRS][without] 1 PASS
0.341
nested_true-unreach-call1.c [OCRS][without] 1 PASS
0.336
overflow_true-unreach-call1.c [OCRS][without] 1 PASS
0.322
phases_false-unreach-call1.c [OCRS][without] 1 PASS
0.355
phases_false-unreach-call2.c [OCRS][without] 1 PASS
0.336
phases_true-unreach-call1.c [OCRS][without] 1 PASS
0.344
phases_true-unreach-call2.c [OCRS][without] 1 PASS
0.342
simple_false-unreach-call1.c [OCRS][without] 1 PASS
0.312
simple_false-unreach-call2.c [OCRS][without] 1 PASS
0.314
simple_false-unreach-call3.c [OCRS][without] 1 PASS
0.322
simple_false-unreach-call4.c [OCRS][without] 1 PASS
0.323
simple_true-unreach-call1.c [OCRS][without] 1 PASS
0.324
simple_true-unreach-call2.c [OCRS][without] 1 PASS
0.324
simple_true-unreach-call3.c [OCRS][without] 1 PASS
0.328
simple_true-unreach-call4.c [OCRS][without] 1 PASS
0.324
underapprox_false-unreach-call1.c [OCRS][without] 1 PASS
0.328
underapprox_false-unreach-call2.c [OCRS][without] 1 PASS
0.323
underapprox_true-unreach-call1.c [OCRS][without] 1 FAIL
0.324
underapprox_true-unreach-call2.c [OCRS][without] 1 PASS
0.322
Total Below Time = 11.783 (was 11.812)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/sv-benchmarks/loop-invgen
apache-escape-absolute_true-unreach-call.c [OCRS][without] 1 PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
PASS
0.835
apache-get-tag_true-unreach-call.c [OCRS][without] 1 PASS
PASS
PASS
FAIL
PASS
FAIL
PASS
PASS
PASS
FAIL
PASS
PASS
PASS
FAIL
PASS
FAIL
0.467
down_true-unreach-call.c [OCRS][without] 1 PASS
0.344
fragtest_simple_true-unreach-call.c [OCRS][without] 1 PASS
0.391
half_2_true-unreach-call.c [OCRS][without] 1 PASS
0.35
heapsort_true-unreach-call.c [OCRS][without] 1 TIMEOUT
300.167 (2.321)
id_build_true-unreach-call.c [OCRS][without] 1 PASS
PASS
0.322
id_trans_false-unreach-call.c [OCRS][without] 1 FAIL
FAIL
FAIL
PASS
0.334
large_const_true-unreach-call.c [OCRS][without] 1 PASS
0.421
MADWiFi-encode_ie_ok_true-unreach-call.c [OCRS][without] 1 PASS
PASS
0.375
nested6_true-unreach-call.c [OCRS][without] 1 FAIL
FAIL
PASS
0.415
nested9_true-unreach-call.c [OCRS][without] 1 PASS
0.466
nest-if3_true-unreach-call.c [OCRS][without] 1 PASS
0.418
NetBSD_loop_true-unreach-call.c [OCRS][without] 1 PASS
PASS
0.342
sendmail-close-angle_true-unreach-call.c [OCRS][without] 1 PASS
FAIL
PASS
FAIL
PASS
FAIL
PASS
FAIL
0.393
seq_true-unreach-call.c [OCRS][without] 1 PASS
0.371
SpamAssassin-loop_true-unreach-call.c [OCRS][without] TIMEOUT
300.119 (1.214)
string_concat-noarr_true-unreach-call.c [OCRS][without] 1 PASS
0.344
up_true-unreach-call.c [OCRS][without] 1 PASS
0.33
Total Below Time = 607.204 (was 10.392)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/sv-benchmarks/loop-lit
afnp2014_true-unreach-call.c [OCRS][without] 1 PASS
0.329
bhmr2007_true-unreach-call.c [OCRS][without] 1 PASS
0.363
cggmp2005b_true-unreach-call.c [OCRS][without] 1 PASS
0.395
cggmp2005_true-unreach-call.c [OCRS][without] 1 PASS
0.334
cggmp2005_variant_true-unreach-call.c [OCRS][without] 1 PASS
0.347
css2003_true-unreach-call.c [OCRS][without] 1 PASS
0.481
ddlm2013_true-unreach-call.c [OCRS][without] 1 PASS
0.398
gcnr2008_false-unreach-call.c [OCRS][without] 1 TIMEOUT
300.168 (0.867)
gj2007b_true-unreach-call.c [OCRS][without] 1 PASS
FAIL
0.357
gj2007_true-unreach-call.c [OCRS][without] 1 PASS
0.354
gr2006_true-unreach-call.c [OCRS][without] 1 PASS
0.353
gsv2008_true-unreach-call.c [OCRS][without] 1 PASS
0.32
hhk2008_true-unreach-call.c [OCRS][without] 1 PASS
0.326
jm2006_true-unreach-call.c [OCRS][without] 1 PASS
0.355
jm2006_variant_true-unreach-call.c [OCRS][without] 1 PASS
0.373
mcmillan2006_true-unreach-call.c [OCRS][without] 1 FAIL
0.383
Total Below Time = 305.636 (was 6.284)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/sv-benchmarks/loop-new
count_by_1_true-unreach-call.c [OCRS][without] 1 PASS
0.331
count_by_1_variant_true-unreach-call.c [OCRS][without] 1 PASS
0.336
count_by_2_true-unreach-call.c [OCRS][without] 1 PASS
0.323
count_by_k_true-unreach-call.c [OCRS][without] 1 PASS
0.331
count_by_nondet_true-unreach-call.c [OCRS][without] 1 FAIL
0.342
gauss_sum_true-unreach-call.c [OCRS][without] 1 PASS
0.345
half_true-unreach-call.c [OCRS][without] EXCEPTION
0.339
nested_true-unreach-call.c [OCRS][without] 1 PASS
0.385
Total Below Time = 2.732 (was 2.746)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/sv-benchmarks/loops
array_false-unreach-call.c [OCRS][without] 1 PASS
0.426
array_true-unreach-call.c [OCRS][without] 1 FAIL
0.435
bubble_sort_false-unreach-call.c [OCRS][without] 4 PASS
PASS
142.413
bubble_sort_true-unreach-call.c [OCRS][without] 1 3.901 (300.004)
compact_false-unreach-call.c [OCRS][without] 1 PASS
0.369
count_up_down_false-unreach-call_true-termination.c [OCRS][without] 1 PASS
0.334
count_up_down_true-unreach-call_true-termination.c [OCRS][without] 1 PASS
0.334
eureka_01_false-unreach-call.c [OCRS][without] 1 PASS
2.273
eureka_01_true-unreach-call.c [OCRS][without] 1 FAIL
2.005
eureka_05_true-unreach-call.c [OCRS][without] 1 FAIL
1.106 (0.764)
for_bounded_loop1_false-unreach-call_true-termination.c [OCRS][without] 1 FAIL
FAIL
PASS
0.334
for_infinite_loop_1_true-unreach-call_false-termination.c [OCRS][without] 1 PASS
0.339
for_infinite_loop_2_true-unreach-call_false-termination.c [OCRS][without] 1 PASS
0.329
heavy_false-unreach-call.c [OCRS][without] 1 PASS
0.546
heavy_true-unreach-call.c [OCRS][without] 1 PASS
0.515
insertion_sort_false-unreach-call.c [OCRS][without] 1 PASS
2.216
insertion_sort_true-unreach-call.c [OCRS][without] 1 FAIL
0.78
invert_string_false-unreach-call.c [OCRS][without] 1 PASS
0.498
invert_string_true-unreach-call.c [OCRS][without] 1 FAIL
0.517
linear_sea.ch_true-unreach-call.c [OCRS][without] 1 FAIL
0.378
linear_search_false-unreach-call.c [OCRS][without] 1 EXCEPTION
0.442
lu.cmp_true-unreach-call.c [OCRS][without] TIMEOUT
300.257 (125.445)
ludcmp_false-unreach-call.c [OCRS][without] TIMEOUT
300.266 (123.075)
matrix_false-unreach-call_true-termination.c [OCRS][without] 1 PASS
1.317 (0.813)
matrix_true-unreach-call_true-termination.c [OCRS][without] 1 FAIL
0.595 (0.51)
n.c11_true-unreach-call.c [OCRS][without] 3 FAIL
0.557 (0.447)
n.c24_false-unreach-call.c [OCRS][without] 3 PASS
27.853 (23.907)
n.c40_true-unreach-call.c [OCRS][without] 1 FAIL
0.416
nec11_false-unreach-call.c [OCRS][without] 1 PASS
0.398
nec20_false-unreach-call.c [OCRS][without] 1 PASS
0.401 (0.358)
nec40_true-unreach-call.c [OCRS][without] 1 FAIL
0.404
string_false-unreach-call.c [OCRS][without] 1 PASS
0.682
string_true-unreach-call.c [OCRS][without] 1 FAIL
0.713 (0.638)
sum01_bug02_false-unreach-call_true-termination.c [OCRS][without] 1 PASS
0.417
sum01_bug02_sum01_bug02_base.case_false-unreach-call_true-termination.c [OCRS][without] 1 PASS
0.353
sum01_false-unreach-call_true-termination.c [OCRS][without] 1 PASS
0.413
sum01_true-unreach-call_true-termination.c [OCRS][without] 1 PASS
0.368
sum03_false-unreach-call_true-termination.c [OCRS][without] 1 PASS
0.527
sum03_true-unreach-call_false-termination.c [OCRS][without] 1 PASS
0.403 (0.352)
sum04_false-unreach-call_true-termination.c [OCRS][without] 1 PASS
0.373
sum04_true-unreach-call_true-termination.c [OCRS][without] 1 PASS
0.334
sum_array_false-unreach-call.c [OCRS][without] 1 PASS
0.682
sum_array_true-unreach-call.c [OCRS][without] 1 FAIL
0.726
terminator_01_false-unreach-call_false-termination.c [OCRS][without] 1 PASS
0.344
terminator_02_false-unreach-call_true-termination.c [OCRS][without] 3 PASS
0.447 (0.404)
terminator_02_true-unreach-call_true-termination.c [OCRS][without] 2 PASS
0.368
terminator_03_false-unreach-call_true-termination.c [OCRS][without] 1 PASS
0.33
terminator_03_true-unreach-call_true-termination.c [OCRS][without] 1 PASS
0.346
trex01_false-unreach-call_true-termination.c [OCRS][without] 3 PASS
0.505 (0.45)
trex01_true-unreach-call.c [OCRS][without] 3 PASS
0.86 (0.681)
trex02_false-unreach-call_true-termination.c [OCRS][without] 3 PASS
0.422
trex02_true-unreach-call_true-termination.c [OCRS][without] 3 PASS
0.428
trex03_false-unreach-call_true-termination.c [OCRS][without] 3 PASS
0.933
trex03_true-unreach-call.c [OCRS][without] 3 PASS
0.934 (0.804)
trex04_true-unreach-call_false-termination.c [OCRS][without] 1 PASS
0.401
veris.c_NetBSD-libc__loop_true-unreach-call.c [OCRS][without] 1 PASS
0.356
veris.c_OpenSER__cases1_stripFullBoth_arr_true-unreach-call.c [OCRS][without] 3 27.762 (24.879)
veris.c_sendmail__tTflag_arr_one_loop_true-unreach-call.c [OCRS][without] 1 PASS
0.389
verisec_NetBSD-libc__loop_false-unreach-call.c [OCRS][without] 1 PASS
0.361
verisec_OpenSER__cases1_stripFullBoth_arr_false-unreach-call.c [OCRS][without] 3 PASS
18.951 (16.662)
vogal_false-unreach-call.c [OCRS][without] 1 PASS
0.871 (0.769)
vogal_true-unreach-call.c [OCRS][without] 1 FAIL
0.906 (0.759)
while_infinite_loop_1_true-unreach-call_false-termination.c [OCRS][without] 1 PASS
0.324
while_infinite_loop_2_true-unreach-call_false-termination.c [OCRS][without] 1 PASS
0.325
while_infinite_loop_3_true-unreach-call_false-termination.c [OCRS][without] 1 PASS
0.315
while_infinite_loop_4_false-unreach-call_true-termination.c [OCRS][without] 1 PASS
0.348
Total Below Time = 856.171 (was 786.358)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/sv-benchmarks/recursive
Ackermann01_true-unreach-call.c [OCRS][without] TIMEOUT
300.031 (2.482)
Ackermann02_false-unreach-call_false-termination.c [OCRS][without] TIMEOUT
300.023 (2.368)
Ackermann03_true-unreach-call.c [OCRS][without] TIMEOUT
300.024 (2.45)
Ackermann04_true-unreach-call.c [OCRS][without] TIMEOUT
300.024 (2.494)
Addition01_true-unreach-call_true-termination.c [OCRS][without] 2 PASS
0.635
Addition02_false-unreach-call_false-termination.c [OCRS][without] 2 PASS
0.604
Addition03_false-no-overflow.c [OCRS][without] 2 PASS
0.6
Addition03_true-unreach-call.c [OCRS][without] 2 PASS
0.596
BallRajamani-SPIN2000-Fig1_false-unreach-call.c [OCRS][without] 2 PASS
0.448
EvenOdd01_true-unreach-call_true-termination.c [OCRS][without] 2 PASS
0.434
EvenOdd03_false-unreach-call_false-termination.c [OCRS][without] 2 PASS
0.407
Fibonacci01_true-unreach-call.c [OCRS][without] EXCEPTION
0.938 (300.011)
Fibonacci02_true-unreach-call_true-termination.c [OCRS][without] EXCEPTION
0.933 (300.008)
Fibonacci03_true-unreach-call_true-termination.c [OCRS][without] EXCEPTION
0.956 (300.01)
Fibonacci04_false-unreach-call_true-termination.c [OCRS][without] EXCEPTION
0.962 (300.01)
Fibonacci05_false-unreach-call_true-termination.c [OCRS][without] EXCEPTION
0.964 (300.009)
gcd01_true-unreach-call_true-termination.c [OCRS][without] 2 PASS
0.555
gcd02_true-unreach-call.c [OCRS][without] 2 FAIL
PASS
1.401 (1.188)
McCarthy91_false-unreach-call_false-termination.c [OCRS][without] 7 PASS
1.52 (1.154)
McCarthy91_true-unreach-call.c [OCRS][without] 7 FAIL
1.463 (1.14)
MultCommutative_true-unreach-call_true-termination.c [OCRS][without] 2 FAIL
0.696 (0.591)
Primes_true-unreach-call.c [OCRS][without] 3 FAIL
2.101
recHanoi01_true-unreach-call_true-termination.c [OCRS][without] 9 FAIL
5.901
recHanoi02_true-unreach-call_true-termination.c [OCRS][without] 2 FAIL
0.376
recHanoi03_true-unreach-call_true-termination.c [OCRS][without] 2 FAIL
0.395
Total Below Time = 1222.987 (was 1526.852)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/sv-benchmarks/recursive-simple
afterrec_2calls_false-unreach-call.c [OCRS][without] 2 PASS
PASS
0.418
afterrec_2calls_true-unreach-call.c [OCRS][without] 2 PASS
PASS
0.383
afterrec_false-unreach-call.c [OCRS][without] 2 PASS
0.36
afterrec_true-unreach-call.c [OCRS][without] 2 PASS
0.351
fibo_10_false-unreach-call.c [OCRS][without] EXCEPTION
1.023 (300.01)
fibo_10_true-unreach-call.c [OCRS][without] EXCEPTION
0.931 (300.005)
fibo_15_false-unreach-call.c [OCRS][without] EXCEPTION
0.948 (300.009)
fibo_15_true-unreach-call.c [OCRS][without] EXCEPTION
0.93 (300.011)
fibo_20_false-unreach-call.c [OCRS][without] EXCEPTION
0.964 (300.008)
fibo_20_true-unreach-call.c [OCRS][without] EXCEPTION
0.934 (300.009)
fibo_25_false-unreach-call.c [OCRS][without] EXCEPTION
0.945 (300.011)
fibo_25_true-unreach-call.c [OCRS][without] EXCEPTION
0.978 (300.005)
fibo_2calls_10_false-unreach-call.c [OCRS][without] EXCEPTION
17.782 (130.129)
fibo_2calls_10_true-unreach-call.c [OCRS][without] EXCEPTION
17.539 (132.597)
fibo_2calls_15_false-unreach-call.c [OCRS][without] EXCEPTION
18.165 (132.259)
fibo_2calls_15_true-unreach-call.c [OCRS][without] EXCEPTION
18.052 (130.99)
fibo_2calls_20_false-unreach-call.c [OCRS][without] EXCEPTION
18.027 (133.172)
fibo_2calls_20_true-unreach-call.c [OCRS][without] EXCEPTION
17.838 (134.009)
fibo_2calls_25_false-unreach-call.c [OCRS][without] EXCEPTION
17.613 (130.286)
fibo_2calls_25_true-unreach-call.c [OCRS][without] EXCEPTION
18.737 (131.984)
fibo_2calls_2_false-unreach-call.c [OCRS][without] EXCEPTION
19.527 (130.311)
fibo_2calls_2_true-unreach-call.c [OCRS][without] EXCEPTION
20.198 (132.068)
fibo_2calls_4_false-unreach-call.c [OCRS][without] EXCEPTION
19.344 (132.107)
fibo_2calls_4_true-unreach-call.c [OCRS][without] EXCEPTION
19.402 (130.824)
fibo_2calls_5_false-unreach-call.c [OCRS][without] EXCEPTION
17.546 (133.221)
fibo_2calls_5_true-unreach-call.c [OCRS][without] EXCEPTION
19.126 (130.453)
fibo_2calls_6_false-unreach-call.c [OCRS][without] EXCEPTION
18.636 (132.425)
fibo_2calls_6_true-unreach-call.c [OCRS][without] EXCEPTION
18.929 (130.422)
fibo_2calls_8_false-unreach-call.c [OCRS][without] EXCEPTION
19.558 (133.708)
fibo_2calls_8_true-unreach-call.c [OCRS][without] EXCEPTION
18.964 (130.714)
fibo_5_false-unreach-call.c [OCRS][without] EXCEPTION
1.025 (300.01)
fibo_5_true-unreach-call.c [OCRS][without] EXCEPTION
1.064 (300.01)
fibo_7_false-unreach-call.c [OCRS][without] EXCEPTION
1.007 (300.009)
fibo_7_true-unreach-call.c [OCRS][without] EXCEPTION
1.042 (300.011)
id2_b2_o3_true-unreach-call.c [OCRS][without] 2 PASS
1.052 (0.952)
id2_b3_o2_false-unreach-call.c [OCRS][without] 2 PASS
1.067
id2_b3_o5_true-unreach-call.c [OCRS][without] 2 PASS
1.071 (0.929)
id2_b5_o10_true-unreach-call.c [OCRS][without] 2 PASS
1.036 (0.915)
id2_i5_o5_false-unreach-call.c [OCRS][without] 2 PASS
0.427
id2_i5_o5_true-unreach-call.c [OCRS][without] 2 PASS
0.428
id_b2_o3_true-unreach-call.c [OCRS][without] 2 PASS
0.611
id_b3_o2_false-unreach-call.c [OCRS][without] 2 PASS
0.643
id_b3_o5_true-unreach-call.c [OCRS][without] 2 PASS
0.58
id_b5_o10_true-unreach-call.c [OCRS][without] 2 PASS
0.595
id_i10_o10_false-unreach-call.c [OCRS][without] 2 PASS
0.378
id_i10_o10_true-unreach-call.c [OCRS][without] 2 PASS
0.385
id_i15_o15_false-unreach-call.c [OCRS][without] 2 PASS
0.383
id_i15_o15_true-unreach-call.c [OCRS][without] 2 PASS
0.372
id_i20_o20_false-unreach-call.c [OCRS][without] 2 PASS
0.391
id_i20_o20_true-unreach-call.c [OCRS][without] 2 PASS
0.363
id_i25_o25_false-unreach-call.c [OCRS][without] 2 PASS
0.378
id_i25_o25_true-unreach-call.c [OCRS][without] 2 PASS
0.382
id_i5_o5_false-unreach-call.c [OCRS][without] 2 PASS
0.383
id_i5_o5_true-unreach-call.c [OCRS][without] 2 PASS
0.41 (0.359)
id_o1000_false-unreach-call.c [OCRS][without] 2 PASS
0.39
id_o100_false-unreach-call.c [OCRS][without] 2 PASS
0.379
id_o10_false-unreach-call.c [OCRS][without] 2 PASS
0.373
id_o200_false-unreach-call.c [OCRS][without] 2 PASS
0.421 (0.376)
id_o20_false-unreach-call.c [OCRS][without] 2 PASS
0.413 (0.373)
id_o3_false-unreach-call.c [OCRS][without] 2 PASS
0.436 (0.374)
sum_10x0_false-unreach-call.c [OCRS][without] 2 PASS
0.414
sum_10x0_true-unreach-call.c [OCRS][without] 2 PASS
0.413
sum_15x0_false-unreach-call.c [OCRS][without] 2 PASS
0.435
sum_15x0_true-unreach-call.c [OCRS][without] 2 PASS
0.417 (0.37)
sum_20x0_false-unreach-call.c [OCRS][without] 2 PASS
0.433
sum_20x0_true-unreach-call.c [OCRS][without] 2 PASS
0.413
sum_25x0_false-unreach-call.c [OCRS][without] 2 PASS
0.442 (0.392)
sum_25x0_true-unreach-call.c [OCRS][without] 2 PASS
0.445 (0.377)
sum_2x3_false-unreach-call.c [OCRS][without] 2 PASS
0.435 (0.386)
sum_2x3_true-unreach-call.c [OCRS][without] 2 PASS
0.405
sum_non_eq_false-unreach-call.c [OCRS][without] 2 PASS
0.476
sum_non_eq_true-unreach-call.c [OCRS][without] 2 PASS
0.476 (0.413)
sum_non_false-unreach-call.c [OCRS][without] 2 PASS
0.424
sum_non_true-unreach-call.c [OCRS][without] 2 PASS
0.432 (0.379)
Total Below Time = 368.093 (was 5991.683)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/rec-sv-benchmarks/rec-loop-lit
rec-afnp2014_true-unreach-call.c [OCRS][without] 2 PASS
0.416
rec-bhmr2007_true-unreach-call.c [OCRS][without] 2 PASS
0.423
rec-cggmp2005b_true-unreach-call.c [OCRS][without] 3 PASS
0.803
rec-cggmp2005_true-unreach-call.c [OCRS][without] 2 PASS
0.375 (0.333)
rec-cggmp2005_variant_true-unreach-call.c [OCRS][without] 2 PASS
0.347
rec-css2003_true-unreach-call.c [OCRS][without] 2 PASS
0.663
rec-ddlm2013_true-unreach-call.c [OCRS][without] EXCEPTION
0.351 (0.461)
rec-gcnr2008_false-unreach-call.c [OCRS][without] 2 PASS
0.983 (0.873)
rec-gj2007b_true-unreach-call.c [OCRS][without] 2 FAIL
FAIL
0.412
rec-gj2007_true-unreach-call.c [OCRS][without] 2 FAIL
0.404
rec-gr2006_true-unreach-call.c [OCRS][without] 2 FAIL
0.484 (0.39)
rec-gsv2008_true-unreach-call.c [OCRS][without] 2 PASS
0.357
rec-hhk2008_true-unreach-call.c [OCRS][without] 2 PASS
0.364
rec-jm2006_true-unreach-call.c [OCRS][without] 2 PASS
0.4 (0.356)
rec-jm2006_variant_true-unreach-call.c [OCRS][without] 2 PASS
0.397
rec-mcmillan2006_true-unreach-call.c [OCRS][without] 2 FAIL
0.439
Total Below Time = 7.618 (was 7.213)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/rec-sv-benchmarks/rec-loop-new
rec-count_by_1_true-unreach-call.c [OCRS][without] 2 PASS
0.338
rec-count_by_1_variant_true-unreach-call.c [OCRS][without] 2 PASS
0.359
rec-count_by_2_true-unreach-call.c [OCRS][without] 2 PASS
0.358
rec-count_by_k_true-unreach-call.c [OCRS][without] 2 PASS
0.39
rec-count_by_nondet_true-unreach-call.c [OCRS][without] 2 FAIL
0.426 (0.376)
rec-gauss_sum_true-unreach-call.c [OCRS][without] 2 PASS
0.371
rec-half_true-unreach-call.c [OCRS][without] EXCEPTION
0.342 (0.392)
rec-nested_true-unreach-call.c [OCRS][without] 3 PASS
0.629
Total Below Time = 3.213 (was 3.127)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/recurrences
cubic_polynomial.c [OCRS][without] 2 PASS
0.546
edit_distance_bottom_up.c [OCRS][without] 3 TIMEOUT
300.009
edit_distance_top_down.c [OCRS][without] 3 FAIL
53.894
log_log_n_solution.c [OCRS][without] 3 FAIL
0.428 (0.934)
log_n_solution.c [OCRS][without] 2 FAIL
0.472
multivariate_1.c [OCRS][without] TIMEOUT
300.007
multivariate_higher_order.c [OCRS][without] 7 FAIL
FAIL
FAIL
28.083 (24.97)
n_cubed_solution.c [OCRS][without] EXCEPTION
0.314
n_log_n_solution.c [OCRS][without] TIMEOUT
300.04 (1.902)
n_squared_two_base_case_even.c [OCRS][without] 2 PASS
0.394
n_squared_two_base_case_odd.c [OCRS][without] 2 PASS
0.4
pascals_dynamic.c [OCRS][without] 3 FAIL
20.524
pascals_iterative.c [OCRS][without] 1 FAIL
5.423 (6.348)
pascals_recursive.c [OCRS][without] TIMEOUT
300.006
squared_solution.c [OCRS][without] TIMEOUT
300.129 (10.489)
two_n_even.c [OCRS][without] 2 PASS
0.396
two_n_odd.c [OCRS][without] 2 PASS
0.426 (0.383)
two_to_n_over_two_even.c [OCRS][without] TIMEOUT
300.024 (45.052)
two_to_n_over_two_odd.c [OCRS][without] TIMEOUT
300.025 (45.52)
two_to_n_squared.c [OCRS][without] 8 TIMEOUT
300.006 (139.068)
two_to_two_to_n.c [OCRS][without] 7 FAIL
1.691
Total Below Time = 2513.237 (was 1249.788)
/bat0/stac/Code/Ark2-Sandbox/WALi-OpenNWA/Examples/cprover/tests/exponential
adding_and_mult.c [OCRS][without] 1 PASS
0.335
exp_add_loop_rec.c [OCRS][without] 1 FAIL
0.366
exp_add_loop_variable.c [OCRS][without] 1 PASS
0.363
exp_with_linear_inner_loop.c [OCRS][without] 1 FAIL
0.419
linear_two_to_n.c [OCRS][without] 8 FAIL
1.592 (1.436)
loop_five_to_n.c [OCRS][without] 1 FAIL
0.356
non_loop_five_to_n.c [OCRS][without] 8 FAIL
10.456 (9.162)
power_log.c [OCRS][without] 1 PASS
0.434 (0.378)
power_log_modified.c [OCRS][without] 1 PASS
0.489 (0.394)
simple_exponential.c [OCRS][without] 1 PASS
0.35
simple_exponential_for.c [OCRS][without] 1 PASS
0.362
two_to_n_minus_n.c [OCRS][without] 8 FAIL
1.795
two_to_n_minus_n_loop.c [OCRS][without] 8 FAIL
2.756
Total Below Time = 20.073 (was 18.215)

>25% better
25%-10% better
No significant change
10%-25% worse
>25% worse