0000059: instrumented code is 20times slower - Mantis

Mantis Bugtracker

Einfache Problemansicht anzeigen [ Zu Notizen wechseln ]

[ erweiterte Anzeige ] [ Problem-Historie ] [ Drucken ]

ID

Kategorie

Auswirkung

Reproduzierbar

Meldungsdatum

Letzte Aktualisierung

0000059

[Bisect] general

Unschönheit

immer

2010-07-06 15:54

2011-02-12 14:45

Reporter

art1

Anzeigestatus

öffentlich

Bearbeitung durch

xclerc

Priorität

normal

Lösung

erledigt

Status

erledigt

Produktversion

1.0

Zusammenfassung

0000059: instrumented code is 20times slower

Beschreibung

Hello,

The instrumented testcases runs 20 times slower then without bisect instrumentation. I tried this to test the simple median-function:

(** median of a list
  * @param lst list of ints
  * @return median
  *)
let int_median (lst: int list) =
  match lst with
      [] -> 0
    | _ ->
        begin
          let ar = Array.of_list lst in
          let len = (Array.length ar) in
          let pivot = (len-1)/2 in
            Array.sort Helper.int_cmp ar;
            if len mod 2 == 0 then
              round_int (float_of_int (ar.(pivot) + ar.(pivot+1)) /. 2.0)
            else
              ar.(pivot)
        end
;;

My testcase was:
let testplan_int_median =
  "math (int_median) " >::: [
    "empty " >:: (fun () -> !==! (int_median []) 0);
    "simple " >:: (fun () -> !==! (int_median [1]) 1);
    "2 positives " >:: (fun () -> !==! (int_median (1::3::[])) 2);
    "2 same " >:: (fun () -> !==! (int_median (1::1::[])) 1);
    "2 negatives " >:: (fun () -> !==! (int_median (~-1::-3::[])) ~-2);
    "2 values " >:: (fun () -> !==! (int_median (~-1::1::[])) 0);
    "3 positives " >:: (fun () -> !==! (int_median (1::3::2::[])) 2);
    "3 negatives " >:: (fun () -> !==! (int_median (~-1::~-3::~-2::[])) ~-2);
    "3 values " >:: (fun () -> !==! (int_median (~-1::1::0::[])) 0);
    "5.000.000 values " >:: (fun () -> !==! (int_median five_mill_int) 1);
  ]
;;

I am using OUnit and the testplan runs 15s on my Athlon64 without using bisect. With bisect switched on, the testplan needs approximately 310s.

Because I have larger testcases that issue makes it hard to run all my tests nightly and to calculate the test coverage.

If you have some useful hints, it would be fine.

Thanks in advance, Bye Andreas

Zusätzliche Information

Tags

Keine Tags zugeordnet.

Angehängte Dateien

Problem-Beziehungen

Notiz
(0000210) xclerc (Administrator) 2010-09-26 14:52	Well, I am afraid that a huge slowdown is unavoidable when using code coverage, as one has roughly to "increment counters everywhere". As far as I can tell, clever profiling approaches (read "sampling") cannot be applied to code coverage. Here a some additional notes: - it would be possible for Bisect to give additional control over the way source code is instrumented (some kind of filter to instrument only some given code parts, or some given code constructs - the currently only available granularity being the file); - it is possible to use the aforementioned file-based granularity to use code coverage for unit testing rather than for integration testing (meaning that at each run only the tested module is instrumented, the remaining modules being "uninstrumented"); - only the tested code need to be instrumented, the testing code (i. e. test framework, and tests encoding) should not be instrumented - this can be particularly costly if your testing code contains loops and/or is random-based. However, I cannot tell if these suggestions can be applied to your settings. I would be glad to read your feedback about the above.

Problem-Historie
Änderungsdatum	Benutzername	Feld	Änderung
2010-07-06 15:54	art1	Neues Problem
2010-09-26 14:52	xclerc	Problemnotiz hinzugefügt: 0000210
2010-09-26 14:52	xclerc	Bearbeitung durch	=> xclerc
2010-09-26 14:52	xclerc	Status	neu => Rückmeldung
2011-02-12 14:45	xclerc	Status	Rückmeldung => erledigt
2011-02-12 14:45	xclerc	Behoben in Version	=> 1.1
2011-02-12 14:45	xclerc	Lösung	offen => erledigt

Mantis 1.1.7[^]

Copyright © 2000 - 2008 Mantis Group