The purpose of the thesis is to make an analysis of various methods and techniques that have been implemented in the literature (ideally to also mention and explain the characteristics of the datasets used in the literature)
A relative comparison of the results presented in the literature
To run the implementations of the methods (where these are available obviously) and to also try to run them with input datasets different from those used by each paper, to make our own observations and draw conclusions and what can be done further.
For this purpose, I also attached an excel with the papers that should at least be analyzed, including the repositories with the implementation of the method, where it is available.
A relative comparison of the results presented in the literature
To run the implementations of the methods (where these are available obviously) and to also try to run them with input datasets different from those used by each paper, to make our own observations and draw conclusions and what can be done further.
For this purpose, I also attached an excel with the papers that should at least be analyzed, including the repositories with the implementation of the method, where it is available.
So I need the following corrections:
1. In section 4 the characteristics of the machine that ran the experiments should also be listed, but they are nowhere to be found.
2. In section 4 there should be links (e.g. github) with the codes used
3. In section 4.1 it states `. Our evaluation metrics include precision, recall, F1-score, and true negative rate (TNR).` The only metric we see in the rest of the chapter is f1.
4. It is not at all clear what techniques they have run for. Section 4.4 mentions 7 techniques.
5. Of these, 1-6 are off-topic and can only serve as a baseline approach, but again, the implemented code for them is nowhere to be found.
6. Also techniques are mentioned once in section 4.4 and not mentioned again. The 7th technique mentioned is also the purpose of the thesis, where it states that we implement two advanced methods. Normally as many of the methods should be run on as many datasets as possible and clearly reported in this chapter.
8 It should also be clearly stated which datasets were used for which methods. The sentence `The datasets chosen include HDFS, BGL, Thunderbird, OpenStack, and ADFA , each presenting unique characteristics and challenges for anomaly detection’ is not enough
9 In chapter 5 there is a table, the purpose of which is difficult to understand. I think it shows which technique is more efficient on each dataset, but the metrics it gives don’t make sense.
eg first column
10 Fifth line gives a number and in parentheses below (LogDeep) what does this information mean? Further down in chapter 5 there is the following sentence “Our experiments show that combining new event types and deviating sequence lengths achieves an impressive F1 score of 90.4% on the LogDeep version, with ECVC further enhancing performance” which again does not explain at all what it says this. How it is set up and how it relates to “Effective Simple Methods”
11 Sixth line Effective Advanced Methods has two techniques, just below the line Advanced methods performance says >94%. What does that mean? that one technique gave a result of 94 and the other above? how much more? maybe 100?
12 Based on the professor’s feedback, this table should be broken for each experiment that was run separately, or at least all the techniques that have been run should be entered in the lines and the f1 score for each dataset should be entered in the cells
13 As for the rest of chapter 5 and 6, almost all observations are generally written and need enrichment
2. In section 4 there should be links (e.g. github) with the codes used
3. In section 4.1 it states `. Our evaluation metrics include precision, recall, F1-score, and true negative rate (TNR).` The only metric we see in the rest of the chapter is f1.
4. It is not at all clear what techniques they have run for. Section 4.4 mentions 7 techniques.
5. Of these, 1-6 are off-topic and can only serve as a baseline approach, but again, the implemented code for them is nowhere to be found.
6. Also techniques are mentioned once in section 4.4 and not mentioned again. The 7th technique mentioned is also the purpose of the thesis, where it states that we implement two advanced methods. Normally as many of the methods should be run on as many datasets as possible and clearly reported in this chapter.
8 It should also be clearly stated which datasets were used for which methods. The sentence `The datasets chosen include HDFS, BGL, Thunderbird, OpenStack, and ADFA , each presenting unique characteristics and challenges for anomaly detection’ is not enough
9 In chapter 5 there is a table, the purpose of which is difficult to understand. I think it shows which technique is more efficient on each dataset, but the metrics it gives don’t make sense.
eg first column
10 Fifth line gives a number and in parentheses below (LogDeep) what does this information mean? Further down in chapter 5 there is the following sentence “Our experiments show that combining new event types and deviating sequence lengths achieves an impressive F1 score of 90.4% on the LogDeep version, with ECVC further enhancing performance” which again does not explain at all what it says this. How it is set up and how it relates to “Effective Simple Methods”
11 Sixth line Effective Advanced Methods has two techniques, just below the line Advanced methods performance says >94%. What does that mean? that one technique gave a result of 94 and the other above? how much more? maybe 100?
12 Based on the professor’s feedback, this table should be broken for each experiment that was run separately, or at least all the techniques that have been run should be entered in the lines and the f1 score for each dataset should be entered in the cells
13 As for the rest of chapter 5 and 6, almost all observations are generally written and need enrichment
Finally, i need the code file with benchmarks tests, in order to know the characteristics of the machine as well as the time it took each algorithm to run.