THREATBYTESTM
All systems operational

Knowledge

SI malware detection

How machine learning classifies malicious software from static and dynamic features, where it fails, and how to evaluate it honestly.

What it is01

SI malware detection trains models to decide whether a file is malicious, and often which family or type it belongs to, from features rather than exact signatures. Where a signature matches one known file, a trained model can generalise to variants it has never seen.

Feature families02
  • Static structure: file headers, section sizes and entropy, imported libraries and functions, embedded strings, certificates.
  • Byte and opcode patterns: n-grams or learned embeddings over raw bytes or disassembled instructions.
  • Dynamic behaviour: processes, file and registry changes, and network activity recorded in a sandbox.
  • Context: packer detection, compiler fingerprints, and similarity to known families.
Model types03

Tree ensembles and logistic regression remain strong baselines on engineered features. Convolutional and transformer models learn directly from bytes or instruction sequences. Graph models represent call graphs and behaviour traces. The research question is less which model scores highest and more which combination generalises to new, adaptive malware.

Common pitfalls04
  • Temporal leakage: training on samples newer than the test set inflates scores; splits must respect first-seen dates.
  • Family leakage: variants of one family in both train and test sets reward memorisation.
  • Label noise: vendor labels disagree; consensus labelling and label-quality checks are needed.
  • Concept drift: accuracy decays as malware evolves; models must be re-evaluated over time.
Evaluating honestly05

Report false positive rate at operationally realistic thresholds, not only accuracy. Use time-aware splits, per-family results, and precision-recall curves on imbalanced data, and compare against simple baselines and established tools so gains are meaningful.