How to Learn Malware Analysis

Malware analysis is the work of taking a suspicious file and answering three questions. What does it do, how do we detect it, and what damage has it already done. That output feeds SOC detection rules, incident response timelines and threat intelligence reports.

It is not the same as penetration testing. Pentesting is about breaking in. This is about taking something apart after it is already inside, usually under time pressure, usually while someone senior is asking when the report will be ready.

• Who hires for it SOC teams, DFIR consultancies, antivirus and EDR vendors, threat intelligence teams, and large banks with in house response teams
• What you need to start a machine with 16 GB RAM if possible, 8 GB at a stretch, and 100 GB of free disk for virtual machines
• What makes it different patience. A single sample can take a full day, and that is normal, not slow

This is the only step you must not improvise. A badly built lab does not just fail, it infects your actual machine and encrypts your actual files.

• Windows analysis VM Windows 10 in VirtualBox or VMware, Defender disabled, automatic updates off, no antivirus. This is where samples run
• FLARE VM a free script from Mandiant that installs the entire Windows analysis toolkit onto that VM in one go
• REMnux a Linux distribution built for malware analysis, used as your second machine for network simulation and static tooling
• Allocate 4 GB RAM and 60 GB disk per VM, more if your host allows

• Host only networking, or an internal network between the Windows VM and REMnux. Never bridged, never NAT with internet access while detonating
• No shared folders, no shared clipboard, no drag and drop while a sample is live
• Disable guest additions network features once setup is done, some malware escapes through them
• Move samples in as password protected archives, the standard password used by researchers is infected

• Take a clean snapshot after setup and after every tool install
• Restore to clean before every new sample, without exception
• Name snapshots clearly, you will be switching between them constantly

You cannot reverse engineer something you do not understand. This is the month that decides whether you finish this roadmap or abandon it at the first assembly screen.

• Processes, threads, handles, and what process injection actually means at the API level
• DLLs, the loader, and why DLL search order matters to attackers
• The registry, run keys, services, scheduled tasks, the standard persistence locations
• Common Win32 APIs worth recognising, CreateProcess, VirtualAlloc, WriteProcessMemory, RegSetValueEx, InternetOpenUrl

• DOS header, NT headers, section table, and what each section usually holds
• Import and export tables, and how imports reveal capability before you run anything
• Entry point, relocations, resources, and where packers hide payloads

• x86 and x64 basics, registers, the stack, calling conventions, common instructions
• Loops, conditionals and function calls as they appear in disassembly
• Enough C to read decompiler output, pointers, structs, string handling

Static analysis means learning everything you can without running the file. It is free, it is safe, and it often answers most of the question before detonation. Squeeze it dry first.

• Hashes, MD5, SHA1 and SHA256, then check them against public intelligence sources
• Real file type, because extensions lie. Check the magic bytes
• Strings, both ASCII and Unicode. Look for URLs, IPs, file paths, registry keys, mutex names, command lines
• Compile timestamp and language hints, though both can be faked

• PEStudio imports, resources, indicators, all in one view. Start here
• Detect It Easy compiler and packer identification
• Entropy high entropy sections usually mean packing, encryption or an embedded payload
• Resources embedded executables, configuration blobs and decoy documents hide here

• Networking APIs plus file writing plus persistence keys suggests a downloader
• Crypto APIs plus file enumeration plus shadow copy deletion suggests ransomware
• Keyboard hooks plus clipboard access plus HTTP POST suggests an infostealer
• Very few imports plus high entropy usually means packed, and the real imports appear only at runtime

Now you run it and watch everything it touches. Behavioural evidence is what SOC teams can actually detect on, so this step produces most of your usable output.

• Procmon file, registry, process and network events. Filter aggressively or you will drown
• Process Hacker live process tree, injected memory regions, loaded modules, strings in memory
• Regshot snapshot the registry before and after, then diff it
• Autoruns reveals persistence the sample created
• Wireshark plus INetSim or FakeNet capture traffic and fake the internet so the sample gets replies without reaching real infrastructure

• Files created, modified and deleted, with full paths
• Registry keys written, especially persistence locations
• Processes spawned, and any process injection into legitimate binaries
• Network indicators, domains, IPs, user agents, request paths, beacon intervals
• Mutexes, because they make excellent detection signatures

• The sample may be checking for a VM, a debugger, a specific language setting or a recent uptime
• It may be waiting. Some families sleep for minutes or hours before acting
• It may need a specific command line argument or an export to be called directly
• Note the evasion itself, that behaviour is intelligence too

Static and dynamic analysis tell you what happened. Reverse engineering tells you why, and answers the questions behaviour cannot, such as what the configuration contains or how the encryption key is derived.

• Ghidra free, powerful, and has a decompiler. The standard starting point in 2026
• IDA Free excellent disassembler, the industry default in many teams
• x64dbg the debugger you will live in for unpacking and runtime inspection
• Scylla or PE-bear for dumping and repairing unpacked binaries

• Find the real entry point, then follow the flow rather than reading top to bottom
• Rename functions and variables as you understand them, your future self depends on it
• Break on interesting APIs instead of stepping through thousands of instructions
• For packed samples, run until the payload is decoded in memory, then dump and fix the imports

• VM detection through registry artefacts, MAC address ranges, driver names and CPU features
• Debugger detection, IsDebuggerPresent, timing checks, exception based tricks
• String obfuscation and API resolution at runtime, so nothing useful appears in a strings dump
• Sleep loops and delayed execution designed to outlast automated sandbox timeouts

Analysis that nobody can act on is a hobby. This step turns your findings into something a SOC can deploy tomorrow morning, and it is the part that gets you hired.

• YARA rules built from stable strings and byte patterns, not from things that change every build
• Sigma rules for behavioural detection from logs, process creation patterns, registry writes, suspicious parent and child chains
• Network indicators domains, IPs, URI patterns and user agents, with a note on how long each is likely to stay valid
• Test every rule against clean files too. A rule that fires on legitimate software is worse than no rule

• Map observed behaviour to MITRE ATT and CK techniques, defence evasion, persistence, credential access, exfiltration
• This is how your report connects to the detections and controls the organisation already has

• Executive summary, three sentences a manager can read
• Sample identification, hashes, file type, size, first seen
• Capability summary, what it does, in order
• Indicators of compromise, grouped by type
• Detection opportunities, the rules you wrote
• Recommended actions, containment and remediation steps

You need real samples, and you need them from places that expect you to be handling them carefully.

• MalwareBazaar free, tagged by family, archives password protected with the standard researcher password
• theZoo and vx-underground curated archives, good for studying known families
• Practical Malware Analysis labs the book's sample set, still the best structured training material in the field
• Public sandbox reports read analyses of samples you have also examined and compare your conclusions with theirs

Based on eight to ten focused hours a week. More time compresses this, but the order stays the same.